+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: General Support (https://forum.kodi.tv/forumdisplay.php?fid=111)
+---- Forum: OS independent / Other (https://forum.kodi.tv/forumdisplay.php?fid=228)
+---- Thread: Multiple Kodi clients with a MySql database and artwork link rot (/showthread.php?tid=366652)
Multiple Kodi clients with a MySql database and artwork link rot - MrCheshire - 2022-01-22
I'm a very happy Kodi user and have been for several years, but there is one aspect of its behavior that I find frustration, and I'd like to understand it better.
I use a MySql database for my Kodi metadata and a NAS to hold my media. It works great, and it's so easy for me to spin up a new client, drop in a copy of my advanced settings xml file, and bam, I can access my whole media library.
However, I am noticing that when I spin up a new client, I am finding that a significant amount of the movie posters are not present. This is not a deal-breaker of an issue, since I can still read the metadata bout the movies and watch my files. But it does significantly degrade the experience of having a media player like Kodi in a way that I would like to improve, if I can.
I believe I understand the cause of this problem. The poster art itself is not stored in the MySql database, it's stored in the cache for each individual client. The database just stores a URL to where the file was downloaded from. When I spin up a new client, it presumably uses that URL to fetch the file and store it locally. However, if the content provider has moved or changed the file since I originally scraped the data, the link is now a 404 and there's nothing for the new client to grab.
I posted about this in the kodi subreddit with a bit of additional information, and got some comments with proposed suggestions, but none that seemed to really be the answer that I'm looking for. I also saw quite a few other people posting that they had run into this issue and that they also were not aware of a satisfactory solution.
The only solution that I know of is to essentially rescrape each movie or show that has this problem, but that is a manual process for each affected piece of media, it doesn't do anything to prevent this issue coming up again in the future, and since it basically deletes and re-adds the media to the database, it removes any tags or watched status in the process.
My first question is - is there a better, officially-supported option for dealing with this problem? Either in terms of a way to run Kodi that makes it more feasible to share these images between clients, or a way to fix this issue that is more efficient and less disruptive, either currently built into Kodi or by using an add-on? I did some looking, but I will absolutely admit that this functionality may already exist and I was just unable to find it.
If something like this doesn't exist, has there been any discussion of improving this issue that other people are aware of? Either in this forum or in the feature requests section of the github repo? It seems to me like Kodi could be able to do a partial scrape of information to replace artwork without disrupted other aspects of the metadata, and it also could be able to flag pieces of artwork in the database when it tries to request them and gets a 404 response, indicating that artwork probably doesn't live at that URL anymore.
If nobody else has started doing this work, I've considered diving into it myself. I am pretty confident I could put together something either in a standalone script, Kodi add-on, or PR, assuming my understanding of the issue and how Kodi interacts with artwork is at least in the general vicinity of being correct.
Apologies if I missed anything obvious or if I wasn't looking in the right places for the information I'm requesting, I'm new to Kodi forums, but any guidance, suggestions, or assistance would be greatly appreciated!
RE: Multiple Kodi clients with a MySql database and artwork link rot - Karellen - 2022-01-22
(2022-01-22, 22:18)MrCheshire Wrote: The poster art itself is not stored in the MySql database, it's stored in the cache for each individual client. The database just stores a URL to where the file was downloaded from. When I spin up a new client, it presumably uses that URL to fetch the file and store it locally.Correct
(2022-01-22, 22:18)MrCheshire Wrote: if the content provider has moved or changed the file since I originally scraped the data, the link is now a 404 and there's nothing for the new client to grab.Yes, this is a problem. The information sites purge artwork on an ad-hoc basis. Early last year TMDB conducted a mass purge of artwork where tens of thousands of images were deleted. This affected Kodi users widely. In addition, the mods routinely delete artwork that is of poor quality, duplicated or does not meet their criteria for other reasons. Some of these links may have already been caught up in users scraping.
(2022-01-22, 22:18)MrCheshire Wrote: My first question is - is there a better, officially-supported option for dealing with this problem?Yes, save your artwork locally. You will then be immune to what the information providers do on their site. Artwork Dump is the best solution. It downloads the original artwork during the scraping process.
(2022-01-22, 22:18)MrCheshire Wrote: If nobody else has started doing this work, I've considered diving into it myself.As far as I am aware, nobody is working on a project like this. The most obvious solution is an Artwork only scraper.
Currently your only fix for the missing artwork is to rescrape the affected titles.
RE: Multiple Kodi clients with a MySql database and artwork link rot - black_eagle - 2022-01-22
Or, use a third party media manager that will download artwork and store it locally (along with an nfo file) and then scrape your stuff using 'local information only'. This is what I do and all the art appears on all my clients. Just need to use the third party MM on the one client (in my case a PC) to download and manage all the art and nfo files.
It's an extra step in scraping/importing new stuff but it always works and gives you full control over the art etc. Takes a bit of setting up and getting used to but once you start doing it, I doubt you will see any need to go back. Certainly, I don't.
RE: Multiple Kodi clients with a MySql database and artwork link rot - MrCheshire - 2022-01-23
(2022-01-22, 22:39)Karellen Wrote: Yes, save your artwork locally. You will then be immune to what the information providers do on their site. Artwork Dump is the best solution. It downloads the original artwork during the scraping process.Thanks for the quick reply! I'll play with Artwork Dump a bit and see how it goes. My first trial run didn't quite work, but it looks like an NFS permissions issue between my client device and my NAS, so I'll dig into that deeper.
RE: Multiple Kodi clients with a MySql database and artwork link rot - MrCheshire - 2022-01-23
(2022-01-22, 22:52)black_eagle Wrote: Or, use a third party media manager that will download artwork and store it locally (along with an nfo file) and then scrape your stuff using 'local information only'. This is what I do and all the art appears on all my clients. Just need to use the third party MM on the one client (in my case a PC) to download and manage all the art and nfo files.
It's an extra step in scraping/importing new stuff but it always works and gives you full control over the art etc. Takes a bit of setting up and getting used to but once you start doing it, I doubt you will see any need to go back. Certainly, I don't.
Thanks for this suggestion, too. It definitely seems like it would be a significant change to my workflow and would require going back through everything I've added to my library over the last couple of years, but I have heard this suggestion from others as well, so it may be worth looking into.
RE: Multiple Kodi clients with a MySql database and artwork link rot - gibxxi - 2022-01-23
...Or copy the Textures13.db and Thumbnails folder from one of your existing (set-up) clients using the same MySQL / MariaDB databases, to your new client.
Dan / Gib.
RE: Multiple Kodi clients with a MySql database and artwork link rot - black_eagle - 2022-01-23
(2022-01-23, 03:21)gibxxi Wrote: ...Or copy the Textures13.db and Thumbnails folder from one of your existing (set-up) clients using the same MySQL / MariaDB databases, to your new client.
That doesn't stop link rot though. Using a 3rd party media manager and storing your art locally does. Plus, if you want to update any art, it's easy to do so in the MM and then you can use "choose art" in Kodi to update it in your db. Do that on one client and it'll auto propagate to all of them.
I've been doing it this way for years simply because it gives me total control of what Kodi will display, in terms of both art and textual information, before anything is even added. That's not to say that Kodi's scrapers do a bad job because they don't. I'd still rather choose the art etc myself though. Bit OCD in that respect!
RE: Multiple Kodi clients with a MySql database and artwork link rot - jmgibson1981 - 2022-01-24
+1 I always seed a new Kodi setup on my lan with my master Thumbnails / Textures.db file and hasn't let me down yet. In my case it's due to a limited bandwidth availability and I just don't want to wait for hours to load my whole thumbnails folder and such. This way, it's instant on load.
Just for the record on the OP. I can't help but wonder. You aren't trying to share a Thumbnails folder are you? That won't work. What you describe could be that. Needs to be a separate one for each install / location. They can all be on the server with path substitution but they can't all be shared from the same folder.
RE: Multiple Kodi clients with a MySql database and artwork link rot - gibxxi - 2022-01-30
Surely Artwork Dump (With the right settings) will combat link rot?
As I understand it, it can regularly update artwork for new and old items on a regular basis, even daily should you require it. I'm just as OCD as you BlackEagle, but I don't have time to sit at a PC manually trawling through huge lists of movies / TV shows just to scrape them, when there are tools in Kodi that can do the same thing without user intervention.
I don't see the need for 3rd party scraping apps any more. Kodi's internal scrapers are all accessing the same sources as these 3rd-party tools anyway, so as long as your folder hierarchy and file naming is what Kodi expects, and you take due dilligence to remember to do NFO exports after a library update on the 1st machine / primary machine (after checking the scraped content, of course), I rely on that. Haven't used a 3rd Party app for scraping in over 2 years, and my library is where I want / need it to be.
Each to their own and all that though.
Dan / Gib.
RE: Multiple Kodi clients with a MySql database and artwork link rot - black_eagle - 2022-01-30
(2022-01-30, 19:13)gibxxi Wrote: Surely Artwork Dump (With the right settings) will combat link rot?
Dunno, I don't use it!
(2022-01-30, 19:13)gibxxi Wrote: I don't have time to sit at a PC manually trawling through huge lists of movies / TV shows just to scrape them
No, neither do I. I don't need to though because I don't bulk add stuff (except maybe a season of a TV series that I have bought). Everything else is already added, so adding one more film is a two minute job. It's just as easy to run a 3rd party manager, grab the required art, make any edits I want and then tell Kodi to update it's contents (it's always running) as it is to do it inside Kodi.
Indeed, the scrapers have come a long way but they're still not going to pick up my custom artwork like these
Either artwork dump or a 3rd party manager can fix link rot but as with most things Kodi, users have a choice as to which road to go down and decide which option offers them the experience they want.
RE: Multiple Kodi clients with a MySql database and artwork link rot - Karellen - 2022-01-30
(2022-01-30, 19:13)gibxxi Wrote: Surely Artwork Dump (With the right settings) will combat link rot?Yes and no, depending on what you are expecting.
All Artwork Dump does is download the original artwork during the scraping process. So link rot is solved because you have the original artwork locally, so you don't need the link.
It does not scrape new artwork links, so no links will be replaced.