How to keep "Thumbnails" storage under control

How to keep "Thumbnails" storage under control - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: General Support (https://forum.kodi.tv/forumdisplay.php?fid=111)
+---- Forum: OS independent / Other (https://forum.kodi.tv/forumdisplay.php?fid=228)
+---- Thread: How to keep "Thumbnails" storage under control (/showthread.php?tid=346182)

How to keep "Thumbnails" storage under control - jcea - 2019-08-11

My "Thumbnails" directory is eating 8GB and growing. How can I manage it?. I can write an addon, no problem with that.

I wonder if I can just delete random files in "Thumbnails" and Kodi will download the media again if needed, like a cache. Would kodi download again, show an empty/broken image or just crash?.

What is the "best practices" to manage "Thumbnails"?.

I could write an addon to check kodi about art files via JSON and delete them via API. More complex but can be done. I just would like to do the lest work possible Smile

.

What is the best practices?. Is there any addon to take care of this?

Thanks!.

RE: How to keep "Thumbnails" storage under control - Karellen - 2019-08-11

There is no real best practice. It is what it is. If you want artwork displayed in Kodi, then it needs to be cached.

Of course if you regularly scrape, then delete titles a build up of stale artwork will occur. Read here for some tips... https://kodi.wiki/view/Artwork#Managing_Artwork

(2019-08-11, 06:18)jcea Wrote: Would kodi download again, show an empty/broken image or just crash?.

https://kodi.wiki/view/Artwork/Cache#Advanced:_delete_textures13.db

RE: How to keep "Thumbnails" storage under control - jcea - 2019-08-11

(2019-08-11, 08:44)Karellen Wrote: IThere is no real best practice. It is what it is. If you want artwork displayed in Kodi, then it needs to be cached.

Of course if you regularly scrape, then delete titles a build up of stale artwork will occur. Read here for some tips... https://kodi.wiki/view/Artwork#Managing_Artwork

(2019-08-11, 06:18)jcea Wrote: Would kodi download again, show an empty/broken image or just crash?.
https://kodi.wiki/view/Artwork/Cache#Advanced:_delete_textures13.db

I see. Looks like it is not enough to delete the file in thumbnails, but I need to delete also the entry in "textures13.db". Doable. That database even includes a "last used" field than can be used to implement a LRU cache.

Do you think that Kodi team would be interested in a "keep cached art files under X Gigabytes" addon?.

Looks like "Texture Cache Maintenance utility" can not do that and it requires SSH and command line skills. I rather prefer an automated addon to take care of this, and I think that other users would find it useful.

When you have about 25.000 media files in the library, many artwork is never displayed at all, ever. And Kodi would download it again if needed.

What do you think?.

RE: How to keep "Thumbnails" storage under control - Klojum - 2019-08-11

(2019-08-11, 06:18)jcea Wrote: What is the "best practices" to manage "Thumbnails"?.

Don't enable the download of actor thumbnails, as those will be downloaded/cached multiple times, especially for tv series.

(2019-08-11, 16:46)jcea Wrote: When you have about 25.000 media files in the library, many artwork is never displayed at all, ever. And Kodi would download it again if needed.

There is currently no automated tool to maintain Kodi's cache. But if you have a large media collection of 25.000 items, a big cache is inevitable. If you have several terabytes of media files, 8GB of cache is not that large in the overall picture.

RE: How to keep "Thumbnails" storage under control - jcea - 2019-08-11

(2019-08-11, 18:17)Klojum Wrote:
(2019-08-11, 06:18)jcea Wrote: What is the "best practices" to manage "Thumbnails"?.

Don't enable the download of actor thumbnails, as those will be downloaded/cached multiple times, especially for tv series.

(2019-08-11, 16:46)jcea Wrote: When you have about 25.000 media files in the library, many artwork is never displayed at all, ever. And Kodi would download it again if needed.

There is currently no automated tool to maintain Kodi's cache. But if you have a large media collection of 25.000 items, a big cache is inevitable. If you have several terabytes of media files, 8GB of cache is not that large in the overall picture.

I agree, but a cache is a cache. Would be nice if it could have a configured max size. In practice, most fanart is never showed at all, or only first time, when added to the library.

Real world example: my OSMC is live since 2015. I have around 8.500 films and 14.000 TV shows:

Code:
root@osmcpi:/home/osmc/.kodi/userdata/Database# sqlite3 Textures13.db

SQLite version 3.28.0 2019-04-16 19:49:53

Enter ".help" for usage hints.

sqlite> select count(*) from sizes where usecount=0;

0

sqlite> select count(*) from sizes where usecount=1;

92488

sqlite> select count(*) from sizes where usecount>1;

42990

2/3 of my artwork was displayed ONLY once.

Code:
sqlite> select count(*) from sizes where lastusetime<'2019-01-01';

86811

sqlite> select count(*) from sizes where lastusetime<'2018-08-11';

74202

sqlite> select count(*) from sizes where lastusetime<'2017-08-11';

44523

64% of my artwork was NOT displayed AT ALL in 2019. 54% was not displayed in the last year. 33% of the artwork was not displayed at all in the last two years.

Code:
sqlite> select count(*) from sizes where lastusetime>'2017-08-11' and usecount>1;

29666

In the last TWO years, only 22% of all my artwork was displayed more than once. The 78% of my thumbnail cache is sitting there, unused at all after two years.

Currently my thumbnails directory is 8GB in size. Looks like it is big enough for me, considering that it is a cache and that any artwork not found there will be downloaded again.

My raspberry is running on a microSD, while the videos live in a remote HTTP server. I see the point of an addon to keep thumbnails cache size under control.

RE: How to keep "Thumbnails" storage under control - pkscout - 2019-08-11

You might take a look at the Texture Cache Maintenance utility.

https://kodi.wiki/view/Texture_Cache_Maintenance_utility

It's a python command line tool, so you can automate it. I have this setup in a cron job to run every day and prune any cached artwork for shows/movies that I deleted.

RE: How to keep "Thumbnails" storage under control - jcea - 2019-08-11

(2019-08-11, 19:41)pkscout Wrote: You might take a look at the Texture Cache Maintenance utility.

https://kodi.wiki/view/Texture_Cache_Maintenance_utility

It's a python command line tool, so you can automate it. I have this setup in a cron job to run every day and prune any cached artwork for shows/movies that I deleted.

Thanks for the suggestion, but that is not the issue that I am having. I have little stale artwork. My issue as that 78% of my cached artwork has been not displayed at all in two years and, in my opinion, microSD space is being wasted keeping it around, moreover when it will be downloaded again if needed in the future.

Even if the command line tool had the option to clean up "not recently used (years)" artwork when the thumbnail directory is over X gigabytes, that would solve the problem for me, but not for other Kodi users. I think this could be an addon in the official Kodi repository. I would configure it to allow up to "14 GB", it would take another four years or more to fill up and then it could automatically delete a handful of files to keep that size. Using LRU, that would mean I am deleting artwork not used for the last 8-9 years...

RE: How to keep "Thumbnails" storage under control - jmh2002 - 2019-08-12

My 'best practice' is to use KCleaner (https://forum.kodi.tv/showthread.php?tid=307919&highlight=kcleaner) and set it to run on a schedule. Everything then becomes automated.

RE: How to keep "Thumbnails" storage under control - Karellen - 2019-08-12

(2019-08-11, 20:07)jcea Wrote: I would configure it to allow up to "14 GB", it would take another four years or more to fill up and then it could automatically delete a handful of files to keep that size. Using LRU, that would mean I am deleting artwork not used for the last 8-9 years...

This sounds like a great idea for an cleaner add-on. In your first post you state you can write an add-on. Is this something you would go ahead with?

RE: How to keep "Thumbnails" storage under control - jmh2002 - 2019-08-12

The KCleaner developer is pretty responsive and you might be able to encourage him to add this option. I could see how it could be useful for other users too.

Just bear in mind that I think he is Europe based and it is currently summer holidays there.