[Milhouse]

Updated to version 0.1.5.

Added "p" option (prune) to identify and display cached files that are not present in the media library, eg. image/thumbnail previews that accumulate in the cache over time, or deleted movies/tv shows/music that remain in the texture cache even after cleaning the database. Option "P" will automatically remove these unnecessary files.

This version also significantly improves the performance of option r/R, which identifies "orphaned" thumbnail files (ie. image files not referenced in the texture cache db)

[Milhouse]

Update to version 0.1.6.

Added additional support for season-all-fanart.jpg and season-all-banner.jpg images.

[charrua]

Thanks for the great script, it's very useful.

Running on Win8 with Python 2.7.3, XBMC 13.0a1 using a remote MySQL DB, I'm having an error with the c/C option, it downloads the first image fine, but then before getting the second image to download it returns the following error:

Code:

texturecache.py C movies "30 Days of Night"
Querying CHYCH-HTPC001:9090 for all Movies...
Movie: 30 Days of Night                          [ + fanart     ]Traceback (most recent call last):
  File "D:\programas\multimedia\XBMC\scripts\texturecache.py", line 1341, in <module>
    main(sys.argv[1:])
  File "D:\programas\multimedia\XBMC\scripts\texturecache.py", line 1306, in main
    libraryQuery("cache", argv[1], argv[2], force=True)
  File "D:\programas\multimedia\XBMC\scripts\texturecache.py", line 355, in libraryQuery
    libraryAllMovies(action, filter, force, extraFields, rescan)
  File "D:\programas\multimedia\XBMC\scripts\texturecache.py", line 482, in libraryAllMovies
    for artwork in art: loadImage(force, artwork, art[artwork], imgErrors)
  File "D:\programas\multimedia\XBMC\scripts\texturecache.py", line 786, in loadImage
    URL = getImageURL(filename)
  File "D:\programas\multimedia\XBMC\scripts\texturecache.py", line 763, in getImageURL
    data = sendJSON(REQUEST, "1")
  File "D:\programas\multimedia\XBMC\scripts\texturecache.py", line 271, in sendJSON
    return json.loads(web.getresponse().read())
  File "C:\Python\lib\httplib.py", line 1030, in getresponse
    response.begin()
  File "C:\Python\lib\httplib.py", line 407, in begin
    version, status, reason = self._read_status()
  File "C:\Python\lib\httplib.py", line 365, in _read_status
    line = self.fp.readline()
  File "C:\Python\lib\socket.py", line 430, in readline
    data = recv(1)
socket.error: [Errno 10053]

Any ideas to solve this problem?

[popcornmix]

Version 0.1.2 now up... when performing a "qa" check, only movies or tvshow episodes added during the previous 30 days will be considered. Increase/reduce this period by specifying a value (in days) for qaperiod in the properties file, eg. "qaperiod=9999" would effectively restore v0.1.1 behaviour which didn't restrict by period.

Also added additional qa tests: rating and mpaa certification (movies), plot/fanart/banner/poster (tv show) and rating (tv show episode).

Seemed to work great detecting missing information.
The question is, is it possible to force a refresh (i.e. download new plot/rating/thumbnail) on the episodes with missing items?

[Milhouse]

Any tips to solve this problem?

Hi yes, I've just been debugging this with another forum member. Not sure why it happens, as I don't get it on either Ubuntu 12.10 or Raspberry Pi, but it seems necessary to close the web server connection between image requests.

Can you try downloading the latest version (0.1.9):

Code:

rm ./texturecache.py
wget http://www.nmacleod.com/public/texturecache.py && chmod +x ./texturecache.py

and then add "webserver.singleshot = yes" to the properties file texturecache.cfg

[charrua]

Any tips to solve this problem?

Hi yes, I've just been debugging this with another forum member. Not sure why it happens, as I don't get it on either Ubuntu 12.10 or Raspberry Pi, but it seems necessary to close the web server connection between image requests.
Can you try downloading the latest version (0.1.9):

Code:

rm ./texturecache.py
wget http://www.nmacleod.com/public/texturecache.py && chmod +x ./texturecache.py

and then add "webserver.singleshot = yes" to the properties file texturecache.cfg

It's fixed, thanks a bunch for the quick answer and solution!

[Milhouse]

Seemed to work great detecting missing information.
The question is, is it possible to force a refresh (i.e. download new plot/rating/thumbnail) on the episodes with missing items?

Yes, it's possible, but not sure how effective it will be as it depends on the XBMC scraper replacing the missing information - from my experience defective meta-data can be incredibly "sticky"...

I've added a "qax" option (it's in latest version 0.1.9), and it's experimental, but if you run it (eg. ./texturecache.py qax tvshows or ./texturecache.py qax tvshows "big bang", it also works for movies) then it will initiate a library rescan for those episode directories (eg. Season 5, Season 6 etc.) containing episodes that fail to pass QA - let me know how you get on...

[Milhouse]

It's fixed, thanks a bunch for the quick answer and solution!

I've just updated the latest version so that it shouldn't require the extra property - if it experiences a web connection problem it will switch to "single shot" mode, can you test this and let me know if it works (you'll need to remove the webserver.singleshot property from the configuration file first).

[charrua]

I've just updated the latest version so that it shouldn't require the extra property - if it experiences a web connection problem it will switch to "single shot" mode, can you test this and let me know if it works (you'll need to remove the webserver.singleshot property from the configuration file first).

The new version is running perfectly (at least in my environment) without the 'webserver.singleshot' property in the config file.
I'm even running two instances at once without any problem.
Great work!

[Milhouse]

Thanks for confirming!

[jmarshall]

MilhouseVH: mind detailing the queries you use on Textures13.db and what you'd thus need exposed the exact same functionality through JSON-RPC?

Cheers,
Jonathan

[Milhouse]

MilhouseVH: mind detailing the queries you use on Textures13.db and what you'd thus need exposed the exact same functionality through JSON-RPC?

Cheers,
Jonathan

The main query to extract from Textures13.db is:

Code:

SELECT t.id, t.cachedurl, t.lasthashcheck, t.url, s.height, s.width, s.usecount, s.lastusetime FROM texture t JOIN sizes s ON (t.id = s.idtexture)

with an optional filter (WHERE) when searching.

I'd also want the ability to delete rows (by id), and also remove the underlying file system object.

In addition, I walk the files in Thumbnails folder, so being able to get a list of all the files in Thumbnails would be nice.

So in summary:

1) Method to search the texture table (joined with sizes table) and return all rows and columns, or a filtered subset
2) Method to delete rows from database tables (using texture.id), with optional deletion of associated filesystem (cachedurl) object
3) Method to get list of all files present in the Thumbnails folder
4) Method to delete files that are in the Thumbnails folder

[ctawn]

I tried to run this on my ATV2 and got this error:

Code:

ATV2:/bin root# ./texturecache.py c
  File "./texturecache.py", line 1209
    for action in {"Skipped", "Deleted", "Error", "Updated" }:
                            ^
SyntaxError: invalid syntax

I installed it in /bin/ and I first installed Python 2.5.1 with "apt-get install python".

What am I doing wrong here?

[Milhouse]

What am I doing wrong here?

Just to make sure there wasn't a problem with versions, can you remove texturecache.py and download it again. I've only ever tested this on Python 2.7.3, and the current version runs fine, and unfortunately I don't have Python 2.5.1 to test with...

EDIT: OK, thought I'd try this on my old OpenELEC 1.0 x86 box which is running Python 2.6.6 and have reproduced - working on a fix...

[Milhouse]

@ctawn: new version (0.2.2) uploaded, working with Python 2.6.6 which had the same error you had with 2.5.1, so hopefully it's now fixed. Not sure if I can support 2.5.1 going forward but we'll see (doesn't the ATV2 have a more recent version of Python?)

Also in this version, webserver authentication is now supported - add webserver.username and webserver.password to the properties file.