Music Library and Thumbnail hashing

Music Library and Thumbnail hashing - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: General Support (https://forum.kodi.tv/forumdisplay.php?fid=111)
+---- Forum: OS independent / Other (https://forum.kodi.tv/forumdisplay.php?fid=228)
+---- Thread: Music Library and Thumbnail hashing (/showthread.php?tid=51744)

Music Library and Thumbnail hashing - KeithB - 2009-05-27

It's been a while since I've been here but this one is driving me around the bend! As a bit of background, I'm knocking up a couple of scripts to do a couple of things. Firstly allow my to merge in thumbnails either from a file or direct from the web when XBMC doesn't pick them up (there might be scripts to do that but I've not seen them). Secondly, yank those thumbnails back out, encode them into the mp3s in the DB for shovelling into an iPod via Linux.

I've got a PERL helper function to generate the thumbnail hashes where the test data provided on the Wiki matches the output I get. All well and good so far. Unfortunately, following the database through to confirm that the hash I generate matches the hash XBMC has generated for an album doesn't give me the same results.

So far I've tried hashing the path taken by linked idPath from the song table and also be taking the full path/song_name.mp3 combination from the first song entered into the DB for the album.

What I must admit I haven't done yet is upgraded to the current version of XBMC but I'm assuming that the hashing algorithm for the thumbnails hasn't changed in quite some time...

If the hashing algorithm has changed (the version I'm running is quite a way before the Atlantis release) then I'll go through the upgrade but I'd rather not do it just for the sake of it.

Am I hashing the wrong thing from the DB or missing the blinding obvious?

Thanks!

- jmarshall - 2009-05-27

Album thumbs are hashed based on album name and album artist. No path stuff at all.

Song thumbs (i.e. those that we can't generate an album name and album artist for) are hashed based on song path.

Folder thumbs (those assigned to a folder - may or may not be an album, but this really isn't of our concern) are hashed based on the folder path.

The thumbs you find in the database will be almost always the former.

Cheers,
Jonathan

- KeithB - 2009-05-27

Thanks for the quick reply. Assuming I'm only working with albums the database already knows about, how do I need to construct the album/artist string to put through the hashing algorithm?

I'm guessing something along the lines of <artist>/<album>.

Edit: Nevermind - I got it Smile

. It's the combination of "<album><artist>".

Thanks for the reply jonathon - that came close to keeping me up half the night figuring it out Smile

- jmarshall - 2009-05-27

Cool - glad you got it going. Please publish the script you use to generate the correct hash so that others can use it Smile

Cheers,
Jonathan

- KeithB - 2009-05-27

That's the easy bit Smile

Code:
#!/usr/bin/perl

sub xbmcHash {

  my( $hashInput ) = shift;

  $hashInput = lc $hashInput;

  my $m_crc = 0xFFFFFFFF;

  for my $byte( unpack 'C*', $hashInput ) {

    $m_crc = $m_crc ^ ( $byte << 24 );

    for (my $rep = 0; $rep < 8; $rep++) {

      if (( $m_crc & 0x80000000) == 0x80000000) {

        $m_crc = ($m_crc << 1) ^ 0x04C11DB7;

      } else {

        $m_crc = ($m_crc << 1); 

      }

    }

  }

  return $m_crc;

}

my $dataToHash = $ARGV[0];

printf "%08x\n", xbmcHash($dataToHash);

Save that out as something like thumbHash.pl then run it:

Code:
./thumbHash.pl "123456789"

0376e6e7

Once I've got the other scripts done I'll post links to them back here to for interest.

- KeithB - 2009-05-31

Sorry to drag this one up again, but another question on how XBMC does the hashing for albums...

I have a compilation album in the DB but try as I might I can't get the hash to match the thumbnail XBMC set when it originally scanned the DB and downloaded the (wrong) cover.

Tracking through the DB purely from the album table, the album is "Bad Boys" with the artist resolving to "Various".

Meanwhile, making the (possibly wrong) assumption that XBMC uses the first song in an album to hash for the album thumbnail, the song resolves through to the "Bad Boys" album but artist of "Various Artists".

Neither combination gives the hash that XBMC used to store the thumbnail it downloaded though.

Can anyone help with the intricacy I'm missing here? I'd rather match XBMCs hashing if possible when updating an existing thumbnail but if all else fails I can just overwrite the DB elsewhere to get the same result. Seems pointless if XBMC may just revert it back later though.

Edit: Nevermind - I've worked it out. It seems when downloading the covers, the hash is based on the combination of what allmusic.com says the album and artist is rather than what XBMC says it is. In this instance, an album search on allmusic.com comes back with an album by Barraca called Bad Boys at the top of the list - which generates the correct hash.

Is this intended functionality to generate the hash based on album name and downloaded artist rather than stored artist, or am I being difficult and finding bugs?

- jmarshall - 2009-06-01

Did it manage to assign that thumb to the album OK? If so, that's strange - it must be storing the hash in the db.

IMO that's incorrect behaviour.

- KeithB - 2009-06-01

The thumbnail was downloaded, stored and the DB updated perfectly happily and the XBMC UI shows the thumbnail it grabbed.

I'm not sure what parameters XBMC submits its searches to allmusic.com with (I'm guessing album name only looking at the manual options available on the site), but...

Building the hash from the album table (i.e. strAlbum and artist as linked to from album) gives a hash of 0b4a322e.
Building the hash from the song table (i.e. strAlbum and artist as linked to from any track in song as they all link to the same artist entry) gives a hash of 1e538e69.
Meanwhile, building the hash from allmusic (i.e. strAlbum and artist as at the top of the allmusic.com list) gives a hash of 23c1b00d. This is the one that XBMC has stored in the thumb table and how it's filed the .tbn.

Interestingly, although it's built the hash from the artist as downloaded from allmusic.com, that artist doesn't exist in the artist table so I'm curious as to what it did with it after going to the effort of extracting it from the download result. That's just me though :p

- jmarshall - 2009-06-02

Actually, that doesn't really sound too bad, other than the failure to get the correct artist. It downloaded information for the album, accepted it (I guess you were prompted to choose it, or did it auto-choose?)

Whether or not the thumb should be cached based on the (possibly incorrect) song/album information, or whether it should be cached based on the (possibly incorrect) downloaded information is up for debate. I'd prefer the former I think?

Cheers,
Jonathan

- KeithB - 2009-06-02

The initial download and save was all done automatically from the original library scan. Any attempts to update album information since then have caused it to crash which I've put down to the fubarred python engine in the build I'm running (is that fixed in babylon btw?).

In terms of what and how it hashes, I agree - I think hashing to matching db contents is better. If nothing else, for consistent and repeatable behaviour. As a half way house, what about updating the db appropriately if it has got something else (artist/album name wise) and the agreement from the user?