2013-02-08, 16:24
I did some source code reading to make myself a little more familiar with this. I found a few interesting things. In the Music Scanning area of the code here is the basic order of operations. Keep in mind I'm summarizing multiple C++ classes in a real general form here.
1) Your sources are loaded folder structure enumerated
2) XBMC creates a hash of each folder based on the contents, then compares this to a hash in the DB.
3) If the hashes don't match it decides to scan (or rescan) this folder for new content
4) All songs in this path are removed from the music DB
5) Tag information for all files in the path are scanned into the DB
6) Once all directories are complete artist info, and album info is both cleaned and updated based on the changes
As you can see removing all songs from the DB is part of the scan process. I'm guessing this was done to ensure that any files that were modified, moved, or deleted are captured in the scan. This is inherintly different than video scanning because Album, Artist, Track, etc type information is actually held in the file via the Tag. If all the songs weren't removed each one would have to be hashed and re-checked for changes anyway, this would add more overhead than what is really needed. I guess the prevailing logic here is "if we have to scan the tag on each individual file anyway why have extra DB queries to see if it already exists, then remove it, then add it. why not just remove them all and scan". A good example of why this is important would be track order on an album. If I suddenly realize that track 10 is really track 12 and change the info in the tag, xbmc needs to be able to detect that change and add it to the music db.
Compared to the video library where the basic steps are more like this:
1) Your sources are loaded folder structure enumerated
2) XBMC creates a hash of each folder based on the contents, then compares this to a hash in the DB.
3) If the hashes don't match it decides to scan (or rescan) this folder for new content
4) Each file is checked to see if it is in the DB, if not video info is scraped
5) at the end clean operations are done, if the advanced setting value is enabled
Also, one interesting snippet I found is the following comment. It appears in the video scanner class when sources do not exist
the m_bClean variable is the one set in your advancedsettings.xml file. It appears that if you are doing a clean on update via the advanced settings, and sources do not exist, they are excluded from the clean operation. This is different than an manual clean where nonexistant sources would just be deleted. It is basically the same as the "verify paths" option that exists in this addon. I know it didn't always work this way so this must have been added at some point. Very cool.
1) Your sources are loaded folder structure enumerated
2) XBMC creates a hash of each folder based on the contents, then compares this to a hash in the DB.
3) If the hashes don't match it decides to scan (or rescan) this folder for new content
4) All songs in this path are removed from the music DB
5) Tag information for all files in the path are scanned into the DB
6) Once all directories are complete artist info, and album info is both cleaned and updated based on the changes
As you can see removing all songs from the DB is part of the scan process. I'm guessing this was done to ensure that any files that were modified, moved, or deleted are captured in the scan. This is inherintly different than video scanning because Album, Artist, Track, etc type information is actually held in the file via the Tag. If all the songs weren't removed each one would have to be hashed and re-checked for changes anyway, this would add more overhead than what is really needed. I guess the prevailing logic here is "if we have to scan the tag on each individual file anyway why have extra DB queries to see if it already exists, then remove it, then add it. why not just remove them all and scan". A good example of why this is important would be track order on an album. If I suddenly realize that track 10 is really track 12 and change the info in the tag, xbmc needs to be able to detect that change and add it to the music db.
Compared to the video library where the basic steps are more like this:
1) Your sources are loaded folder structure enumerated
2) XBMC creates a hash of each folder based on the contents, then compares this to a hash in the DB.
3) If the hashes don't match it decides to scan (or rescan) this folder for new content
4) Each file is checked to see if it is in the DB, if not video info is scraped
5) at the end clean operations are done, if the advanced setting value is enabled
Also, one interesting snippet I found is the following comment. It appears in the video scanner class when sources do not exist
Code:
Note that this will skip clean (if m_bClean is enabled) if the directory really doesn't exist rather than a NAS being switched off. A manual clean from settings will still pick up and remove it though.the m_bClean variable is the one set in your advancedsettings.xml file. It appears that if you are doing a clean on update via the advanced settings, and sources do not exist, they are excluded from the clean operation. This is different than an manual clean where nonexistant sources would just be deleted. It is basically the same as the "verify paths" option that exists in this addon. I know it didn't always work this way so this must have been added at some point. Very cool.
It's incredibly useful though and is much appreciated.