[black_eagle]

I think this is a Jarvis issue as I've not noticed it on any of the other upgrades I've done.

All my music library is sorted into artist directories containing the relevant artist.nfo and albums. Each album also has an album.nfo in it's directory. All tagging is handled with Picard.

So, I have a BeeGees album, "Their Greatest Hits_ The Record". When this is scanned in, everything is fine until it gets to the track "guilty" which is by "Barbra Streisand with Barry Gibb". Please note that in the following debug log excerpt, the URL returned for the artist does indeed resolve to Barbra Streisand and that the MB arid inserted into the database is incorrect for the artist it is inserting, and correct for Barbra.

PHP Code:

07:08:10 T:140663482525440   DEBUG: Mysql execute: replace into song_artist (idArtist, idSong, strArtist, strJoinPhrase, boolFeatured, iOrder) values(7830,51341,'Bee Gees','',0,0)
07:08:10 T:140663482525440   DEBUG: Mysql execute: INSERT INTO albuminfosong (idAlbumInfoSong,idAlbumInfo,iTrack,strTitle,iDuration) VALUES (NULL,2777,1,'Stayin\' Alive',0)
07:08:10 T:140663482525440   DEBUG: Mysql execute: INSERT INTO albuminfosong (idAlbumInfoSong,idAlbumInfo,iTrack,strTitle,iDuration) VALUES (NULL,2777,2,'How Deep Is Your Love',0)
07:08:10 T:140663482525440   DEBUG: Mysql execute: INSERT INTO albuminfosong (idAlbumInfoSong,idAlbumInfo,iTrack,strTitle,iDuration) VALUES (NULL,2777,3,'Night Fever',0)
07:08:10 T:140663482525440   DEBUG: Mysql execute: INSERT INTO albuminfosong (idAlbumInfoSong,idAlbumInfo,iTrack,strTitle,iDuration) VALUES (NULL,2777,4,'More Than a Woman',0)
07:08:10 T:140663482525440   DEBUG: Mysql execute: INSERT INTO albuminfosong (idAlbumInfoSong,idAlbumInfo,iTrack,strTitle,iDuration) VALUES (NULL,2777,5,'Emotion',220)
07:08:10 T:140663482525440   DEBUG: Mysql execute: INSERT INTO albuminfosong (idAlbumInfoSong,idAlbumInfo,iTrack,strTitle,iDuration) VALUES (NULL,2777,6,'Too Much Heaven',0)
07:08:10 T:140663482525440   DEBUG: Mysql execute: INSERT INTO albuminfosong (idAlbumInfoSong,idAlbumInfo,iTrack,strTitle,iDuration) VALUES (NULL,2777,7,'Tragedy',0)
07:08:10 T:140663482525440   DEBUG: Mysql execute: INSERT INTO albuminfosong (idAlbumInfoSong,idAlbumInfo,iTrack,strTitle,iDuration) VALUES (NULL,2777,8,'Love You Inside Out',0)
07:08:10 T:140663482525440   DEBUG: Mysql execute: INSERT INTO albuminfosong (idAlbumInfoSong,idAlbumInfo,iTrack,strTitle,iDuration) VALUES (NULL,2777,9,'Guilty',0)
07:08:10 T:140663482525440   DEBUG: Mysql execute: INSERT INTO albuminfosong (idAlbumInfoSong,idAlbumInfo,iTrack,strTitle,iDuration) VALUES (NULL,2777,10,'Heartbreaker',0)
07:08:10 T:140663482525440   DEBUG: Mysql execute: INSERT INTO albuminfosong (idAlbumInfoSong,idAlbumInfo,iTrack,strTitle,iDuration) VALUES (NULL,2777,11,'Islands in the Stream',0)
07:08:10 T:140663482525440   DEBUG: Mysql execute: INSERT INTO albuminfosong (idAlbumInfoSong,idAlbumInfo,iTrack,strTitle,iDuration) VALUES (NULL,2777,12,'You Win Again',0)
07:08:10 T:140663482525440   DEBUG: Mysql execute: INSERT INTO albuminfosong (idAlbumInfoSong,idAlbumInfo,iTrack,strTitle,iDuration) VALUES (NULL,2777,13,'One',0)
07:08:10 T:140663482525440   DEBUG: Mysql execute: INSERT INTO albuminfosong (idAlbumInfoSong,idAlbumInfo,iTrack,strTitle,iDuration) VALUES (NULL,2777,14,'Secret Love',0)
07:08:10 T:140663482525440   DEBUG: Mysql execute: INSERT INTO albuminfosong (idAlbumInfoSong,idAlbumInfo,iTrack,strTitle,iDuration) VALUES (NULL,2777,15,'For Whom the Bell Tolls',0)
07:08:10 T:140663482525440   DEBUG: Mysql execute: INSERT INTO albuminfosong (idAlbumInfoSong,idAlbumInfo,iTrack,strTitle,iDuration) VALUES (NULL,2777,16,'Alone',0)
07:08:10 T:140663482525440   DEBUG: Mysql execute: INSERT INTO albuminfosong (idAlbumInfoSong,idAlbumInfo,iTrack,strTitle,iDuration) VALUES (NULL,2777,17,'Immortality',0)
07:08:10 T:140663482525440   DEBUG: Mysql execute: INSERT INTO albuminfosong (idAlbumInfoSong,idAlbumInfo,iTrack,strTitle,iDuration) VALUES (NULL,2777,18,'This Is Where I Came In',0)
07:08:10 T:140663482525440   DEBUG: Mysql execute: INSERT INTO albuminfosong (idAlbumInfoSong,idAlbumInfo,iTrack,strTitle,iDuration) VALUES (NULL,2777,19,'Spicks & Specks',0)
07:08:10 T:140663482525440   DEBUG: Mysql execute: INSERT INTO albuminfosong (idAlbumInfoSong,idAlbumInfo,iTrack,strTitle,iDuration) VALUES (NULL,2777,20,'Love Me',0)
07:08:10 T:140663482525440   DEBUG: Mysql execute: INSERT INTO albuminfosong (idAlbumInfoSong,idAlbumInfo,iTrack,strTitle,iDuration) VALUES (NULL,2777,21,'You Should Be Dancing',0)
07:08:10 T:140663482525440   DEBUG: Mysql execute: UPDATE art SET url='nfs://192.168.1.50/media/sde1/home/home/xbmc/Music/Bee Gees/Their Greatest Hits_ The Record/folder.jpg' where art_id=11402
07:08:10 T:140663264880384   DEBUG: ffmpeg[7FEEB7EFA700]: [aac] ChannelElement 1.0 missing
07:08:10 T:140663482525440   DEBUG: Mysql commit transaction
07:08:10 T:140663482525440   DEBUG: UpdateDatabaseArtistInfo downloading info for: Barbra Streisand with Barry Gibb
07:08:10 T:140663482525440   DEBUG: scraper: ResolveIDToUrl returned <url>http://musicbrainz.org/ws/2/artist/56cd15a1-0d74-438b-8244-c96ffe1cae03?inc=url-rels</url>
07:08:10 T:140663264880384   DEBUG: ffmpeg[7FEEB7EFA700]: [aac] ChannelElement 1.0 missing
07:08:12 T:140663482525440   DEBUG: Previous line repeats 43 times.
07:08:12 T:140663482525440   DEBUG: -- nfo-scraper: Universal Artist Scraper
07:08:12 T:140663482525440   DEBUG: -- nfo url: http://musicbrainz.org/ws/2/artist/56cd15a1-0d74-438b-8244-c96ffe1cae03?inc=url-rels
07:08:12 T:140663482525440   DEBUG: Found matching nfo file: artist.nfo
07:08:12 T:140663482525440   DEBUG: DownloadArtistInfo Got details from nfo
07:08:12 T:140663482525440   DEBUG: Mysql execute: UPDATE artist SET  strArtist = 'Adam Ant',  strBorn = 'London (1954)', strFormed = '', strGenres = 'Pop/Rock',  strMoods = 'Cynical/Sarcastic / Rousing / Tense/Anxious / Theatrical / Visceral / Witty / Aggressive / Celebratory / Confident / Freewheeling / Quirky / Rebellious / Snide / Boisterous / Detached / Eccentric / Fierce / Fun / Humorous / Playful / Raucous / Rollicking / Brassy / Bravado / Bright / Dramatic / Earnest / Energetic / Gutsy / Intense / Lively / Sexual / Sexy / Street-Smart / Swaggering / Manic / Urgent / Hedonistic / Ironic / Irreverent / Menacing / Nocturnal / Ominous / Sleazy / Sophisticated / Trashy / Volatile / Campy / Exuberant', strStyles = 'Alternative/Indie Rock / Contemporary Pop/Rock / New Wave / Post-Punk / Punk/New Wave / Dance-Rock', strInstruments = '',  strBiography = 'Adam Ant is best known as a singer. He led Adam and the Ants in the early 1980s. Soon after he began a solo career with hits like \"Goody Two Shoes\" His most recent solo album was 1995\'s \"Wonderful\", which spawned the top 40 US single by the same name. His biggest contribution to music was probably his pioneering use of videos in the early 1980s.', strDied = '', strDisbanded = '',  strYearsActive = '1970s - 2010s', strImage = '<thumb preview=\"http://assets.fanart.tv/fanart/music/e188a520-9cb7-4f73-a3d7-2f70c6538e92/artistthumb/ant-adam-500052f733720.jpg/preview\">http://assets.fanart.tv/fanart/music/e188a520-9cb7-4f73-a3d7-2f70c6538e92/artistthumb/ant-adam-500052f733720.jpg</thumb><thumb preview=\"http://www.theaudiodb.com/images/media/artist/thumb/ant-adam-500052f733720.jpg/preview\">http://www.theaudiodb.com/images/media/artist/thumb/ant-adam-500052f733720.jpg</thumb><thumb>http://cps-static.rovicorp.com/3/JPG_1080/MI0003/489/MI0003489224.jpg?partner=allrovi.com</thumb><thumb>http://cps-static.rovicorp.com/3/JPG_1080/MI0001/467/MI0001467944.jpg?partner=allrovi.com</thumb>', strFanart = '<fanart><thumb preview=\"http://assets.fanart.tv/fanart/music/e188a520-9cb7-4f73-a3d7-2f70c6538e92/artistbackground/adam-ant-4ec52aa8dfbff.jpg/preview\">http://assets.fanart.tv/fanart/music/e188a520-9cb7-4f73-a3d7-2f70c6538e92/artistbackground/adam-ant-4ec52aa8dfbff.jpg</thumb><thumb preview=\"http://assets.fanart.tv/fanart/music/e188a520-9cb7-4f73-a3d7-2f70c6538e92/artistbackground/adam-ant-4ec52ab1e2e3a.jpg/preview\">http://assets.fanart.tv/fanart/music/e188a520-9cb7-4f73-a3d7-2f70c6538e92/artistbackground/adam-ant-4ec52ab1e2e3a.jpg</thumb></fanart>',  lastScraped = '2016-03-04 07:08:12', strMusicBrainzArtistID = '56cd15a1-0d74-438b-8244-c96ffe1cae03' WHERE idArtist = 7835 


Obviously, there is no mention of Adam Ant in the actual 'artist.nfo' in the directory, nor in the tags of that track and the strMusicBrainzArtistID inserted is for Barbra Streisand.

If anyone could shed any light on this I would be most grateful.

[DaveBlake]

I think this is a Jarvis issue as I've not noticed it on any of the other upgrades I've done.

All my music library is sorted into artist directories containing the relevant artist.nfo and albums. Each album also has an album.nfo in it's directory. All tagging is handled with Picard.

So, I have a BeeGees album, "Their Greatest Hits_ The Record". When this is scanned in, everything is fine until it gets to the track "guilty" which is by "Barbra Streisand with Barry Gibb". Please note that in the following debug log excerpt, the URL returned for the artist does indeed resolve to Barbra Streisand and that the MB arid inserted into the database is incorrect for the artist it is inserting, and correct for Barbra.

Obviously, there is no mention of Adam Ant in the actual 'artist.nfo' in the directory, nor in the tags of that track and the strMusicBrainzArtistID inserted is for Barbra Streisand.

If anyone could shed any light on this I would be most grateful.

Had to quote wide post just to read it. Can it be edited?

Have a look at the tags for this song please and how it is in the library before the scrapper attempts to update the artist data. Even Jarvis will see TPE1/ARTIST="Barbra Streisand with Barry Gibb" as a single artist, yet probably that song has 2 artist mbids. Does it also have ARTISTS tags? When this track is first scanned into the library do you get debug message about mis-match?

From Kodi's perspective you have a Streisand track in a Bee Gees folder with a Bee Gees NFO, not sure what the consequnces of that will be. But the Adam Ant update seems to imply that somehow Barbra Streisand's mbid is in the database with Adam Ant's name?

How big is your MyMusicXX.db? Could I take a look, it might give some clues.

[black_eagle]

Hi Dave, thanks for getting back to me. I do apologize that the code part was rather large. My library isn't actually that large at the moment, as I have shifted a load of stuff out of the music path in an attempt to debug this.

I have exported MyMusic56 (it's an SQL database not sqlite). It's a straight text file but can be imported directly into a MySQL instance. I've zipped it up and included the offending track, and the artist and album.nfo's. Size is only 4.7Mb and you can get it from https://www.dropbox.com/s/rfo7pupbbssllz...l.zip?dl=0

If you look up the mbid that Kodi inserts into the database for Adam Ant on musicbrainz, it's actually the correct mbid for Barbra Streisand.

Looking forward to what your experienced eye might spot, as I am struggling with this one !!

Cheers

[DaveBlake]

Quite a pickle in the data! Not having MySql available I did have a bit of messing to get it into SQLite Kodi library, but pretty sure I have the knub of it. The issue must start in the tags, or the scraper has gone crazy?

First "Guilty" tagged as it is has 2 Musicbrainz_Artist_ID tags for Barbra and Barry, but Kodi will only see one artist name "Barbra Streisand with Barry Gibb", after all how is Kodi meant to know that "with" is not part of a name. Best resolution is to re-tag this track adding ARTISTS (note the s) tags one for each artist name, then rescan.

But the Adam Ant situation is bonkers, lots of artist records for him with different mbids. How did it get that way?

Have you got "Fetch additional information during updates" and "Prefer online information" enabled?
If you have then could you disable both of these and then remove the source, clean the library, and then start again. This time just scan these sources into the library, no scraping, and let me have the resulting MyMusic56. I want to see what we get before we start scraping.

That will show what your tagging produces. But my hunch is that the scraper is getting lost maybe with some of the chars in Celine Dion songs, and then running amuck. So one step at a time, what does the library have from just tags first.

[black_eagle]

I do really appreciate you taking this on board, many thanks.

So, yes I did have both "fetch additional information during updates" and "Prefer online information" enabled.

I disabled them both, dropped the database (outside of Kodi), restarted Kodi, removed the source and answered 'yes' to "remove items in this path" (even though it was a clean database).

I then set the default scrapers (settings -> audio and playback) to local info only and scanned the 'Bee Gees' directory and the 'Celine Dion' directory afterwards. Please note that these are my wifes artist choices !!!

I have exported the MySQL database again and zipped it up. The zip file also contains the debug log of the scans and three screenshots showing the result in Kodi. Interesting lines in the debug log are 11038 where the scan of the original offending album begins, 11504 where it scans the track & 11853 where it inserts the correct stuff into the database. The scan of Celine Dion begins at 14827

I'm unsure as to why Celine Dion now appears as 'Celine Dion & Bee Gees' ? See the screenshots. A1 is the artist view in the music library, A2 & A3 are the first two entries under that view.

If it will help you, I'm quite prepared to drop the advancedsettings.xml on the main Kodi instance, and build a MyMusicXX.db in SQLite.

Download link is https://www.dropbox.com/s/gsm9o6vc473omx...e.zip?dl=0

Once again, thanks for looking into this, it's much appreciated.

[DaveBlake]

I'm happy to help. I always learn stuff from this kind of thing, moreover it shows me where Kodi needs to be made more robust or flexible. And you are providing good info to work with, which really helps.

The issue (with your wife's music, mine has similar tastes so I know where you are coming from) is tagging, possibly made worse by improvements in Jarvis that process the ARTISTS (note the s) tag at least giving different behaviour from Isengard etc. The issue is mis-matching mbid and artist tags. In some music files the number of names found in TPE1 does not match the number of MUSICBARINZ_ARTIST_ID tags, library chaos ensues.

For example Celine Dion now appears as 'Celine Dion & Bee Gees' because the song "Immortality" has TPE1 = 'Celine Dion & Bee Gees', and 2 mbids that of "Celine Dion" and "Bee Gees", causing Kodi to create an artist with name 'Celine Dion & Bee Gees' and Celine Dion's mbid. Subsequently every Celine Dion track we load (with her mbid) finds the extisting artist using that mbid and does not change the name.

Jarvis does not know that Celine Dion & Bee Gees is two artist names, same with "Celine Dion feat. Peabo Bryson" or "Barbra Streisand with Barry Gibb". Picard (or mp3tag) has output that as TPE1 yet also the multiple mbids, and Jarvis just does what it can with the first mbid it finds. In Krypton there are attempts at repairing this kind of mis-match, or at least warning you there is an issue, But in Jarvis you just need to fix those tags.

Fix the tags by adding ARTISTS tags to match the mbids, and all will be well. You will then see "Celine Dion feat. Peabo Bryson" while playing that track, but Celine Dion and Peabo Bryson will appear correctly as separate artists. Picard gives ARTISTS tags now for some tracks, but not all yet, so this may need to be a manual job.

How to find those files that need a tagging change? Well you could try using Krypton, the files with mismatched tags get listed in the debug. Otherwise scan with what you have then use some SQL

Code:

SELECT song.idsong, song.strTitle, song.strArtists, song_artist.strArtist, artist.strArtist FROM song
JOIN song_artist on song_artist.idSong = song.idSong
JOIN artist on song_artist.idArtist = artist.idArtist
WHERE song_artist.strArtist <> artist.strArtist
order by artist.strArtist

that will find some of the issues.

Unfortunately it is this kind of behaviour that has given mbids a bad reputation. I hope you will stick with it, the mbids do make the scraping results much better as long as they are consistent. I am aware of the pain this causes users, and Krypton will be smoother. Let me know if I can clarify anything.

[black_eagle]

You're a star !!!

Turns out the version of Picard I was using was rather out of date as I couldn't see any method of adding the ARTISTS tag you mentioned. Updated it and reloaded "one night only" and bingo ! An artists tag with "Bee Gees;Celine Dion" appeared. I guess I will have to re-check all my stuff with this new version then, although that SQL query will be extremely handy, so thanks for that.

I will be sticking with mbid's - the more that people use them (and report issues with them), the better the experience should become.

As regards my 'Adam Ant' experience, this happened again with Barry Gibb - I got the artist bio for Adam and associated fanart, yet the artist name was Barry Gibb. This was with "fetch additional updates" turned on, but scrapers set to local info only, which I believe should only use artist & album .nfo files. Refreshing the artist information updated the artist bio correctly. Unfortunately I don't have a log of this as it happened when I was messing with tags and artist naming in a vain attempt to fix the Bee Gees /Celine Dion issue.

[DaveBlake]

Thanks for your positive attitude, makes helping you worthwhile

I hope that you don't have to retag everything, depending on the size of your music collection, but anywhere you have artists names "hidden" in the TPE1 string (along with "and", "with", "feat." etc) and mbids for each, you will need to check.

I think that the "Adam Ant" and the scraper problems, were what happens if you scrape on top of a confused library. The longer term Kodi goal is for scanning and scraping to be one invisible process, but we are not there yet. Best for now to scan, be happy that the library makes sense, and then scrape for fan art (local or online).

I wish I had back ported some of the tagging fixes that are in Krypton into Jarvis, but it was late in the cycle and no one seemed enthusiastic to take the risk. But you would not be having this issue if I had. Get a few more users with the same thing perhaps and maybe the dev team will reconsider?

Anyway good luck with updating those tags. Beware that Picard by default may use semi-colon as separator and Kodi wants space-slash-space (unless you change that in one or the other). Then let me know how it goes or if you need some more library debugging.

[black_eagle]

Thanks for the info. Picard does indeed use a semi-colon and I see no easy way of changing that, other than writing a little plugin to do it with.

This leads me to the question though of why, if we agree that mbid's are the way to go and that the most obvious way of tagging with these is by the use of Picard, does Kodi expect a different separator ? Surely the default separator is the more logical choice, or is there a technical reason for Kodi requiring space-slash-space ?

[DaveBlake]

Thanks for the info. Picard does indeed use a semi-colon and I see no easy way of changing that, other than writing a little plugin to do it with.

A couple of lines in the script section of Picard will do it Options>Advanced>Scripting, for example

Code:

$set(albumartist,$replace(%albumartist%,; , / ))
$set(albumartist,$replace(%albumartist%,\, , / ))
$set(artist,$replace(%artist%,; , / ))
$set(artist,$replace(%artist%,\, , / ))

This leads me to the question though of why, if we agree that mbid's are the way to go and that the most obvious way of tagging with these is by the use of Picard, does Kodi expect a different separator ? Surely the default separator is the more logical choice, or is there a technical reason for Kodi requiring space-slash-space ?

I think it is just historic. IMO Kodi should deal with multiple separators, semi-colon and slash-space-slash as defaults but user configurable. It is a change I feel more encouraged to pursue in Krypton, but changes to tag processing is always risky (almost bound to upset someone). Parsing "with", "and", "&", "Feat." etc. is tricky because like slash (no spaces) and comma they can be in artist names, but semi-colon makes a good separator. More work!

Meanwhile, if all your tags use semicolon as the separator, inc. genre/TCON then you could set that in Kodi in advancedsettings.xml. My tagging as mixed so I took the Picard script approach.

[scott967]

Hate to get on my hobby horse, but that's why ID3 2.4, APE, Xiph, etc tags are so much better: no in-band separators, though I guess TIPL and TMCL break that rule.

scott s.
.

[DaveBlake]

Hate to get on my hobby horse, but that's why ID3 2.4, APE, Xiph, etc tags are so much better: no in-band separators, though I guess TIPL and TMCL break that rule.

Yes unfortunately they do, and some tagging software resuts in single Xiph/Vorbis/FLAC tags containing a union of values with separators e.g. GENRE = "genre1 / genre2"

It is just a fact of life that users have music files with multiple values in a single tag with separators, other players cope so IMO Kodi needs to handle it too.

[black_eagle]

Everything added in the last 3~4 years is (as far as I have been able to determine) correctly tagged with ID3 2.4 tags. However, my music collection (tagging wise) stretches back to the last century and I am currently checking & re-tagging anything which isn't 100% correct.

I have got to 'D' so far .....

[scott967]

Hate to get on my hobby horse, but that's why ID3 2.4, APE, Xiph, etc tags are so much better: no in-band separators, though I guess TIPL and TMCL break that rule.

Yes unfortunately they do, and some tagging software resuts in single Xiph/Vorbis/FLAC tags containing a union of values with separators e.g. GENRE = "genre1 / genre2"

It is just a fact of life that users have music files with multiple values in a single tag with separators, other players cope so IMO Kodi needs to handle it too.

One issue is that I think the Kodi separator is also used in video nfo file parsing and maybe elsewhere. Also in Kodi the display separator is also the parser separator.

From what I've seen in other software, the UI has a user-set separator, and that is used for display and for user manual (keyboard) input. The actual database does not store any separator, but uses a tag-format parser to read/write tags with the appropriate separator.

scott s.
.

[DaveBlake]

One issue is that I think the Kodi separator is also used in video nfo file parsing and maybe elsewhere. Also in Kodi the display separator is also the parser separator.

From what I've seen in other software, the UI has a user-set separator, and that is used for display and for user manual (keyboard) input. The actual database does not store any separator, but uses a tag-format parser to read/write tags with the appropriate separator.

Thanks for your thoughts Scott.

Video has it's own item separator, they just happen to both default to space-slash-space, so not worries there. My initial idea is to use the first parser separator (if more than one) as the display separator. But since the addition of ARTISTS tag processing the display separator is not used as much. The artist string for a song is now stored in the song table and used in display preserving the "artist1 feat. artist2". But it is used for building a genre string for display.

Kodi (Gotham to Jarvis) stores the separator for every song artist, but other parts don't use that to rebuild the name string any way. Krypton aleady removes that redundant storage, we have and use the string in song.strArtists no need to build it. The idea of having separate parse (multiple) and display (single) separators is interesting, or maybe allowing parsing to be completely automatic?

I have just ripped a new pile of CDs, and scanned into the library as is for a test without fussing over the tags too much. It makes me well aware that for best results Kodi still needs careful tagging work from the user. I'm with you Black_eagle, my classical music collection still needs lots of painstaking tidying before Kodi will behave nicely.