[monisriz]

Thanks! I have multiple devices with Kodi so an update will definitely save me the trouble of installing manually

[flskydiver]

So, folks, I've got a problem I could use some help with.

When I scrape television shows with this Python Scraper, the main cast members (season regulars) get written to the episode-level .NFO files last.
Yes, they have appropriate actor <sort> tags, but that doesn't help me in my use case: I view my content on my TV's using the Infuse App on Apple TV, and Infuse does not support actor sort tags.
Further, Infuse only shows the first 15 listed actors as read from the top down in the .nfo file.  Often an episode may have so many listed guest stars, recurring characters, and credited extras that the very stars of the show aren't even displayed.

I've found (and I don't know why this works), that once I create .NFO files with the Python "TMDb TV Shows" scraper, I can then Change content on the relevant folders, and select the older "The Movie Database" scraper, and rescan my library.  Once this is complete, if I re-export the .NFO files (overwriting what was created earlier), the only change that occurs to the .NFO files created with "TMDb TV Shows" scraper is that the main cast is moved to the top of the actors list —  all <actor> tags are now arranged in the file in order, top down, by the value listed in the <sort> tag.

So that works, but is incredibly tedious!

How do I go about creating .NFO files with Season Regular actors up top from the start?

Can I change something in the scraper?  Is there accessible code somewhere I can edit?

Oddly, I've found using the older "The Movie Database" television scraper alone, on series regular cast are include in the ouput .NFOs.  No guest stars, recurring characters, or credited extras are include at all.

[pkscout]

This is an unfortunate side effect of something core Kodi does plus something TMDb does.  Kodi appends the series cast list to every episode, but at the end, and TMDb doesn't give you the series cast as part of the episode cast that is returned from the API.  During development of the scraper, I added the series cast at the beginning and then added the guest stars, but then the series cast was repeated at the end again.  We had to pick one of two bad ways to display the cast, so I went with displaying the guest cast for an episode first and then let core Kodi display to series cast at the end.

If you export NFO files, reorder them, and then import them, that bypasses the scraper, as NFO files are used before scrapped data.  So you get the cast in the order you want.   I know this isn't great, but it is what it is for now.

[flskydiver]

[...] During development of the scraper, I added the series cast at the beginning and then added the guest stars, but then the series cast was repeated at the end again.  We had to pick one of two bad ways to display the cast, so I went with displaying the guest cast for an episode first and then let core Kodi display to series cast at the end.[...]

Thank you, very much, for your reply and this explanation.

While trying to figure this out, I tried bypassing Kodi and scraping television from TMDB with TinyMediaManager instead.  Though TMM apparently doesn't do as Kodi does, and seems to get the cast sorted top to bottom in the .NFOs, I noticed it didn't correctly save at least one other episode tag that either Kodi scraper manages fine (the <mpaa> tag).  Guess I'll just go ahead and wipe out all my NFOs and start from scratch (using the Kodi dual-scraper process I identified above), and wipe and update the episode thumbnails while I'm at it.

Thanks again.  Sorry for my late reply.

[Karellen]

(using the Kodi dual-scraper process I identified above)

Don't do this.
The Movie Database is broken and has been removed from the repo. It is known to return spurious data, misses first episodes and first seasons, and mixes or omits the very important UniqueID's.
Also, if you want local thumbs, install the Artwork Dump so it works in tandem with the scraper.

[flskydiver]

(using the Kodi dual-scraper process I identified above)

Don't do this.
The Movie Database is broken and has been removed from the repo. [...]
Also, if you want local thumbs, install the Artwork Dump so it works in tandem with the scraper.

The only thing that seemed to change when I had The Movie Database overwrite NFO files that were already created by TMDb TV Show was that the main cast was moved to the top of the list.  Nothing that the Python scraper scraped was removed form my test NFOs.  In fact, the only way I could tell the NFOs generated by the Python Scraper apart from the ones overwritten by The Movie Database were the ordering of the cast (by actor order tag).  Had I only run the old scraper, the NFO files generated would have looked completely different from those of the Python scraper.

I have no idea why it seems to work this way, but I'll certainly keep an eye on things as I move through this process.

Thank you for the head's up.

[flskydiver]

(using the Kodi dual-scraper process I identified above)

The Movie Database is broken [...] it misses first episodes and first seasons, and mixes or omits the very important UniqueID's.

To explain a few other things ...  I don't really need The Movie Database scraper to scrape anything new ... so if it misses stuff that probably won't be a problem because it will already be in the NFOs.

Further, my use-case requires I be obsessive about file naming (to match TMDB exactly) because the software I use view to my content on my televisions (via Apple TVs) insists on scraping it's own data through it's own API interface with TMDB ... it does not query local NFO files for this data.

While preferred artwork can be completely customized, Infuse only has limited support for local textual metadata.  Users can only alter the displayed titles of movies, television series, and episodes; user ratings, plot descriptions, genre tags, and premiered dates (for whatever reason).  I've only ever needed to override titles of movies and series and, rarely, episode titles. And I'll batch delete a few poorly implemented genre tags I don't want cluttering up my display (such as "TV Movie"). 

For specific skydiving and BASE related productions that were commercially released within the community but weren't given wider releases by major studios or assigned IMDB entries, I'll alter all of the above, plus the plot, and that gets the job done of adding a title that doesn't exist on TMDB (and wouldn't be allowed there) into my library.

I was happy letting Infuse handle the rest.  But they recently decided to add support for local cast and crew ... and that messed things up for me.  Not giving users the option to opt out, the current situation demands you either manage your own cast and crew or get rid of your NFO files (and everything else they for you) altogether.  Neither of which is satisfactory.

[flskydiver]

This is an unfortunate side effect of something core Kodi does plus something TMDb does.  Kodi appends the series cast list to every episode, but at the end, and TMDb doesn't give you the series cast as part of the episode cast that is returned from the API.  During development of the scraper, I added the series cast at the beginning and then added the guest stars, but then the series cast was repeated at the end again.  We had to pick one of two bad ways to display the cast, so I went with displaying the guest cast for an episode first and then let core Kodi display to series cast at the end.

So ... I've since confirmed with extensive testing and side-by-side comparisons two odd things:

If I scrape with TMDb and write NFOs to disc, I get results as you described.
If I then change content, rescrape with The Movie Database, and overwrite the previously created NFOs ...  NOTHING CHANGES except the order of the actor tag blocks (now being arranged in order of the actor-sort tag value). The Movie Database Scraper changes NOTHING ELSE when overwriting NFOs written by (or after first scraping with) the TMDb TV Show scraper. 

So, essentially, that bizarre workaround has solved my original issue.

However, I did notice something new.

TMDB cast listings accessible via their website appear to be episode accurate (at least on the high profile shows I've examined).

For example, take the sci-fi series "The Expanse".  4 actors in the main cast are noted as having appeared in all 62 episodes. Cas Anvar, would have been the fifth, if he hadn't been famously fired during Season 5 and killed off through CGI during the Season 5 finale.  He does not appear in TMDB's cast list for season 6.

But he still appears as main cast member for every episode of Season 6 in the .NFO files I created (from scratch, without editing) with Kodi, after scraping with only the TMDB TV Show Scraper.

In fact, eight other actors in this series, who did not appear in all episodes, are also given credit (at the bottom of all 62 episode.NFO files) for being in every episode. 

IS this is what you mean by Kodi insisting on adding whatever it considers to be main cast members to the end of each of the NFO files created using the TMDB TV Show Scraper?

What makes Kodi believe these 13 actors needed to be listed for every episode (when Kodi only knows what the scraper tells it)? 

Why is the number of "permanent" actors (actors listed in the NFO files for every episode whether or not they actually appeared in all of them) different with each series?

I've noticed this behavior is universal.  All television series are affected.

What can be done?

Is there a way to have the scraper grab content on a per-episode basis, as Infuse seems to be able to do natively with its TMDB API, or at least on a per-season basis?

Or is this an impossibility with Kodi ,and do I need to look elsewhere?  If so, where?

Thank you for your continued assistance helping me understand this perplexing issue.

[pkscout]

The actors are done per season, but Kodi wants the show to have actors, so we put all actors from any season there.  Episodes from TMDb only have guest cast, not season cast.  And we're back to where we were.  Kodi core puts the show actors at the end of the list of episode actors.  If that behavior ever changes, we could update the scraper to do main actors by season on each episode for a given season and then the guests for each episode after.

[pkscout]

@flskydiver I took another look at the cast code, and I've updated it so it's somewhat better.  The scraper will now take the cast list for a given season and use that as the core of the cast list for each episode in that season and then add the guest stars.  The series cast list is still all actors from any season, so the end of the episode cast list still has any actors that weren't already loaded.  At least they seem to not be duplicating, which either means I'm misremembering or something changed between Leia and Matrix and I never noticed.

I tested with the modern Doctor Who (which is a very good test for cast changes), and it does work, with some caveats.  There is at least one season where there is a major cast change midway through.  So for the episodes in that season both the "first half" and "second half" cast members are always listed as cast.  There's just no way around that with the data TMDb provides.  The other caveat is that the order is taken from TMDb, and some shows (like Doctor Who) try and order the season cast in a way that they will show up in chronological order.  That mostly works, but sometimes a guest star is ordered pretty high, so they end up intermixed.  Again, there is no way around that with the data TMDb provides.

I also tried with Stargate SG-1 and Stargate Atlantis.  Those work as well, although the ordering issue is more prevalent there.

The big bummer is you have to rescrape your shows to get updated cast lists.  Again, there's just no way around that given how Kodi works.

I've submitted the pull request for this to the scraper repo.  I just have to wait for someone on the team to review the code before it is pushed out.  Once out, the Matrix version will be 1.4.15 and the Nexus version is 1.5.5.

[flskydiver]

@flskydiver I took another look at the cast code, and I've updated it so it's somewhat better.  The scraper will now take the cast list for a given season and use that as the core of the cast list for each episode in that season and then add the guest stars. [...]

This is fantastic news!  Thank you!!

I've been preparing to rescrape everything anyway, so no worries there —  but I'm so very happy I won't need to risk using TMM (which seems to do approximately the same thing, less appending leftover series cast to the end). But no matter there, it's the top of the list that matters most to me (especially since we haven't been able to convince the Firecore folks to permit Infuse users to show more than the first 15 cast members listed —  though to be fair, when in the right order, that's usually enough). TMM seems to have a nasty habit (that I can't seem to break no matter how much I research it) of writing NFO files (and any images you might have wanted it to verify) to disc as soon as you import something —  never asking for permission or giving you a heads up; nor, nonsensically, even waiting for you to actually scrape the metadata it needs or give you the opportunity to edit it first.  I certainly prefer Kodi's style of never writing anything to disk until I explicitly tell it too, and even then, giving me detailed control of what gets written and what stays protected; every time.

Ah! Sooo very happy to see this.  You're awesome!  Very much looking forward to getting to work with the new version.  Yay!

Blue skies, my friend.

[flskydiver]

Once out, the Matrix version will be 1.4.15

I couldn't wait.  lol.  I implemented the changes from GitHub for the Matrix version and happy happy happy to report it's working for me just as described.  Yay!!!  Thank you so much for your attention to this issue.  You rock, sir.

[flskydiver]

The Movie Database is broken and has been removed from the repo. It is known to return spurious data, misses first episodes and first seasons, and mixes or omits the very important UniqueID's.

How do I remove this from my installation of Kodi? It was still there in 19.4 and still there in 20.0 Alpha 2.  Don't want to use it accidentally.

pkscouts's edits to TMDb Scraper have worked wonderfully for me, after scraping my entire collection anew.  Thank you!!

[Karellen]

How do I remove this from my installation of Kodi?

Easiest method is to not apply it to any sources.
Otherwise you would need to delete the folder from the \addons\ folder in the program installation folder.

[flskydiver]

Otherwise you would need to delete the folder from the \addons\ folder in the program installation folder.

Oops.  Did that, and now I can't open the app.  "ERROR: Unable to create application. Exiting."
To be fair, I'm on Alpha 2 so who knows what might have been the problem.

You'd think I'd have learned to make copies before deleting things by this point.  Sigh.

Hey, one other thing I noticed.  No biggie.  I'm on the simple setting for images (just posters, fanart, and thumbs) but 31 of 372 series also saved banner.jpg files to disk.  Easy enough to batch delete them, but something I noticed on 19.4 too.

*Update:  Just reinstalled on top to get the folders back (both of the non-python versions).  Before trying to open the new version, pulled the folders (but copied them this time).  No bueno.  Wouldn't open until both were back in programs\kodi\addons folder.  No worries, I'll just deal with it.