Release TMDb TV Show scraper (Python - Default Matrix Scraper) - Printable Version +- Kodi Community Forum (https://forum.kodi.tv) +-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33) +--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27) +---- Forum: Information Providers (scrapers) (https://forum.kodi.tv/forumdisplay.php?fid=147) +----- Forum: TV Show Scrapers (https://forum.kodi.tv/forumdisplay.php?fid=305) +----- Thread: Release TMDb TV Show scraper (Python - Default Matrix Scraper) (/showthread.php?tid=357232) |
RE: TheMovieDB Python - TV Show scraper - Karellen - 2020-11-17 @Karlson2k Can you click on the below link and tell us what you see. Is the metadata in Russian or English https://api.tmdb.org/3/tv/70384?api_key=6a5be4999abf74eba1f9a8311294c267&language=ru&append_to_response=content_ratings,credits,external_ids,images&include_image_language=ru,en,null RE: TheMovieDB Python - TV Show scraper - pkscout - 2020-11-17 (2020-11-16, 21:26)Hebotsuki Wrote:Writer is the only type of crew that is added in Kodi. That's to keep it consistent with the movie scraper. I can look at adding the other ones, but it's a bit of a whack-a-mole problem. Anytime another kind of writer is added, I'd have to update the scraper. I'll take another look though. There might be a way to do it using "department" instead of roles.(2020-11-15, 05:34)pkscout Wrote:Thanks, @pkscout. I installed the latest version on leia and seems to work fine: actors, episodes, fallback to English works with titles and plot with no translation, etc. I don't see any missing information. I only have one question: which writing credits are scraped for the episodes? I tested a couple of TV shows and one of them doesn't scrape the writing credits, but in TMDb there are "screenplay" and "storyboard" for the writing credits. The other one only scraped the "writer" credits, but no the "stroryboard" writing credit, which is in TMDb too.(2020-11-14, 20:58)Hebotsuki Wrote: It's not working for me either. According to Travis from TheMoviedb, some new fileds have been added to the credits in order to a future support for translated person names. That's probably what broke the scraper. I guess the scraper developpers are already aware of the problem.Just FYI that there is an update available from the repo now for this. If you don't get it automatically, you can force check for updates, and that should download it. RE: TheMovieDB Python - TV Show scraper - Karellen - 2020-11-17 @Karlson2k We think we have found the problem, but it will take some time to fix. In the meantime, can you Refresh the tv show with the new scraper (no need to refresh the episodes when asked). This will update the episodeguide URL and allow you to scrape with the new scraper. RE: TheMovieDB Python - TV Show scraper - jads - 2020-11-17 Hi, I found what I think is another issue with the episode groups scraping. It seems like the scraper maps groups onto season numbers by just numbering them 1 to N, in the order in which the they were returned by the API. While this generally works fine with most groupings, where the groups are just seasons 1 to N or volumes 1 to N, it breaks on some shows that still have a "specials" grouping. If your specials group is at the beginning it ends up being season #1 and every other season/volume is off-by-one, or it ends up being some extra later season. For an example of this see https://www.themoviedb.org/tv/2418-the-venture-bros/episode_group/5d4644c52d1e40219bbfa30e which has a Specials group ordered as #0 but which the scraper will number as "season 8". I think the original TMDB scraper used the "order" field as the season number, which generally doesn't have this problem; most of the time it matches the sorted order anyway, and in cases where it doesn't it's an editable element of the data that's can be fixed. RE: TheMovieDB Python - TV Show scraper - Karellen - 2020-11-17 (2020-11-17, 08:10)jads Wrote: Hi, I found what I think is another issue with the episode groups scrapingHmmm, yes I see what you mean. You can spot it in the log on line 970 and 1809... https://paste.kodi.tv/osezudimov.kodi [split] TheMovieDB - TV Show scraper - hellcry - 2020-11-17 I did not manage under any circumstances to paste the log on any of the two mentioned sites, but i did upload it to dropbox. Link: https://www.dropbox.com/s/dbz3bo6gun11rlj/kodi.log?dl=0 I have 3 shows that don't work, Fear the Walking Dead, Las Fierbinti and The Walking Dead, please have a look maybe you can find the issue, thank you in advance! RE: TheMovieDB Python - TV Show scraper - Karellen - 2020-11-17 @hellcry You are using an old version of the scraper. Before I look at the issue, please update to v1.1.23 by downloading the zip file and installing via zip... https://github.com/xbmc/metadata.tvshows.themoviedb.org.python/tree/leia There have been quite a few updated in the last two days and one or two more to come, and once all problems have been fixed it will be pushed to the repo, but that will be a couple of days still. Let us know if you still have problems. @Karlson2k The link above for v1.1.23 is the fix for your problem. RE: TheMovieDB Python - TV Show scraper - hellcry - 2020-11-17 (2020-11-17, 09:46)Karellen Wrote: @hellcry Thank you very much, I did the zip install and I can confirm all is working great. RE: TheMovieDB Python - TV Show scraper - pkscout - 2020-11-17 (2020-11-17, 08:10)jads Wrote: Hi, I found what I think is another issue with the episode groups scraping. It seems like the scraper maps groups onto season numbers by just numbering them 1 to N, in the order in which the they were returned by the API. While this generally works fine with most groupings, where the groups are just seasons 1 to N or volumes 1 to N, it breaks on some shows that still have a "specials" grouping. If your specials group is at the beginning it ends up being season #1 and every other season/volume is off-by-one, or it ends up being some extra later season. For an example of this see https://www.themoviedb.org/tv/2418-the-venture-bros/episode_group/5d4644c52d1e40219bbfa30e which has a Specials group ordered as #0 but which the scraper will number as "season 8". Thank you for pointing this out. Yes, the seasons are being done sequentially. All the shows I tested with didn't have specials in the episode grouping, so it worked. I have an update for you to test that is not yet part of the main branch that uses the order field instead of the sequence counting. I tested with Venture Bros, Firefly, and Money Heist, and it seems to work as expected. Matrix: https://github.com/xbmc/metadata.tvshows.themoviedb.org.python/tree/updated-groupings Leia: https://github.com/xbmc/metadata.tvshows.themoviedb.org.python/tree/updated-groupings-leia Just as a note, I didn't test the Leia code, but unless I made a typo, it is the same as the Matrix code (which I did test) and should work. RE: TheMovieDB Python - TV Show scraper - Karlson2k - 2020-11-17 (2020-11-17, 01:48)Karellen Wrote: @Karlson2kIt's Russian. (2020-11-17, 09:46)Karellen Wrote: @hellcryI've tested 1.1.24 Works fined, picked-up correctly data from non-python scraper. https://paste.kodi.tv/sohidemitu.kodi Two things to note:
RE: TheMovieDB Python - TV Show scraper - Karlson2k - 2020-11-17 (2020-11-17, 12:28)pkscout Wrote: Matrix: https://github.com/xbmc/metadata.tvshows.themoviedb.org.python/tree/updated-groupings Tested on Leia, works fine as well. https://paste.kodi.tv/imijupuruk.kodi (addon.xml still states that it's 1.1.24, while actually it's 1.1.25) Warnings about "invalid handle" are still present. Even when refreshed info without data update, there is "invalid handle" warning: https://paste.kodi.tv/oyazolujef.kodi RE: TheMovieDB Python - TV Show scraper - Karlson2k - 2020-11-17 @pkscout, why this scraper put temporally files to ".kodi/userdata/addon_data/metadata.tvshows.themoviedb.org.python/cache" instead of standard location ".kodi/temp/scrapers/metadata.tvshows.themoviedb.org.python"? I have ".kodi/temp" mounted on RAM disk to avoid SSD trashing and speedup data processing and now it's required to use some tricks to move this scraper temp files to RAM drive. RE: TheMovieDB Python - TV Show scraper - jads - 2020-11-17 (2020-11-17, 12:28)pkscout Wrote: Thank you for pointing this out. Yes, the seasons are being done sequentially. All the shows I tested with didn't have specials in the episode grouping, so it worked. I have an update for you to test that is not yet part of the main branch that uses the order field instead of the sequence counting. I tested with Venture Bros, Firefly, and Money Heist, and it seems to work as expected. I tried running a copy of the Leia code and it seems to do the right thing on my shows now. I don't have a Matrix instance set up so I can't (quickly) test that one myself. Just as an aside, it might also be safer to sort the episodes list by the 'order' key before sequentially numbering them. While I haven't seen any cases where the API returns episodes in a different order (and unlike with the season groups, you can't explicitly set or edit the order #) I'm not sure it's safe to assume the API will actually return the episode list in a particular order. RE: TheMovieDB Python - TV Show scraper - pkscout - 2020-11-17 (2020-11-17, 13:02)Karlson2k Wrote: I've tested 1.1.24I know #1 feels suboptimal, but because of the order of operations for Kodi and the way the TMDb API works, it's actually better to get the season information for a show when the show is loaded than to wait until the episode is loaded to do the season. Also I need the seasons if a show has a custom episode grouping, and that happens early in the process as well. For #2 I have no idea where that handle is being generated, but I'm like 98% sure it's not in the scraper code, so it might be whatever core process calls the scraper. RE: TheMovieDB Python - TV Show scraper - pkscout - 2020-11-17 (2020-11-17, 13:59)Karlson2k Wrote: @pkscout, why this scraper put temporally files to ".kodi/userdata/addon_data/metadata.tvshows.themoviedb.org.python/cache" instead of standard location ".kodi/temp/scrapers/metadata.tvshows.themoviedb.org.python"?I'm not familiar with the temp location. I use a python call to get the addon_data location and then store the addon's data there. If there is some other Python call that will return the path to that temp directory on every platform, I'd be happy to change it. I'd rather not start doing platform specific code though. As an aside, I'm using the same location the TV Maze scraper uses to store it's cache files. |