Release TMDb TV Show scraper (Python - Default Matrix Scraper) - Printable Version +- Kodi Community Forum (https://forum.kodi.tv) +-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33) +--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27) +---- Forum: Information Providers (scrapers) (https://forum.kodi.tv/forumdisplay.php?fid=147) +----- Forum: TV Show Scrapers (https://forum.kodi.tv/forumdisplay.php?fid=305) +----- Thread: Release TMDb TV Show scraper (Python - Default Matrix Scraper) (/showthread.php?tid=357232) |
RE: TheMovieDB Python - TV Show scraper - ramis52 - 2020-11-16 (2020-11-16, 19:19)ramis52 Wrote:(2020-11-16, 19:11)pkscout Wrote:thank you. i can confirm its working.(2020-11-16, 17:31)ramis52 Wrote: Hi,It appears there were some undocumented changes to the API response from TMDb (sometimes episode name is not included in the response rather than being blank). I've got an update that I believe fixes this issue. Please try: ok. i must say. it is not all fixed. for some reason it now adds episodes, but when there is no translation found for de-DE and Fallback (english) it adds the episode title now as an empty string. it happens for me on this show https://www.themoviedb.org/tv/86189-betty-en-ny/season/1 RE: TheMovieDB Python - TV Show scraper - pkscout - 2020-11-16 (2020-11-16, 19:37)ramis52 Wrote:OK, once more into the breach. I think I have it fixed. I tested with Betty en NY with German as my language, and it now scrapes. It's worth noting that many of the episodes have neither German nor English translations, so after about episode 45 you aren't going to get much data.(2020-11-16, 19:19)ramis52 Wrote:(2020-11-16, 19:11)pkscout Wrote: It appears there were some undocumented changes to the API response from TMDb (sometimes episode name is not included in the response rather than being blank). I've got an update that I believe fixes this issue. Please try:thank you. i can confirm its working. @roidy Could you try this update as well? The log you sent didn't have debug logging enabled, but I was able to see that the show you were trying to do was Normal People. With this update I was able to scrape that show (using German as my primary language I got all the English fallbacks), so I'm hoping it will work for you as well. Matrix (1.3.5): https://github.com/xbmc/metadata.tvshows.themoviedb.org.python/tree/matrix Leia (1.1.21): https://github.com/xbmc/metadata.tvshows.themoviedb.org.python/tree/leia RE: TheMovieDB Python - TV Show scraper - Hebotsuki - 2020-11-16 (2020-11-15, 05:34)pkscout Wrote:Thanks, @pkscout. I installed the latest version on leia and seems to work fine: actors, episodes, fallback to English works with titles and plot with no translation, etc. I don't see any missing information. I only have one question: which writing credits are scraped for the episodes? I tested a couple of TV shows and one of them doesn't scrape the writing credits, but in TMDb there are "screenplay" and "storyboard" for the writing credits. The other one only scraped the "writer" credits, but no the "stroryboard" writing credit, which is in TMDb too.(2020-11-14, 20:58)Hebotsuki Wrote: It's not working for me either. According to Travis from TheMoviedb, some new fileds have been added to the credits in order to a future support for translated person names. That's probably what broke the scraper. I guess the scraper developpers are already aware of the problem.Just FYI that there is an update available from the repo now for this. If you don't get it automatically, you can force check for updates, and that should download it. Anyway, thanks a lot for the work! RE: TheMovieDB Python - TV Show scraper - roidy - 2020-11-16 (2020-11-16, 20:53)pkscout Wrote:@pkscout Ok, using 1.1.21 the show Normal People scraps to the library with no errors shown, however all the episode titles are missing.(2020-11-16, 19:37)ramis52 Wrote:OK, once more into the breach. I think I have it fixed. I tested with Betty en NY with German as my language, and it now scrapes. It's worth noting that many of the episodes have neither German nor English translations, so after about episode 45 you aren't going to get much data.(2020-11-16, 19:19)ramis52 Wrote: thank you. i can confirm its working. Here'a a log with debug logging enabled, and again it comes out to about 3.2MB so I've zipped it and uploaded to https://gofile.io/d/XhhY7y RE: TheMovieDB Python - TV Show scraper - Karellen - 2020-11-16 (2020-11-16, 21:29)roidy Wrote: however all the episode titles are missing.@pkscout I wonder if it is related to what happened here... https://github.com/xbmc/metadata.tvshows.themoviedb.org.python/issues/18 Both Money Heist and Normal People use Episode x as the episode title https://www.themoviedb.org/tv/71446-la-casa-de-papel/season/1 https://www.themoviedb.org/tv/89905-normal-people/season/1 RE: TheMovieDB Python - TV Show scraper - pkscout - 2020-11-16 (2020-11-16, 21:29)roidy Wrote:OK, here's another to try. I've tested it both with Normal People and Money Heist. It looks like TMDb changed the way they are returning data in the API. Technically the show Normal People has no episode names. On the TMDb web site they show as Episode xx, and up until a few days ago, the API would return that same episode title (i.e. Episode 1). But now the API returns NULL for episode name in cases like this. So I had to add logic to the scraper to create an episode title if one does not exist. Now Kodi should show the same thing the web site does. My apologies for this turmoil. I only find out about TMDb API changes when you all start reporting issues, as the TMDb folks don't announce changes ahead of time.(2020-11-16, 20:53)pkscout Wrote:@pkscout Ok, using 1.1.21 the show Normal People scraps to the library with no errors shown, however all the episode titles are missing.(2020-11-16, 19:37)ramis52 Wrote: ok. i must say. it is not all fixed. for some reason it now adds episodes, but when there is no translation found for de-DE and Fallback (english) it adds the episode title now as an empty string.OK, once more into the breach. I think I have it fixed. I tested with Betty en NY with German as my language, and it now scrapes. It's worth noting that many of the episodes have neither German nor English translations, so after about episode 45 you aren't going to get much data. Matrix (1.3.6): https://github.com/xbmc/metadata.tvshows.themoviedb.org.python/tree/matrix Leia (1.1.22): https://github.com/xbmc/metadata.tvshows.themoviedb.org.python/tree/leia One other thing. If you are testing, please go to addon_data/metadata.tvshows.themoviedb.org.python/cache and delete all the files there before rescraping. Those cache files are valid for about 3 hours, and if the scraper uses them the updated code might not ever get invoked. RE: TheMovieDB Python - TV Show scraper - roidy - 2020-11-16 (2020-11-16, 20:53)pkscout Wrote: It looks like TMDb changed the way they are returning data in the API. Technically the show Normal People has no episode names. On the TMDb web site they show as Episode xx, and up until a few days ago, the API would return that same episode title (i.e. Episode 1). But now the API returns NULL for episode name in cases like this. That's strange because the TV Show Humans has 3 seasons and all 3 seasons use the same Episode xx naming. Using the 1.1.21 add-on Season 1 scrapes with the correct episode titles but seasons 2 and 3 are missing the titles 1.1.22 is working perfectly, all episode titles are now correct. Big thanks for sorting this RE: TheMovieDB Python - TV Show scraper - Karlson2k - 2020-11-16 Still not working for me. Leia 18.9. Results always "No information found for item..." https://paste.kodi.tv/amizepahaq.kodi RE: TheMovieDB Python - TV Show scraper - Karellen - 2020-11-16 Hi @Karlson2k Not enough info. Do you have the full log thanks? Do you have nfo files in your tv show folder? RE: TheMovieDB Python - TV Show scraper - Karlson2k - 2020-11-17 (2020-11-16, 23:53)Karellen Wrote: Hi @Karlson2kI don't have .nfo files, but I haven't done full library rescan after changing scraper from non-Python version. I rescanned one of TV-shows and information updated just fine. It's a pity that I have to run a full rescan of library. Is it possible to pick-up TV-Show data from non-Python version? To clarify: I tried to scan two new files in directory that was scanned before with non-python version of scraper. Looks like new files were not added because TV-Show "url" from database wasn't recognized. RE: TheMovieDB Python - TV Show scraper - pkscout - 2020-11-17 (2020-11-16, 23:12)roidy Wrote:It's entirely possible that someone entered "Episode 1" for the title when adding season 1, but then someone else did no titles for season 2 and 3. The web site would automatically add titles for the other 2 seasons, so you can't tell at a glance. You'd have to do an API query for the episode before you'd know for sure. But at least now the scrapper deals with all of these changes. Thanks for the feedback.(2020-11-16, 20:53)pkscout Wrote: It looks like TMDb changed the way they are returning data in the API. Technically the show Normal People has no episode names. On the TMDb web site they show as Episode xx, and up until a few days ago, the API would return that same episode title (i.e. Episode 1). But now the API returns NULL for episode name in cases like this. I suspect I'll hold the update for a day or two to make sure nobody finds anything else and then submit the update to the official repo. RE: TheMovieDB Python - TV Show scraper - Karellen - 2020-11-17 (2020-11-17, 00:12)Karlson2k Wrote: but I haven't done full library rescan after changing scraper from non-Python version.You should not need to. I am assuming your previous scraper was TheMovieDB-TV Shows. If you were using TVDB, then yes you will need to Refresh the tv show (not the episodes) as the two <episodeguides> are incompatible between the two sites. If you can provide the full log, I can check. RE: TheMovieDB Python - TV Show scraper - Karlson2k - 2020-11-17 (2020-11-17, 00:25)Karellen Wrote:My previous scraper was "TheMovieDB (TV Shows)" non-Python version. No information was found for new files in existing (and scanned) directories.(2020-11-17, 00:12)Karlson2k Wrote: but I haven't done full library rescan after changing scraper from non-Python version.You should not need to. I am assuming your previous scraper was TheMovieDB-TV Shows. If you were using TVDB, then yes you will need to Refresh the tv show (not the episodes) as the two <episodeguides> are incompatible between the two sites. New scraper didn't pick-up old URL. It's clear from my log excerpt https://paste.kodi.tv/amizepahaq.kodi Now all files are scanned, I need to reproduce situation. It will take time. RE: TheMovieDB Python - TV Show scraper - Karlson2k - 2020-11-17 (2020-11-17, 00:25)Karellen Wrote:Here is the full log: https://paste.kodi.tv/wayoduwuzi.kodi(2020-11-17, 00:12)Karlson2k Wrote: but I haven't done full library rescan after changing scraper from non-Python version.You should not need to. I am assuming your previous scraper was TheMovieDB-TV Shows. If you were using TVDB, then yes you will need to Refresh the tv show (not the episodes) as the two <episodeguides> are incompatible between the two sites. Search for 'Tin Star'. The file "Tin.Star.S01E03" was not in database and is not added. RE: TheMovieDB Python - TV Show scraper - Karellen - 2020-11-17 Thanks @Karlson2k leave that with me for a bit. I want to discuss with pkscout |