Release TMDb TV Show scraper (Python - Default Matrix Scraper) - Printable Version +- Kodi Community Forum (https://forum.kodi.tv) +-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33) +--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27) +---- Forum: Information Providers (scrapers) (https://forum.kodi.tv/forumdisplay.php?fid=147) +----- Forum: TV Show Scrapers (https://forum.kodi.tv/forumdisplay.php?fid=305) +----- Thread: Release TMDb TV Show scraper (Python - Default Matrix Scraper) (/showthread.php?tid=357232) |
RE: TheMovieDB Python - TV Show scraper - pkscout - 2020-11-17 (2020-11-17, 16:37)jads Wrote: Just as an aside, it might also be safer to sort the episodes list by the 'order' key before sequentially numbering them. While I haven't seen any cases where the API returns episodes in a different order (and unlike with the season groups, you can't explicitly set or edit the order #) I'm not sure it's safe to assume the API will actually return the episode list in a particular order.Good point. I hadn't really noticed that order field in the API return until you pointed it out. I've made that update, and it'll be included in the next release. RE: TheMovieDB Python - TV Show scraper - Karlson2k - 2020-11-18 (2020-11-17, 20:17)pkscout Wrote:(2020-11-17, 13:59)Karlson2k Wrote: @pkscout, why this scraper put temporally files to ".kodi/userdata/addon_data/metadata.tvshows.themoviedb.org.python/cache" instead of standard location ".kodi/temp/scrapers/metadata.tvshows.themoviedb.org.python"?I'm not familiar with the temp location. I use a python call to get the addon_data location and then store the addon's data there. If there is some other Python call that will return the path to that temp directory on every platform, I'd be happy to change it. I'd rather not start doing platform specific code though. As an aside, I'm using the same location the TV Maze scraper uses to store it's cache files. I'm not an addon API expert, but by quick checking of kodi source code, it seems that simplest way to use temp path is something like os.path.join('special://temp/scrapers', ADDON.getAddonInfo('id')). This path is automatically created before run of scraper and cleared from outdated items (no need to check for old files in your code). Expire time is defined in addon.xml in <addon> <extension cachepersistence="00:15"/></addon>. RE: TheMovieDB Python - TV Show scraper - pkscout - 2020-11-18 (2020-11-18, 18:10)Karlson2k Wrote:Thanks. I'll definitely take a look at that.(2020-11-17, 20:17)pkscout Wrote:(2020-11-17, 13:59)Karlson2k Wrote: @pkscout, why this scraper put temporally files to ".kodi/userdata/addon_data/metadata.tvshows.themoviedb.org.python/cache" instead of standard location ".kodi/temp/scrapers/metadata.tvshows.themoviedb.org.python"?I'm not familiar with the temp location. I use a python call to get the addon_data location and then store the addon's data there. If there is some other Python call that will return the path to that temp directory on every platform, I'd be happy to change it. I'd rather not start doing platform specific code though. As an aside, I'm using the same location the TV Maze scraper uses to store it's cache files. RE: TheMovieDB Python - TV Show scraper - Hebotsuki - 2020-11-18 (2020-11-17, 02:24)pkscout Wrote:(2020-11-16, 21:26)Hebotsuki Wrote:Writer is the only type of crew that is added in Kodi. That's to keep it consistent with the movie scraper. I can look at adding the other ones, but it's a bit of a whack-a-mole problem. Anytime another kind of writer is added, I'd have to update the scraper. I'll take another look though. There might be a way to do it using "department" instead of roles.(2020-11-15, 05:34)pkscout Wrote: Just FYI that there is an update available from the repo now for this. If you don't get it automatically, you can force check for updates, and that should download it.Thanks, @pkscout. I installed the latest version on leia and seems to work fine: actors, episodes, fallback to English works with titles and plot with no translation, etc. I don't see any missing information. I only have one question: which writing credits are scraped for the episodes? I tested a couple of TV shows and one of them doesn't scrape the writing credits, but in TMDb there are "screenplay" and "storyboard" for the writing credits. The other one only scraped the "writer" credits, but no the "stroryboard" writing credit, which is in TMDb too. Much appreciated. If there were a way to get a "department" writing credits would be great. I don't know how scrapers work, but the XML version of the scraper gets these credits I mentioned. I'm interested in this because many anime shows use this kind of credits and and even many have no writer role. [split] TheMovieDB - TV Show scraper - montezuma - 2020-11-18 (2020-11-18, 21:40)Karellen Wrote:Thanks. I have installed the Python scraper, but it also fails with:(2020-11-18, 21:23)montezuma Wrote: Sorry.Ok, and yes I can confirm this problem also. I suspect the issue is due to none of the episodes having an overview. https://www.themoviedb.org/tv/47877-5-dni-z-zycia-emeryta Here are the logs: https://paste.kodi.tv/igijuqovup.kodi 0-11-18 21:32:43.369 T:1806672768 DEBUG: [metadata.tvshows.themoviedb.org.python (1.1.18)]: Calling URL "https://api.themoviedb.org/3/search/tv?query=s01e01+Smierc+pana+Kaliny&api_key=af3a53eb387d57fc935e9128468b1899&language=pl-PL" 2020-11-18 21:32:43.584 T:1806672768 DEBUG: [metadata.tvshows.themoviedb.org.python (1.1.18)]: the api response: {u'page': 1, u'results': [], u'total_pages': 0, u'total_results': 0} RE: TheMovieDB - TV Show scraper - Karellen - 2020-11-18 @montezuma I have moved you to the correct thread. (2020-11-18, 22:41)montezuma Wrote: Thanks. I have installed the Python scraper, but it also fails with:It works fine here for me. What I can see in the log is it is not a scraper problem. In the below, which folder have you set as the Source... /media/HDD6/Seriale/5 dni z zycia emeryta [1985] (PL)/s01e01 Smierc pana Kaliny.mp4
RE: TheMovieDB - TV Show scraper - montezuma - 2020-11-19 (2020-11-18, 23:33)Karellen Wrote: @montezuma> What I can see in the log is it is not a scraper problem. In the below, which folder have you set as the Source... "5 dni z zycia emeryta [1985] (PL)" What is actually wrong with the folder name (as I said it worked ~2 weeks ago) ? https://kodi.wiki/view/Naming_video_files/TV_shows Naming TV Shows follows the same guidelines as for Movies. Name the TV Show folder the same way as shown on the TV Show page at the scraper site. It is always best practice to include the year along with the title. This makes scraping more accurate and differentiates between original and remake versions. This guideline also applies to Country of Origin TV Shows. For example Hells Kitchen (UK), Hells Kitchen (US), Hells Kitchen (AU), Hells Kitchen (IT) etc. The title and (year) displayed at the scraper site is how you should name your show. Anyway I use the "mixed" approach and provide a URL for every TV show in the tvshow.nfo file (to avoid disambiguity): <?xml version="1.0" encoding="UTF-8" standalone="yes" ?> <tvshow> <title>5 dni z życia emeryta (PL)</title> <tag>PL</tag> </tvshow> https://www.themoviedb.org/tv/47877-5-dni-z-zycia-emeryta And btw. I could scrap today the TV show with the XML scraper 3.5.8 RE: TheMovieDB - TV Show scraper - montezuma - 2020-11-19 (2020-11-19, 08:55)montezuma Wrote: And btw. I could scrap today the TV show with the XML scraper 3.5.8I also tried the Python scraper and it works, too (previously I selected the folder with the TV show for scraping (my mistake) and not a folder containing the folder with the TV show). However from 32 TV shows I have on the disk, I could only scrape 22 with the Python scraper, and the XML scraper 3.5.8 managed all 32. RE: TheMovieDB Python - TV Show scraper - Karellen - 2020-11-19 (2020-11-19, 08:55)montezuma Wrote: > What I can see in the log is it is not a scraper problem. In the below, which folder have you set as the Source...There is nothing wrong with the folder name, but you have set the source on the wrong location. The Source should be set on /Seriale/. But it sounds like you figured that out. (2020-11-19, 09:18)montezuma Wrote: However from 32 TV shows I have on the disk, I could only scrape 22 with the Python scraper, and the XML scraper 3.5.8 managed all 32.Without any details, there is nothing I can do about that. [split] TheMovieDB - TV Show scraper - Hebotsuki - 2020-11-19 (2020-11-18, 23:45)Karellen Wrote: @Hebotsuki @Karellen I tried the Python scraper as asked:
RE: TheMovieDB - TV Show scraper - Karellen - 2020-11-19 (2020-11-19, 19:56)Hebotsuki Wrote: I tried the Python scraper as asked:That is not a scraper issue. That is MySQL being difficult again. I have seen this problem a few time already, but never been able to figure out the cause. The Kodi/MySQL combination has a little glitch and the episode files are not seen or scanned, but if you were to switch over to local SQL, it would work perfectly. Your show is here... smb://192.168.1.100/Animoteca/TV/Violet Evergarden (2018)/ You just need to change the path so Kodi reads it as a new tv show and starts the scan fresh. Maybe smb://192.168.1.100/Animoteca/TV/Violet Evergarden Special (2018)/ which is a valid alternate title at TMDB.Let us know if that works. RE: TheMovieDB - TV Show scraper - Hebotsuki - 2020-11-19 (2020-11-19, 20:26)Karellen Wrote:I tried renaming the folder as suggested, but, unfortunately, it didn't work. Before scraping, I cleaned the libray, just in case, but it only gets the special episode. Nor the Python scraper worked.(2020-11-19, 19:56)Hebotsuki Wrote: I tried the Python scraper as asked:That is not a scraper issue. That is MySQL being difficult again. I had scraped the show before without any issue, but I changed the files -same name but different extension-. Could it be related? RE: TheMovieDB Python - TV Show scraper - Karellen - 2020-11-19 (2020-11-19, 22:36)Hebotsuki Wrote: I tried renaming the folder as suggested, but, unfortunately, it didn't work.Do you have the Debug Log? Can you also post a screenshot of the Season 1 folder so I can clearly see every file in there. (2020-11-19, 22:36)Hebotsuki Wrote: I had scraped the show before without any issue, but I changed the files -same name but different extension-. Could it be related?No, not now that you changed the name of the tv show folder. The episodes are not being seen by the scanner, so the scraper is not called. Can you scan it with the default SQL setup? RE: TheMovieDB Python - TV Show scraper - pkscout - 2020-11-20 (2020-11-16, 21:26)Hebotsuki Wrote:Do you have a couple show names I can use for testing where someone has the roll screenplay or storyboard? I think I see how to use the department rather than the role, but without a couple shows with which to test, I won't know if this works.(2020-11-15, 05:34)pkscout Wrote:Thanks, @pkscout. I installed the latest version on leia and seems to work fine: actors, episodes, fallback to English works with titles and plot with no translation, etc. I don't see any missing information. I only have one question: which writing credits are scraped for the episodes? I tested a couple of TV shows and one of them doesn't scrape the writing credits, but in TMDb there are "screenplay" and "storyboard" for the writing credits. The other one only scraped the "writer" credits, but no the "stroryboard" writing credit, which is in TMDb too.(2020-11-14, 20:58)Hebotsuki Wrote: It's not working for me either. According to Travis from TheMoviedb, some new fileds have been added to the credits in order to a future support for translated person names. That's probably what broke the scraper. I guess the scraper developpers are already aware of the problem.Just FYI that there is an update available from the repo now for this. If you don't get it automatically, you can force check for updates, and that should download it. RE: TheMovieDB Python - TV Show scraper - Hebotsuki - 2020-11-20 (2020-11-19, 23:18)Karellen Wrote:(2020-11-19, 22:36)Hebotsuki Wrote: I tried renaming the folder as suggested, but, unfortunately, it didn't work.Do you have the Debug Log? Can you also post a screenshot of the Season 1 folder so I can clearly see every file in there. Here is the debug log. And this is a screenshot of season 1 folder: By the way, when I navigate through the Video>Archives to the show folder, there aren't files showing up, so what you say about Kodi not seeing the files is true. These are screenshots of the video archives: So, having in mind what you said about Kodi not seeing the files, I decided to create a fake video with the name "Violet Evergarden - S01E01 - This is a fake video.mp4" and this fake file did show up in the Video archives: If I scrape again the show with all the episodes I have plus this fake video, then it scrapes the fake episode!!! So it's not an issue with the scraper nor with the database, but with my files. For whatever reason, Kodi doesn't recognise my files. Considering this, I tried moving the files to another HD and the moving them again to the show folder and now Kodi are seeing them!!! That's quite strange, it seems an issue with the file system or something. Anyway, after doing that, I can now scrape the show even keeping the show name "Violet Evergarden". @Karellen, @olympia, much appreciated your help and time to fix the problem!!! And sorry for the long post!! |