+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27)
+---- Forum: Information Providers (scrapers) (https://forum.kodi.tv/forumdisplay.php?fid=147)
+----- Forum: TV Show Scrapers (https://forum.kodi.tv/forumdisplay.php?fid=305)
+----- Thread: Release TMDb TV Show scraper (Python - Default Matrix Scraper) (/showthread.php?tid=357232)
RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - roidy - 2021-04-07
(2021-04-07, 21:07)pkscout Wrote:from the api:- "status": "Returning Series"(2021-04-07, 21:01)roidy Wrote: @pkscoutI don't understand the question. What status are you talking about?
Quick question, is a shows status updated on a library update?
Returned in Kodi as ListItem.Status
RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - pkscout - 2021-04-07
(2021-04-07, 21:14)roidy Wrote:The scraper does set it (see line 235 of data_utils.py), but I don't think just doing a library update triggers that level of update for the show info. I think you would have to refresh each show that you wanted updated stati for.(2021-04-07, 21:07)pkscout Wrote:from the api:- "status": "Returning Series"(2021-04-07, 21:01)roidy Wrote: @pkscoutI don't understand the question. What status are you talking about?
Quick question, is a shows status updated on a library update?
Returned in Kodi as ListItem.Status
RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - roidy - 2021-04-07
(2021-04-07, 21:21)pkscout Wrote:Yep, it updates on a full show refresh but nobody is going to refresh each and every tv show they have just to update the status(2021-04-07, 21:14)roidy Wrote:The scraper does set it (see line 235 of data_utils.py), but I don't think just doing a library update triggers that level of update for the show info. I think you would have to refresh each show that you wanted updated stati for.(2021-04-07, 21:07)pkscout Wrote: I don't understand the question. What status are you talking about?from the api:- "status": "Returning Series"
Returned in Kodi as ListItem.Status
Is there no way to update it on standard library update?
Looking at the log during a standard library update the following api url's are called:-
Code:
2021-04-07 19:47:12.467 T:9320 DEBUG <general>: [metadata.tvshows.themoviedb.org.python (1.4.4)]: Calling URL "https://api.themoviedb.org/3/tv/95057?api_key=af3a53eb387d57fc935e9128468b1899&language=en-US&append_to_response=credits%2Ccontent_ratings%2Cexternal_ids%2Cimages%2Cvideos&include_image_language=en%2Cen%2Cnull"
2021-04-07 19:47:12.713 T:9320 DEBUG <general>: [metadata.tvshows.themoviedb.org.python (1.4.4)]: Calling URL "https://api.themoviedb.org/3/tv/95057/season/0?api_key=af3a53eb387d57fc935e9128468b1899&language=en-US&append_to_response=credits%2Cimages&include_image_language=en%2Cen%2Cnull"
2021-04-07 19:47:12.943 T:9320 DEBUG <general>: [metadata.tvshows.themoviedb.org.python (1.4.4)]: Calling URL "https://api.themoviedb.org/3/tv/95057/season/1?api_key=af3a53eb387d57fc935e9128468b1899&language=en-US&append_to_response=credits%2Cimages&include_image_language=en%2Cen%2Cnull"The first one being the full show details, so the information is there it should just be a matter of refreshing the value in Kodi's database.
RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - Karellen - 2021-04-07
(2021-04-07, 21:29)roidy Wrote: Is there no way to update it on standard library update?If you are referring to a first scrape of a particular tv show, then for me it works. I always get "returning series" or "ended"
RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - pkscout - 2021-04-07
(2021-04-07, 21:14)roidy Wrote: The first one being the full show details, so the information is there it should just be a matter of refreshing the value in Kodi's database.
That's not how library updates work. When a library update is triggered, Kodi looks for new or changed files/folders. For TV Shows, if it's an episode file that is new, then the updates happen in the context of the episode. Because of how the TMDb API works, the scraper has to pull some show information to get a full episode list, but the show's list item in Kodi (if I remember correctly) doesn't exist, so it can't be updated when an episode is updated. And I'm definitely not going to start doing direct writes to the database. Refreshing a show's info is the only way to update a show's info once it has been scraped. If you want that returning series information, you might look at this add-on:
https://forum.kodi.tv/showthread.php?tid=345847
If the skin supports it, then information like the returning series information is pulled real time.
RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - roidy - 2021-04-07
(2021-04-07, 21:50)Karellen Wrote:(2021-04-07, 21:29)roidy Wrote: Is there no way to update it on standard library update?If you are referring to a first scrape of a particular tv show, then for me it works. I always get "returning series" or "ended"
As I said in my last post, doing a full refresh of a show, which would also be the same as a "first scrape" then the value is set, that isn't the problem.
The problem is it never gets updated when you do a standard library update looking for new episodes.
This means if you "first scrape" a tv show and the status gets set to say "returning series" then months down the road the series gets cancelled or ends then the status would never get updated in the database.
RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - roidy - 2021-04-07
(2021-04-07, 21:54)pkscout Wrote:Thanks, I wasn't sure how the python scraper functioned during an episode update, I wrongly assumed it would allow updating the show info as well as the episode.(2021-04-07, 21:14)roidy Wrote: The first one being the full show details, so the information is there it should just be a matter of refreshing the value in Kodi's database.
That's not how library updates work. When a library update is triggered, Kodi looks for new or changed files/folders. For TV Shows, if it's an episode file that is new, then the updates happen in the context of the episode. Because of how the TMDb API works, the scraper has to pull some show information to get a full episode list, but the show's list item in Kodi (if I remember correctly) doesn't exist, so it can't be updated when an episode is updated. And I'm definitely not going to start doing direct writes to the database. Refreshing a show's info is the only way to update a show's info once it has been scraped. If you want that returning series information, you might look at this add-on:
https://forum.kodi.tv/showthread.php?tid=345847
If the skin supports it, then information like the returning series information is pulled real time.
RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - Karellen - 2021-04-07
(2021-04-07, 22:00)roidy Wrote: This means if you "first scrape" a tv show and the status gets set to say "returning series" then months down the road the series gets cancelled or ends then the status would never get updated in the database.Yea, all the tv show scrapers are the same. Adding new episodes won't change what is already in the library. Same as it won't update ratings. I think the reason for this is not to overwrite changes to the entry made by the user.
For these dynamic fields, I use the metadata editor add-on to change, or hop into the database and directly change them if I have to do a few edits.
RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - roidy - 2021-04-07
(2021-04-07, 22:14)Karellen Wrote:Do you know if the shows status could be changed using JSON-RPC?(2021-04-07, 22:00)roidy Wrote: This means if you "first scrape" a tv show and the status gets set to say "returning series" then months down the road the series gets cancelled or ends then the status would never get updated in the database.Yea, all the tv show scrapers are the same. Adding new episodes won't change what is already in the library. Same as it won't update ratings. I think the reason for this is not to overwrite changes to the entry made by the user.
For these dynamic fields, I use the metadata editor add-on to change, or hop into the database and directly change them if I have to do a few edits.
If so it could be a interesting little side project to write a service addon that checks a few shows every few days or may be every week and make the changes that way.
Answered my own question, yes you can:- https://kodi.wiki/view/JSON-RPC_API/v12#VideoLibrary.SetTVShowDetails
I feel a new project in the works
RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - aneurysm- - 2021-04-09
I'm going to ask a stupid question. So it's clear due to the api of themoviedb not giving back a fallback language (yet, but since it's an open request for a while now, don't see it changing soon) when there is no input in your chosen language, I've set English as the language instead of Dutch. The episode info in English doesn't bother me that much, but for the rest of the family here it would be nice if the show info was in Dutch.
So here the stupid question. I presume I can simply create a tvshow.nfo for my show with the info in Dutch in there ... it'll pick that up (with a new show, or if I refresh the show) .. and then every new episode that I add and is scraped, is simply scraped in the language set in the scraper (aka English here). Correct ?
RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - pkscout - 2021-04-09
(2021-04-09, 10:04)aneurysm- Wrote: I'm going to ask a stupid question. So it's clear due to the api of themoviedb not giving back a fallback language (yet, but since it's an open request for a while now, don't see it changing soon) when there is no input in your chosen language, I've set English as the language instead of Dutch. The episode info in English doesn't bother me that much, but for the rest of the family here it would be nice if the show info was in Dutch.That sounds right. The tvshow.nfo file will be used to create the metadata for the show. If that file doesn't include the episode guide, I'm pretty sure Kodi will ask the scraper to get it from the TMDb web site.
So here the stupid question. I presume I can simply create a tvshow.nfo for my show with the info in Dutch in there ... it'll pick that up (with a new show, or if I refresh the show) .. and then every new episode that I add and is scraped, is simply scraped in the language set in the scraper (aka English here). Correct ?
RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - Karellen - 2021-04-10
(2021-04-09, 17:36)pkscout Wrote: If that file doesn't include the episode guide, I'm pretty sure Kodi will ask the scraper to get it from the TMDb web site.I am not sure about your scraper, but the XML scrapers won't. It just errors in the log with "... check the nfo file contains an episodeguide..." or words to that effect.
Also if the wrong episodeguide is added, then it will still fail. But I think your scraper overcomes that.
RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - aneurysm- - 2021-04-11
(2021-04-10, 02:39)Karellen Wrote:I got it to work. Thanks guys! The episodeguide line is indeed needed. It didn't work at first, but that was my stupid mistake. I saw that in my test build I was not using your scraper, but the original themoviedb one. And the episodeguide tag in your scraper is different.(2021-04-09, 17:36)pkscout Wrote: If that file doesn't include the episode guide, I'm pretty sure Kodi will ask the scraper to get it from the TMDb web site.I am not sure about your scraper, but the XML scrapers won't. It just errors in the log with "... check the nfo file contains an episodeguide..." or words to that effect.
Also if the wrong episodeguide is added, then it will still fail. But I think your scraper overcomes that.
With you it's just (for instance) :
<episodeguide>2190</episodeguide>
While in the original one it's (for instance) :
<episodeguide> <url cache="tmdb-2190-en.json">http://api.themoviedb.org/3/tv/2190?api_key=6a5be4999abf74eba1f9a8311294c267&language=en</url> </episodeguide>
So at least I now have plot info in the language I want. Hopefully tmdb api will allow fallback language in the near future.
RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - kidbizkit - 2021-04-11
I gave up on this scraper and reverted back to the old TheMovieDB - TV Show scraper and everything worked as it should again for the TV Shows.
I don't have a problem with the python scraper for movies though.
That is after more than 5 re-installs of Kodi trying to figure out what is the problem.
RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - aneurysm- - 2021-04-11
@pkscout ...
I asked for this also in the metadata editor thread, but wanted to check with you as well .. since I'm not sure which of the 2 is the real "culprit"
So I load in a new show through the scraper. When I then use the metadata editor to add or change anything (tested with adding a rating and changing some text in plot), kodi won't scrape new episodes anymore.
I believe it has to do with the way the scraper writes episodeguide.
If I let kodi export to nfo (before using metadata editor).. I for instance get this in the nfo :
<episodeguide>2190</episodeguide>
After I use metadataeditor and update the nfo, it gets changed to :
<episodeguide>
<url cache="tmdb-2190-en.json">http://api.themoviedb.org/3/tv/2190?api_key=6a5be4999abf74eba1f9a8311294c267&amp;language=en</url>
</episodeguide>
And then the scraper won't pick up new episodes. To fix this, I can edit the nfo manually back to the first episodeguide, and then in kodi renew the show with the nfo file. Or I can tell kodi to renew all episodes, that works as well.
Neither is a good solution of course.
All this is on kodi 18.9 btw