Kodi Community Forum
Release TMDb TV Show scraper (Python - Default Matrix Scraper) - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27)
+---- Forum: Information Providers (scrapers) (https://forum.kodi.tv/forumdisplay.php?fid=147)
+----- Forum: TV Show Scrapers (https://forum.kodi.tv/forumdisplay.php?fid=305)
+----- Thread: Release TMDb TV Show scraper (Python - Default Matrix Scraper) (/showthread.php?tid=357232)

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47

RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - Karellen - 2023-04-28

(2023-04-28, 11:28)papampi Wrote:
Quote:
Is this enough:
https://gist.githubusercontent.com/papam...tmdb_error
Should be.
Looks like one for @pkscout

Also, all those artwork entries in your advancedsettings.xml will no longer work in v21 and later... https://github.com/xbmc/xbmc/pull/22748

RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - papampi - 2023-04-28

(2023-04-28, 12:08)Karellen Wrote:
(2023-04-28, 11:28)papampi Wrote:
Quote: 
Is this enough:
https://gist.githubusercontent.com/papam...tmdb_error
Should be.
Looks like one for @pkscout

Also, all those artwork entries in your advancedsettings.xml will no longer work in v21 and later... https://github.com/xbmc/xbmc/pull/22748

Thanks mate,
Will remove'em with v21.

RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - pkscout - 2023-04-28

(2023-04-28, 11:28)papampi Wrote:
(2023-04-28, 11:02)Karellen Wrote:
(2023-04-28, 10:23)papampi Wrote: Here is the log: https://pastebin.com/hpzxi7NU
Need the full log
Is this enough:
https://gist.githubusercontent.com/papampi/bff0d9cf761c350586a10059339f5fb1/raw/8e4ec464607c3f2ab9e8ef9c875566b1f74b0c21/kodi_tmdb_error
I can't duplicate the issue here, and that error log leads me to believe you are having network connectivity issues to YouTube.  The http response seems to be terminating unexpectedly in a way that can't really be corrected for when it's trying to validate the URL for the trailer.  I'd suggest turning off the option that enables the TV show trailer.

RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - papampi - 2023-04-29

(2023-04-28, 20:30)pkscout Wrote:
(2023-04-28, 11:28)papampi Wrote:
(2023-04-28, 11:02)Karellen Wrote: Need the full log
Is this enough:
https://gist.githubusercontent.com/papampi/bff0d9cf761c350586a10059339f5fb1/raw/8e4ec464607c3f2ab9e8ef9c875566b1f74b0c21/kodi_tmdb_error
I can't duplicate the issue here, and that error log leads me to believe you are having network connectivity issues to YouTube.  The http response seems to be terminating unexpectedly in a way that can't really be corrected for when it's trying to validate the URL for the trailer.  I'd suggest turning off the option that enables the TV show trailer.

Thanks,
Will disable it, lets see how it goes.

Edit:
Looks like that was it, haven't got any errors anymore.
Thanks.

RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - darkside40 - 2023-07-13

Hi there i have just a quick question too see if i simple overlooked something or if i am expecting something of the scraper which is simply not implemented.

I am trying to scrape Phineas and Ferb in the Production order, which works fine after i created a tvshow.nfo in the root folder of the show which contains the link to the production order site: https://www.themoviedb.org/tv/1877-phineas-and-ferb/episode_group/5f30d9417739410035ea03dc

Now i have a question. Althought i set the scraper to use German as language which also also available on the Website, it downloads titles, summaries etc. in english.
Are other languages in this case simply not implemented or am i doing something wrong?

RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - pkscout - 2023-07-13

(2023-07-13, 15:36)darkside40 Wrote: Now i have a question. Althought i set the scraper to use German as language which also also available on the Website, it downloads titles, summaries etc. in english.
Are other languages in this case simply not implemented or am i doing something wrong?

The scraper settings are a little different from other settings. The addon settings are basically a template for per source settings. If you setup the source and then changed the addon settings, those don’t affect an existing source. Go to your source and do EDIT SOURCE and the check the scraper settings there. If it doesn’t say German, change and save it.

If it does say German, I’d need to see a full debug log with the scrape to see if I can figure out what’s going on.

RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - darkside40 - 2023-07-13

Thanks for the hint, changed that to German.
Now title and summeries of the Episodes are in German, but also in the Aired Order and not in the Production Order anymore that i wanted.

RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - pkscout - 2023-07-13

(2023-07-13, 16:02)darkside40 Wrote: Thanks for the hint, changed that to German.
Now title and summeries of the Episodes are in German, but also in the Aired Order and not in the Production Order anymore that i wanted.

That change should not have affected the NFO file being used for sort order. Try removing the show from the library and rescraping it. If that doesn’t do it, I’ll need a debug log.

RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - darkside40 - 2023-07-14

To be honest i tried multiple times removing and rescraping the show. Now the scraper does not even pick up the tvshow.nfo anymore, at least it seems so because also in english it is in aired order right now.
I think the best would be to rename all the files to aired order which makes far less problems.

RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - burekas - 2023-07-23

Why this addon is so slow after pressing on the "Refresh" button.
The dialog comes only 10-20 secs after that.
With the previous one (xml version) it was much faster.

It's look like there are commnication or parsing issues.

* By the way, I made a fix to myself, I separated the Details and the Images languages.
Because sometimes I wand the descriptions in my language and the images as English. (So if you want it I can upload it to git)

RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - pkscout - 2023-07-23

(2023-07-23, 15:42)burekas Wrote: Why this addon is so slow after pressing on the "Refresh" button.
The dialog comes only 10-20 secs after that.
With the previous one (xml version) it was much faster.

It's look like there are commnication or parsing issues.

* By the way, I made a fix to myself, I separated the Details and the Images languages.
Because sometimes I wand the descriptions in my language and the images as English. (So if you want it I can upload it to git)
For your last thing - you're welcome to make a PR against the Nexus branch.  I'll review it and see if it makes sense to include in the general release.

As for the speed, we know the Python scrapers are currently slower than the XML ones.  Much of that is because Kodi has to spin up a Python interpreter multiple times every time a TV show is scraped, and that is a CPU intensive activity.  It's more noticeable on things like RPis than on a Windows or Mac computer.  Right now there is nothing to be done about that.  The Kodi core logic for scraping was built for the old XML scrapers.  Until we officially retire those, no changes to core can be made that would substantially improve things.  The XML scrapers are all marked broken now and (I believe) will be removed for Kodi 21.  That still means it'll be Kodi 22 at the earliest before you see changes in core and the scrapers.

RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - burekas - 2023-07-28

(2023-07-23, 21:45)pkscout Wrote:
(2023-07-23, 15:42)burekas Wrote: Why this addon is so slow after pressing on the "Refresh" button.
The dialog comes only 10-20 secs after that.
With the previous one (xml version) it was much faster.

It's look like there are commnication or parsing issues.

* By the way, I made a fix to myself, I separated the Details and the Images languages.
Because sometimes I wand the descriptions in my language and the images as English. (So if you want it I can upload it to git)
For your last thing - you're welcome to make a PR against the Nexus branch.  I'll review it and see if it makes sense to include in the general release.

As for the speed, we know the Python scrapers are currently slower than the XML ones.  Much of that is because Kodi has to spin up a Python interpreter multiple times every time a TV show is scraped, and that is a CPU intensive activity.  It's more noticeable on things like RPis than on a Windows or Mac computer.  Right now there is nothing to be done about that.  The Kodi core logic for scraping was built for the old XML scrapers.  Until we officially retire those, no changes to core can be made that would substantially improve things.  The XML scrapers are all marked broken now and (I believe) will be removed for Kodi 21.  That still means it'll be Kodi 22 at the earliest before you see changes in core and the scrapers.

If I added something to the language file, how can I duplicate it to all languages?
Is there any automation tool for that or should I do it manually? Or it's enough to update only the English file?

RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - pkscout - 2023-07-28

(2023-07-28, 13:16)burekas Wrote:
(2023-07-23, 21:45)pkscout Wrote:
(2023-07-23, 15:42)burekas Wrote: Why this addon is so slow after pressing on the "Refresh" button.
The dialog comes only 10-20 secs after that.
With the previous one (xml version) it was much faster.

It's look like there are commnication or parsing issues.

* By the way, I made a fix to myself, I separated the Details and the Images languages.
Because sometimes I wand the descriptions in my language and the images as English. (So if you want it I can upload it to git)
For your last thing - you're welcome to make a PR against the Nexus branch.  I'll review it and see if it makes sense to include in the general release.

As for the speed, we know the Python scrapers are currently slower than the XML ones.  Much of that is because Kodi has to spin up a Python interpreter multiple times every time a TV show is scraped, and that is a CPU intensive activity.  It's more noticeable on things like RPis than on a Windows or Mac computer.  Right now there is nothing to be done about that.  The Kodi core logic for scraping was built for the old XML scrapers.  Until we officially retire those, no changes to core can be made that would substantially improve things.  The XML scrapers are all marked broken now and (I believe) will be removed for Kodi 21.  That still means it'll be Kodi 22 at the earliest before you see changes in core and the scrapers.

If I added something to the language file, how can I duplicate it to all languages?
Is there any automation tool for that or should I do it manually? Or it's enough to update only the English file?
The default language file is en_gb.  If you add a string there, it will get used if the string is not available in other languages.  DO NOT duplicate the string to every language file.  If the PR is accepted, the en_gb update is pushed to the service we use for translations, and as folks translate the string it gets added to the addon.

RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - burekas - 2023-07-29

(2023-07-23, 21:45)pkscout Wrote:
(2023-07-23, 15:42)burekas Wrote: Why this addon is so slow after pressing on the "Refresh" button.
The dialog comes only 10-20 secs after that.
With the previous one (xml version) it was much faster.

It's look like there are commnication or parsing issues.

* By the way, I made a fix to myself, I separated the Details and the Images languages.
Because sometimes I wand the descriptions in my language and the images as English. (So if you want it I can upload it to git)
For your last thing - you're welcome to make a PR against the Nexus branch.  I'll review it and see if it makes sense to include in the general release.

Here is my PR:
https://github.com/xbmc/metadata.tvshows.themoviedb.org.python/pull/115

Thanks.

RE: TMDb TV Show scraper (Python - Default Matrix Scraper) - pkscout - 2023-07-29

(2023-04-28, 11:28)papampi Wrote:
(2023-04-28, 11:02)Karellen Wrote:
(2023-04-28, 10:23)papampi Wrote: Here is the log: https://pastebin.com/hpzxi7NU
Need the full log
Is this enough:
https://gist.githubusercontent.com/papampi/bff0d9cf761c350586a10059339f5fb1/raw/8e4ec464607c3f2ab9e8ef9c875566b1f74b0c21/kodi_tmdb_error
The add-on crashed with an unhandled exception because of a connection error to the the scraper site.  I could probably figure out how to trap for the error, but all that would do is cause the scraper to silently fail.  You'll need to figure out why you aren't getting a good connection to the scraper site.  Once that is addressed, the crashes will go away.