Release - TMDb TV Show scraper (Python - Default Matrix Scraper)

Release TMDb TV Show scraper (Python - Default Matrix Scraper) - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27)
+---- Forum: Information Providers (scrapers) (https://forum.kodi.tv/forumdisplay.php?fid=147)
+----- Forum: TV Show Scrapers (https://forum.kodi.tv/forumdisplay.php?fid=305)
+----- Thread: Release TMDb TV Show scraper (Python - Default Matrix Scraper) (/showthread.php?tid=357232)

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47

RE: TheMovieDB Python - TV Show scraper - pkscout - 2020-11-16

(2020-11-15, 09:29)Karellen Wrote:
(2020-11-15, 09:24)MeC!as Wrote: The Python scraper works, I apparently used an example with umlauts (mysteriöse Mordfälle). This one did not work.
Thanks. I have tried and it errors for me also. I thought we fixed this.

@pkscout anything can be done about this? Line 1254...https://paste.kodi.tv/ususoyanor.kodi

Well, I have good news, and I have bad news. The good news is that I fixed the crash with TV shows with extended characters. The bad news is that if you search TMDb using extended characters, you don't get any results from the API. You do though if you search on the web site using the search form, so I don't know what TMDb did so that their search works but not the API. Short of writing a mapping conversion for every extended character to an ASCII "equivalent" I'm not sure what to do yet. I'm going to keep poking at this though.

Oh, and I think you're thinking of the fix on the Extras addon for extended characters. I haven't had to address that with the scraper before.

RE: TheMovieDB Python - TV Show scraper - Karellen - 2020-11-16

(2020-11-16, 00:18)pkscout Wrote: Oh, and I think you're thinking of the fix on the Extras addon for extended characters. I haven't had to address that with the scraper before.

Yep, you are dead right, it was Extras I was thinking of. Thanks!!

RE: TheMovieDB Python - TV Show scraper - pkscout - 2020-11-16

(2020-11-16, 00:18)pkscout Wrote:
(2020-11-15, 09:29)Karellen Wrote:
(2020-11-15, 09:24)MeC!as Wrote: The Python scraper works, I apparently used an example with umlauts (mysteriöse Mordfälle). This one did not work.
Thanks. I have tried and it errors for me also. I thought we fixed this.

@pkscout anything can be done about this? Line 1254...https://paste.kodi.tv/ususoyanor.kodi
Well, I have good news, and I have bad news. The good news is that I fixed the crash with TV shows with extended characters. The bad news is that if you search TMDb using extended characters, you don't get any results from the API. You do though if you search on the web site using the search form, so I don't know what TMDb did so that their search works but not the API. Short of writing a mapping conversion for every extended character to an ASCII "equivalent" I'm not sure what to do yet. I'm going to keep poking at this though.

@Karellen @MeC!as I believe I have fixed this now (both the crash and the failed searches). I want to hold updates for now only because we just did an update, and the team members doing code review might kill me if I do another little update less than 24 hours later. For now you can get the updated scraper (for Leia or Matrix) at:

Leia: https://github.com/xbmc/metadata.tvshows.themoviedb.org.python/tree/leia
Matrix: https://github.com/xbmc/metadata.tvshows.themoviedb.org.python/tree/matrix

RE: TheMovieDB Python - TV Show scraper - MeC!as - 2020-11-16

Thanks a lot. I can confirm the search for titles with umlauts works as to expect. I could just test leia. Works nicely though. Blush

EDIT

There is a but.
It seems like now the scraper ignorers episodes where tmdb has no translation for yet. The fallback to English does not work.
I switched the scraper language to English, the last missing episodes get scraped.

RE: TheMovieDB Python - TV Show scraper - burekas - 2020-11-16

Hi,
@pkscout

I'm trying to use the last version 1.1.18
but when I activate the scraper on a Tv Show, kodi is crashing with the Error popup:
"Error: Exception caught on main loop. Exiting"

Is it a known issue?

RE: TheMovieDB Python - TV Show scraper - Karellen - 2020-11-16

(2020-11-16, 11:21)burekas Wrote: I'm trying to use the last version 1.1.18

I just tested with v1.1.19 and it is all working well. Check a couple of posts back and download the zip version.

RE: TheMovieDB Python - TV Show scraper - burekas - 2020-11-16

(2020-11-16, 11:35)Karellen Wrote:
(2020-11-16, 11:21)burekas Wrote: I'm trying to use the last version 1.1.18
I just tested with v1.1.19 and it is all working well. Check a couple of posts back and download the zip version.

With 1.1.19 I'm getting the same error with a new one :/

RE: TheMovieDB Python - TV Show scraper - ramis52 - 2020-11-16

Hi,

i have alot of Problems these days with the scraper. Attached you find a Log where i try to search for new Content on the TV Show "All American". Scraper is set to de-DE.

https://privatebin.net/?23253c6aaa6e696b#GzZAT2bzRF8gw9S7JfZFb2zdBpnpftpUPXFW2mstZS1x

RE: TheMovieDB Python - TV Show scraper - pkscout - 2020-11-16

(2020-11-16, 07:35)MeC!as Wrote: Thanks a lot. I can confirm the search for titles with umlauts works as to expect. I could just test leia. Works nicely though.

EDIT

There is a but.
It seems like now the scraper ignorers episodes where tmdb has no translation for yet. The fallback to English does not work.
I switched the scraper language to English, the last missing episodes get scraped.

Can you tell me which TV Show that is and which language you're using? Once I can duplicate it, it's easier to troubleshoot. Thanks.

RE: TheMovieDB Python - TV Show scraper - roidy - 2020-11-16

(2020-11-15, 23:28)pkscout Wrote:
(2020-11-15, 20:58)roidy Wrote:
(2020-11-15, 18:42)pkscout Wrote: I've fixed that, but I'm not submitting an update yet, as I have another bug I need to try and squash before submitting an update. In the interim, you can download updated addon from:

Leia: https://github.com/xbmc/metadata.tvshows.themoviedb.org.python/tree/leia
Matrix: https://github.com/xbmc/metadata.tvshows.themoviedb.org.python/tree/matrix

and install from zip.

Unfortunately I'm still getting the same error.
Could you please post an entire log with debug on? The first one didn't seem to be in debug mode and was cut off right where the error would have been reported, and this snippet is helping even less.

The problem is that the add-on generates so much logging information that a simple clean log with just starting Kodi, doing a Library update and then stopping Kodi generated a log that is 4.2MB in size and no paste site will host it. I can zip it up but do you have any idea where I can host it?

RE: TheMovieDB Python - TV Show scraper - pkscout - 2020-11-16

(2020-11-16, 17:31)ramis52 Wrote: Hi,

i have alot of Problems these days with the scraper. Attached you find a Log where i try to search for new Content on the TV Show "All American". Scraper is set to de-DE.

https://privatebin.net/?23253c6aaa6e696b#GzZAT2bzRF8gw9S7JfZFb2zdBpnpftpUPXFW2mstZS1x

It appears there were some undocumented changes to the API response from TMDb (sometimes episode name is not included in the response rather than being blank). I've got an update that I believe fixes this issue. Please try:

Matrix (v 1.3.4): https://github.com/xbmc/metadata.tvshows.themoviedb.org.python/tree/matrix
Leia (v 1.1.20): https://github.com/xbmc/metadata.tvshows.themoviedb.org.python/tree/leia

RE: TheMovieDB Python - TV Show scraper - pkscout - 2020-11-16

(2020-11-16, 18:45)roidy Wrote:
(2020-11-15, 23:28)pkscout Wrote:
(2020-11-15, 20:58)roidy Wrote: Unfortunately I'm still getting the same error.
Could you please post an entire log with debug on? The first one didn't seem to be in debug mode and was cut off right where the error would have been reported, and this snippet is helping even less.

The problem is that the add-on generates so much logging information that a simple clean log with just starting Kodi, doing a Library update and then stopping Kodi generated a log that is 4.2MB in size and no paste site will host it. I can zip it up but do you have any idea where I can host it?

There are two "modes" for debug logging with the scrapper. In the normal mode you enable debug logging only in Kodi, but NOT the setting in the scrapper that says "Verbose debug logging." If you enabled the verbose debug logging, you are definitely going to get a metric crap ton of data, so we ask folks not to enable that unless specifically asked. Sorry for the confusion. If you disable that in your source settings, that should help.

Having said all that, could you try the update I just posted about right above this (1.1.20 for Leia and 1.3.4 for Matrix). I think that might solve your issue.

RE: TheMovieDB Python - TV Show scraper - ramis52 - 2020-11-16

(2020-11-16, 19:11)pkscout Wrote:
(2020-11-16, 17:31)ramis52 Wrote: Hi,

i have alot of Problems these days with the scraper. Attached you find a Log where i try to search for new Content on the TV Show "All American". Scraper is set to de-DE.

https://privatebin.net/?23253c6aaa6e696b#GzZAT2bzRF8gw9S7JfZFb2zdBpnpftpUPXFW2mstZS1x
It appears there were some undocumented changes to the API response from TMDb (sometimes episode name is not included in the response rather than being blank). I've got an update that I believe fixes this issue. Please try:

Matrix (v 1.3.4): https://github.com/xbmc/metadata.tvshows.themoviedb.org.python/tree/matrix
Leia (v 1.1.20): https://github.com/xbmc/metadata.tvshows.themoviedb.org.python/tree/leia

thank you. i can confirm its working.

RE: TheMovieDB Python - TV Show scraper - roidy - 2020-11-16

@pkscout

Thanks, I tried 1.1.20 and the error is the same.

I have verbose debug turned off in the source settings and the main add-on settings but it still generates a 4.2MB log Sad

I've zipped it and posted it here:- https://gofile.io/d/hGOS7I

RE: TheMovieDB Python - TV Show scraper - MeC!as - 2020-11-16

(2020-11-16, 18:14)pkscout Wrote:
(2020-11-16, 07:35)MeC!as Wrote: Thanks a lot. I can confirm the search for titles with umlauts works as to expect. I could just test leia. Works nicely though.

EDIT

There is a but.
It seems like now the scraper ignorers episodes where tmdb has no translation for yet. The fallback to English does not work.
I switched the scraper language to English, the last missing episodes get scraped.
Can you tell me which TV Show that is and which language you're using? Once I can duplicate it, it's easier to troubleshoot. Thanks.

Yeah that was not really clear. Sorry for that.
I scrape all Episodes in German if available and the fallback in most cases will be English.
I checked with NCIS New Orleans Season 7 and Young Sheldon Season 4. There is just English (at the moment) available and no German translation yet.

But
It seems like that the new leia version 1.1.20 solved that issue.