Problem with movie scraping - probably version dependent

Problem with movie scraping - probably version dependent - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: General Support (https://forum.kodi.tv/forumdisplay.php?fid=111)
+---- Forum: OS independent / Other (https://forum.kodi.tv/forumdisplay.php?fid=228)
+---- Thread: Problem with movie scraping - probably version dependent (/showthread.php?tid=368180)

Problem with movie scraping - probably version dependent - SilenyKrecek - 2022-05-09

Hello,

tried to search forums, so if I missed a thread with the same issue, please just point me there Wink

Anyway... using for a looong time LibreElec+Kodi on RaspberryPi, currently Leia* on RPi4, without problems.
* yes, I have been lazy to go through an update as it works Smile

Recently I tried to install Kodi on Android tablet for my son, to let him watch kids stuff. All my movie/series stuff is on LAN on NAS in a shared folder, divided into some directories. Let's say I use source /media/KidMovies/... in there are directories for each movie, or, if there is a movie set, there is directory like "WinniePooh (collection)" and inside again directory for each movie.

In the movie directory, there is a video file of the actual movie, fanart.jpg, poster.jpg and movie.nfo

Movie.nfo's ARE NOT fully filled nfo. It is either parse type nfo with just URL to themoviedb.org, or combination type nfo, with URL on end and just some corrections like set or sorttitle as tags, because national characters are not sorted correctly in Kodi, unfortunately.

Problem is, on Android, when I set up this directory as the source, scraper either classic or Python version, and played with both with different settings, does not scrape info correctly. About half of movies are scraped OK, a few are scraped as a completely different movie, had the rest (also about half of the movies) are not scraped at all.

Thought it might be an Android issue, so I tried to install the latest Kodi on my Windows notebook, but the results were the same.

I tried to remove the same source on Raspberry, add it and rescan again, and it scraped correctly every single one.

I am out of ideas about what might be wrong. Anyone any clue for me? Please feel free to ask for more info as necessary.

Best regards,

SilenyKrecek

RE: Problem with movie scraping - probably version dependent - Klojum - 2022-05-09

(2022-05-09, 13:11)SilenyKrecek Wrote: Movie.nfo's ARE NOT fully filled nfo. It is either parse type nfo with just URL to themoviedb.org, or combination type nfo, with URL on end and just some corrections like set or sorttitle as tags, because national characters are not sorted correctly in Kodi, unfortunately.

An example of such a partially filled nfo file would be useful. Upload it on the paste webservice of your choice so we can have a look.

RE: Problem with movie scraping - probably version dependent - SilenyKrecek - 2022-05-09

(2022-05-09, 13:26)Klojum Wrote: An example of such a partially filled nfo file would be useful. Upload it on the paste webservice of your choice so we can have a look.

This is unscraped one. Second one in set is scraped ok and third again not scraped. All nfos look same to me, did not found any pattern what is scraped and what's not... Sad

https://paste.kodi.tv/erimafokub

RE: Problem with movie scraping - probably version dependent - Karellen - 2022-05-09

(2022-05-09, 13:11)SilenyKrecek Wrote: there is a video file of the actual movie, fanart.jpg, poster.jpg and movie.nfo

Section 2 probably has your answer... https://kodi.wiki/view/NFO_files/Movies

Also, in your nfo file, the clear="" attribute is not used for those items, it is used for any tag that allows multiple entries like Genre, Tags and I think Studio.

RE: Problem with movie scraping - probably version dependent - SilenyKrecek - 2022-05-09

(2022-05-09, 21:15)Karellen Wrote: Section 2 probably has your answer... https://kodi.wiki/view/NFO_files/Movies

Also, in your nfo file, the clear="" attribute is not used for those items, it is used for any tag that allows multiple entries like Genre, Tags and I think Studio.

Hello Karelen,
let me start with information, that I downloaded Leia 18.9 to my PC and it got scraped the whole source 100% OK, nothing missing, nothing wrong, so, these nfos and official Kodi scraper (The Movie Database, set to separate directories:on and scan recursively:on) included in 18.9 have no problem with my directory structure or files. Also, fanarts and posters are used from the movie directory, as expected.

So, definitely, it is version related problem (in my case).

While using latest (From Play Store it gets 19.3, so I installed on PC too):

Which information from Section 2 You think is related to my problem? movie.nfo should be OK, as I read that wiki page - as long as separate directories option is on and no <filename>.nfo is present, movie.nfo is expected to be used. Which in some cases (about half of movies) IS positively used (because it contains movie name modification - slightly modified movie name than in online source). It does not matter if movie is in set or not. Some movies are scraped from "collection" subdirectories.

Removing clear="true" attributes had no effect. Also other scraped movies have same structure of nfo as this unscraped one.

What should I try next, on v19?

RE: Problem with movie scraping - probably version dependent - Karellen - 2022-05-09

First you should fix the nfo file like this...

xml:
<movie>

	<sorttitle>Jak vycviczit draka 1</sorttitle>

    <set>

        <name>Jak vycvičit draka</name>

        <overview>About a dragon blah, blah.</overview>

    </set>

</movie>

http://www.themoviedb.org/movie/10191-how-to-train-your-dragon

So the problem is that the nfo files are being ignored? Provide a Debug Log that captures you trying to scrape the movie where the nfo file is ignored.

RE: Problem with movie scraping - probably version dependent - SilenyKrecek - 2022-05-09

Roger, will modify nfo and try to get logs (never did that yet). Tomorrow. it is almost midnight here.
Thanks for Your help so far!

About nfo - interesting thing is, that How to train Your dragon 2 scrapes OK with the same nfo, directory and filenaming structure. I.e. it is under the same directory "How to train your dragon (collection)" in its own subdirectory. Only movie.nfo content difference is in sorttitle tag (it has 2 instead of 1 inside) and of course in URL.Othervise is identical, and scraps. (Of course directory and file names are in Czech, but You get the point Wink

)

RE: Problem with movie scraping - probably version dependent - SilenyKrecek - 2022-05-11

Hello Karellen,

I tried to do that logging yesterday evening.

Which settings should be enabled to get You relevant information? In settings, there are: switch "Enable debug logging", "Enable component-specific logging", "Enable event logging" and subswitch "Enable notification event logging". Which components should be logged in the second option?

I did enable debug, in components I chose Database and SMB, and enabled event logging.

Nothing else changed. Then I ran "Update library". And it scanned the whole source 100% OK! You can imagine my bewilderment...

I thought about it until I fell asleep. Is there a possibility, that 19.x uses different API than 18.x to communicate with tmdb and that this API was temporary malfunctioning? That is the only thing which I thought of which might explain why it was not working and now it works on the very same and unchanged system...

At least I learned about logging. So, please at least guide me on which logs/components are relevant in case it happens again, so I will then be able to come "armed" with logs to begin with...

RE: Problem with movie scraping - probably version dependent - Karellen - 2022-05-11

(2022-05-11, 08:45)SilenyKrecek Wrote: Enable debug logging

That is all you need to enable. We don't need component logging, but enabling database and smb does not hurt. Some of the other options explode the logging and it becomes too big and confusing.

Same API for all versions of the scraper. It could be a temporary glitch at TMDB, which has happened to me- it doesn't work, then 20 minutes later its all fine. Using a VPN causes problems, or if you are in Russia as the API has been blocked there.

But great to hear it is all working again!!

RE: Problem with movie scraping - probably version dependent - SilenyKrecek - 2022-05-11

I am from the Czech Republic. Fortunately, no need to use VPN here, though the government did block some websites recently saying it spreads false information (that approach I personally disapprove, as blocking might be misused for true censorship).

Thanks for Your effort and time! Next time I will have logs Wink

I am happy it works too because I am a long time happy Kodi user.