+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27)
+---- Forum: Information Providers (scrapers) (https://forum.kodi.tv/forumdisplay.php?fid=147)
+----- Forum: Movie Scrapers (https://forum.kodi.tv/forumdisplay.php?fid=302)
+----- Thread: Release Universal Movie Scraper (/showthread.php?tid=129821)
Pages:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
RE: [Release] Universal Movie Scraper - hac_ - 2017-06-23
(2017-06-03, 19:57)scudlee Wrote: html/xml tags are automatically cleaned out by default.
You'd need to add (e.g.) noclean="1" as an attribute to the <expression> for that not to happen (on the first capturing group).
Would probably do it.Code:<RegExp input="$$1" output="<outline>\1</outline>" dest="2">
<expression fixchars="1" trim="1">itemprop="description">(.+?)</div></expression>
</RegExp>
Sorry, I forgot about this for a while.
Thank you, your fix works. I don't have Sully in my library anymore, but Little Dieter Needs to Fly and Fog of War got fixed after I used your regexp in the plugin.
Edit: Helps also with Citizenfour. I checked some movies that were correct before and they seem to be fine also with this new version.
RE: [Release] Universal Movie Scraper - Mordred - 2017-08-06
Is anyone able to get this scraper to grab posters/fanart from fanart.tv? I've enabled both, and still the only images I get are from themoviedb.org. I've tried with multiple movies now and nothing ever comes up. I disabled tmdb in the fanart section -- leaving fanart.tv enabled -- just to see if there was some precedence thing, and my movies suddenly had no fanart.
I tried Ghost in the Shell, Raw, Moana, and Genius (2016) which seems like a representative sample. All had fanart available on fanart.tv and none ever got added to the database.
RE: [Release] Universal Movie Scraper - Karellen - 2017-08-06
@Mordred
Look through this thread. Issue is discussed and a solution is available... https://forum.kodi.tv/showthread.php?tid=114633&pid=2610986#pid2610986
RE: [Release] Universal Movie Scraper - Mordred - 2017-08-08
Hey Karellan. I looked at that thread, and I don't think it's solving my problem. I went and updated the metadata.common.fanart.tv scraper as I think it was out of date, but I'm still not having any luck. After a restart and trying to refresh data for Ghost in the Shell (2017):
Code:
22:31:07.404 T:11644 DEBUG: scraper: GetFanartTvFanartByIdChain returned <details><url function="ParseFanartTvFanart" cache="fanarttv-tt1219827.json">http://webservice.fanart.tv/v3/movies/tt1219827?api_key=ed4b784f97227358b31ca4dd966a04f1</url></details>
22:31:07.405 T:11644 DEBUG: scraper: ParseFanartTvFanart returned <details><fanart></fanart></details>the URL works, and I even checked the cached file on a whim, and it has the same data.
RE: [Release] Universal Movie Scraper - Karellen - 2017-08-08
Hi Mordred,
How do you have the scraper setup?
- Plot info is coming from IMDB or TMDB?
- In Posters, you specifically have fanart.tv enabled?
- In Fanart, you specifically have fanart.tv enabled?
- If you select TMDB as the provider for fanart and posters, does it work?
- Can you provide a debug log recording you trying to scrape the movie.
RE: [Release] Universal Movie Scraper - Mordred - 2017-08-08
1. IMDB (cast from TMDB though)
2. Yes
3. Yes
4. Yes, when TMDB is enabled it gets TMDB fanart and posters (whether fanart.tv is enabled or not) and when disabled I get nothing.
5. https://pastebin.com/7p0wtp3U
RE: [Release] Universal Movie Scraper - Mordred - 2017-08-11
@Karellen was that log any help?
RE: [Release] Universal Movie Scraper - olympia - 2017-08-14
Yes, fanarttv scraping is broken due to the api switched to https. I wonder for how long this is the case...
Will submit a fix now.
You will need fanarttv scraper library v3.1.5.
RE: [Release] Universal Movie Scraper - wgstarks - 2017-08-14
(2017-08-14, 13:53)olympia Wrote: Yes, fanarttv scraping is broken due to the api switched to https. I wonder for how long this is the case...
Will submit a fix now.
You will need fanarttv scraper library v3.1.5.
Will this also fix downloading from TMDB? Or do we still need to use the developer API's.
RE: [Release] Universal Movie Scraper - olympia - 2017-08-14
(2017-08-14, 14:04)wgstarks Wrote:(2017-08-14, 13:53)olympia Wrote: Yes, fanarttv scraping is broken due to the api switched to https. I wonder for how long this is the case...
Will submit a fix now.
You will need fanarttv scraper library v3.1.5.
Will this also fix downloading from TMDB? Or do we still need to use the developer API's.
I am not aware of this issue, could you refer me to the discussion around this?
RE: [Release] Universal Movie Scraper - wgstarks - 2017-08-14
(2017-08-14, 14:15)olympia Wrote:(2017-08-14, 14:04)wgstarks Wrote:(2017-08-14, 13:53)olympia Wrote: Yes, fanarttv scraping is broken due to the api switched to https. I wonder for how long this is the case...
Will submit a fix now.
You will need fanarttv scraper library v3.1.5.
Will this also fix downloading from TMDB? Or do we still need to use the developer API's.
I am not aware of this issue, could you refer me to the discussion around this?
I don't know what I'm thinking. Too early in the morning here and I haven't finished my coffee yet.
I think the cause may be the same, but it's an Artwork Downloader issue. Starts at post 2740 (ish).
RE: [Release] Universal Movie Scraper - willemd - 2017-09-06
(2017-05-31, 12:03)willemd Wrote: I think the scraper has a small bug in reading IMDb plot outlines which contain a hyperlink.I think I found another bug in the plot outlines/summary department.
For example: https://www.imdb.com/title/tt3263904/
The plot outline ("The story of Chesley Sullenberger, an American pilot who became a hero ... etc. etc.") contains a link to Chesley Sullenberger as a character (or a real person in the IMDb database). The scraper discards the link and anything after it. So the plot outline in the Kodi database becomes "The story of ". Bad, bad behaviour!
For now I've solved it with making .nfo files with an <outline> entry, but I'm not sure all Kodi users will be able to do so.
The skin I use (Mimic 4) shows plot outlines in the movies lists. However, on the IMDb website, those movies without a short plot outline are shown with the first few senteces of the longer summary, which is followed by 'See full summary >>'. This is only with less known movies. See the screenshot here:
The scraper then scrapes that undesired sentence as part of the plot outline, which shows up in Kodi like this:
I think this is undesired behaviour but it might be easily fixed? (I'm no programmer so that's a guess!)
So it's a small problem which only shows up with these two conditions:
1) The movie only has a longer plot summary, no short plot outline. (So, only the lesser visited movies on IMDb.)
2) The Kodi skin shows the plot outlines instead of the summary.
Just my 2 cents
Edit: I just realized, perhaps these two bugs are related. By allowing hyperlinks (to character names) in the plot outlines, this might just have enabled these links to 'See full summary' to show up in the Kodi database?)
RE: [Release] Universal Movie Scraper - seth.feinberg - 2017-09-12
I had to readd my media recently and I guess I screwed up one of the Universal Movie Scraper options, cause now my foreign movies have foreign names, instead of the English equivalent's I'm used to and want (i.e. shows Lola Rentt and not my preferred Run Lola Run). Since I THOUGHT I knew what I was doing and obviously didn't I'll just pretend I have NO idea what I'm talking about:
- Exactly what settings do I set to force Universal Movie Scraper to use the English title and not the Foreign title?
- What are the exact steps for fixing all the movies that are currently displaying the foreign movie title? The easiest way that is. Is it a change in the "Set Content" settings? If I make those changes and changes in the Universal Movie Scraper settings can I just refresh each offending movie from "Movie Information" in the Context Menu? Is there a way for me to refresh all of them once the settings are right? Or do I have to do it individually?
RE: [Release] Universal Movie Scraper - Karellen - 2017-09-12
@seth.feinberg
That's painful, having to rebuild the library. You should consider running this every now and then... http://kodi.wiki/view/Import-Export_library . Once you have your library built exactly as you want it, then just Export the library to Separate Files. If you ever need to rebuild the library, it will re-create your library exactly as you had it. Even resume points, play counts etc...
1. Go to the scraper settings in "Change Content" of the context menu. In General tab, ensure "Preferred Title Language from" is set to USA
2. When you make the change and save it, you should be asked "Do you want to refresh all items..." Selecting Yes should rescrape that source with the new settings.
RE: [Release] Universal Movie Scraper - seth.feinberg - 2017-09-13
(2017-09-12, 22:57)Karellen Wrote: @seth.feinberg
That's painful, having to rebuild the library. You should consider running this every now and then... http://kodi.wiki/view/Import-Export_library . Once you have your library built exactly as you want it, then just Export the library to Separate Files. If you ever need to rebuild the library, it will re-create your library exactly as you had it. Even resume points, play counts etc...
1. Go to the scraper settings in "Change Content" of the context menu. In General tab, ensure "Preferred Title Language from" is set to USA
2. When you make the change and save it, you should be asked "Do you want to refresh all items..." Selecting Yes should rescrape that source with the new settings.
I have done that a few times, but I guess it had been a long time since I last did it. I haven't fully checked but I'm guessing the metadata I lost is those files I changed since my last "export to individual files" so hopefully there aren't THAT many titles to fix. I'll try this tonight, thanks so much!