Login at Kodi Home

ultrabrutal · 2009-01-12, 10:12

Just to take one, imdb:

IMDB Conditions

"Robots and Screen Scraping: You may not use data mining, robots, screen scraping, or similar data gathering and extraction tools on this site, except with our express written consent as noted below."

Do XBMC have this permission? Just curious...

**spiff** · 2009-01-12, 13:37

no, i was not aware of those terms (or atleast i could claim so).

not so much any longer. r17032 disables the scraper

ultrabrutal · 2009-01-12, 14:47

Shit! What a party pooper I turned out be Sad

I'm afraid that most sites have conditions like this so most scrapers are in fact violating terms of the site

theuni · 2009-01-12, 15:59

Quote:We do allow the limited use of robots and crawlers, such as those from certain search engines, with our express written consent. If you are interested in receiving our express written permission to use robots or crawlers on our site, please contact our Licensing Department.

I'd be happy to kick-off communications. I'd think this is the type of project that they would be open to.

Spiff: Let me know if this is something I should look into, or if it's up to the XBMC team to handle.

TheUni

**spiff** · 2009-01-12, 16:01

i think we will have to handle this ourself.

thanks for the offer though

ultrabrutal · 2009-01-12, 16:19

I'm afraid we eventually have to create our own movie database ala the mymovies database which we can all contribute to Sad

**spiff** · 2009-01-12, 16:20

themoviedb.org is just that. i have already written the scraper a few months ago, but as the site had so little info i didn't bother with polishing it.

**pike** · 2009-01-12, 18:21

Let me quote Conan O Brien "Nicely Done" Cool

ultrabrutal Wrote:Just to take one, imdb:

IMDB Conditions

"Robots and Screen Scraping: You may not use data mining, robots, screen scraping, or similar data gathering and extraction tools on this site, except with our express written consent as noted below."

Do XBMC have this permission? Just curious...

ultrabrutal · 2009-01-12, 18:32

Pike, when commitmonitor showed me the commits spiff had done I felt really bad. I use the scrapers also hehe. Of course XBMC cannot violate any licenses, so I totally get spiff.

spiff, I didn't know about themoviedb.org. Do you use an api or really scrape the html? I don't see an imdb link, so you use the movie name as a param?

**spiff** · 2009-01-12, 18:41

it's an xml based api.

see r17035

**mkortstiege** · 2009-01-12, 18:43

ultrabrutal Wrote:Pike, when commitmonitor showed me the commits spiff had done I felt really bad. I use the scrapers also hehe. Of course XBMC cannot violate any licenses, so I totally get spiff.

spiff, I didn't know about themoviedb.org. Do you use an api or really scrape the html? I don't see an imdb link, so you use the movie name as a param?

A basic themoviedb.org scraper just hit the SVN in revision 17035. Searches and IMDB-ID translation is done via their API.

FYI, it's the new default movie scraper ..

ultrabrutal · 2009-01-12, 18:49

The first thing that springs to mind when looking at the site, is... Who owns the copyright of fanart and posters? I think there is a grey area there also I'm afraid Sad

offtopic: spiff, have looked into the api info. looks easy to use. any indications on speed vs the others?

LaTropa64 · 2009-01-12, 20:10

No more imdb scraping is going to kill one of the best things about XBMC. Sad

I've added a few things to themoviedb.org when movies were missing fanart but manually entering all the actors and plot details doesn't sound like much fun. I've exported my library as nfo files though so I wonder if there's any chance we could just upload all our nfo's to a central location (like themoviedb.org) for scraping?

ultrabrutal · 2009-01-12, 20:30

The nfo's are based on copyright data so they are a violation also I think.

I don't think that manually entering the data is that hard when enough people contribute. It's our best bet.
You can use mymovies.dk but it only has movies which can be bought on media

marlboroman1 · 2009-01-12, 20:36

I'm would realy like some one to quietly delete this thread so we can carry on in blissfull ignorance as if non of this unpleasantness occured