Kodi Community Forum
Ember scrapes very slowly? - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Supplementary Tools for Kodi (https://forum.kodi.tv/forumdisplay.php?fid=116)
+---- Forum: Ember Media Manager (https://forum.kodi.tv/forumdisplay.php?fid=195)
+---- Thread: Ember scrapes very slowly? (/showthread.php?tid=369468)

Ember scrapes very slowly? - Afonzo - 2022-08-30

I installed two Embers today. They scrape very slowly. For the second, I left the defaults. I tried it with a movie folder containing only about 100 movies. The scraping lasts about 20 to 30 minutes.

Is there a way to make Ember scrape faster?

RE: Ember scrapes very slowly? - DanCooper - 2022-08-30

Disable the YouTube, Apple and DavesTrailerPage trailer scrapers.

RE: Ember scrapes very slowly? - Afonzo - 2022-08-30

Aren't they disabled by defaut? They obviously all are disabled:

Image

RE: Ember scrapes very slowly? - Afonzo - 2022-08-31

How could one resize the column width?
Image

RE: Ember scrapes very slowly? - codygp - 2022-08-31

I have the same issue with Ember scraping very slowly. And would like to figure it out. 

I followed the advise you posted earlier and made sure the extra scrapers were disabled, they already were.

What I notice is that after images and trailer scrape is done it goes into "Downloading and saving contents into database"

Here is a log output from one I am working on right now. It shows a 4 minute delay between the rename and some OnNFOSave_Movie task.

https://paste.kodi.tv/sacahevebi

And just to clarify, this is running on an i7-7700K, very high spec system with 1Gbps fiber internet connected via 10GB wired ethernet. System power and internet speed are not issues here.

EDIT: And my issue is solved. Its just the trailers being downloaded. TMDB is as slow as Youtube. Disable all trailer scrapers and all goes just fine.

RE: Ember scrapes very slowly? - DanCooper - 2022-09-01

(2022-08-31, 21:42)codygp Wrote: TMDB is as slow as Youtube.
TMDb also use links to YouTube trailers.

RE: Ember scrapes very slowly? - codygp - 2022-09-02

Any idea why it's so slow? I can download via browser extensions quite quickly from YouTube.

RE: Ember scrapes very slowly? - jbinkley60 - 2022-09-02

(2022-09-01, 21:26)DanCooper Wrote:
(2022-08-31, 21:42)codygp Wrote: TMDB is as slow as Youtube.
TMDb also use links to YouTube trailers.

I am not sure what library is being used to grab the You Tube trailers but if it is youtube-dl there are many reported issues of slowness and it capping out at 60kbs.  I have confirmed that myself.


Jeff

RE: Ember scrapes very slowly? - madmax2 - 2022-09-02

(2022-09-02, 01:56)codygp Wrote: Any idea why it's so slow? I can download via browser extensions quite quickly from YouTube.

yeah, downloading of trailers increase the scraping time quite significantly.

I also disabled this after comparing it on/off.

Just set it to autoscrape then go do something and come back later.
if you got like 200 movies it could take about approx 1hr+

RE: Ember scrapes very slowly? - madmax2 - 2022-09-02

(2022-09-01, 21:26)DanCooper Wrote:
(2022-08-31, 21:42)codygp Wrote: TMDB is as slow as Youtube.
TMDb also use links to YouTube trailers.
I also notice an issue, 
the trailer downloader can redownload new trailers with .mov extension
even though the directory might already have a trailer.

Anyway to fix this issue?