+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27)
+---- Forum: Program Add-ons (https://forum.kodi.tv/forumdisplay.php?fid=151)
+---- Thread: Artwork Downloader (/showthread.php?tid=114633)
Pages:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
RE: [Release] Artwork Downloader - Ghostdivision - 2013-05-24
(2013-05-11, 13:40)Martijn Wrote:(2013-05-11, 13:35)jolt Wrote: If AD would at least check for local files when a new movie is scanned into the library there would be no additional traffic at all.
And to be honest I don't quiet get it. Are you saying that if xbmc/AD automaticly downloads the artwork for a new movie there would be to much traffic, but if I manually download it there wouldn't? As I see it the only reason why more traffic would be generated is that more people would use artwork on more movies. But it is kinda sad to develop such a nice extension and community around it and then try to deter people from fully using it for traffic reasons.
let me give you my bank account number if you think it doesn't matter at all so you can donate the server costs
if you think you can do it better don't let me stop you!
some people run their library update 10 times a day. do i need to explain it some more ..............
I know this has been mentioned before, but just a suggestion to reduce server load, I would be perfect happy with a 24 hour clock allowing us to automate artwork downloader to run once every 24 hours, if you want to run it more you would have to do it manually. I just find everything in xbmc can be automated and my family loves the hands off approach to scrapping xbmc brings its what makes xbmc the best. It automatically scrapes artwork without manual upkeep and intervention whenever you add a title, but since last year this program needs to be run every time manually whenever someone in the family adds something. Just a suggestion about the 24 hour interval option, it would be far less taxing, and still allow us to automate it and run it. I would for instance have it set to run once in the morning.
RE: [Release] Artwork Downloader - voip-ninja - 2013-05-24
Now you have to multiply the 24 hour query you are proposing by the 10's of thousands of users who all are running the software. It is still a lot of traffic being generated. Remember, there's not some rich guy running the sites that host artwork.... they are typically small operations with very limited funds and are supported by donation or advertising... the advertising does no good when a user hits them up and downloads art for an XBMC installation.
RE: [Release] Artwork Downloader - dazz87 - 2013-05-25
(2013-05-24, 17:57)CaptainKen Wrote:(2013-05-24, 01:05)dazz87 Wrote: Just wondering about AD, is there a way that it will update new extrathumb art on the fly? What Im doing right now is manually going in and replacing the old thumbs with new ones that I want to display. In order for AD to update these thumbs what I need to do is rename the movie folder from "Cape Fear" ---> "Cape Fear (1991)" and then do a Update Library. The problem is that I have to do this to all Movies that i want to edit. I also have to go back to load the logo and disc art for each of the new movie. I try hitting the refresh button under Movie Info, but it keeps the same old thumb (even tho there are no longer in the extrathumb folder) instead of showing the new one. Any shortcut I could do to have AD reflect the new thumbs?
I think if you delete the movie from xbmc and update library and rerun AD it will see new artwork.
No its not working. When I update the library the movie does not show up. Adding the "year" to the movie and hitting the update button, the movie shows up again.
RE: [Release] Artwork Downloader - CaptainKen - 2013-05-25
(2013-05-25, 01:02)dazz87 Wrote:(2013-05-24, 17:57)CaptainKen Wrote:(2013-05-24, 01:05)dazz87 Wrote: Just wondering about AD, is there a way that it will update new extrathumb art on the fly? What Im doing right now is manually going in and replacing the old thumbs with new ones that I want to display. In order for AD to update these thumbs what I need to do is rename the movie folder from "Cape Fear" ---> "Cape Fear (1991)" and then do a Update Library. The problem is that I have to do this to all Movies that i want to edit. I also have to go back to load the logo and disc art for each of the new movie. I try hitting the refresh button under Movie Info, but it keeps the same old thumb (even tho there are no longer in the extrathumb folder) instead of showing the new one. Any shortcut I could do to have AD reflect the new thumbs?
I think if you delete the movie from xbmc and update library and rerun AD it will see new artwork.
No its not working. When I update the library the movie does not show up. Adding the "year" to the movie and hitting the update button, the movie shows up again.
I too have found that the year in the movie folder title is very important.
RE: [Release] Artwork Downloader - Ghostdivision - 2013-05-25
(2013-05-24, 23:27)voip-ninja Wrote: Now you have to multiply the 24 hour query you are proposing by the 10's of thousands of users who all are running the software. It is still a lot of traffic being generated. Remember, there's not some rich guy running the sites that host artwork.... they are typically small operations with very limited funds and are supported by donation or advertising... the advertising does no good when a user hits them up and downloads art for an XBMC installation.
I think your looking at this wrong, instead of letting it run once a day, people are running it each and every time something is added, i have spoke to people over the months on here regarding this topic. I might add a bunch of movies and run it multiple times a day every time a movie or show is added, and in fact I do, where as if i could schedule it to run once a day, or 2-3 times a week i would just leave it at that. Your never going to completely reduce server load unless you scrap this addon altogether.
The problem I thought with server load came running the addon at startup/login feature, which could happen multiple times a day, in fact in our house it was happening 4 times a day at login, until i realized it was going way to much.
And I am aware of these server costs, I have donated to fanart.tv i have had discussions here a year ago already when automation was first removed about this issue with the costs of server, and I am in no way advocating making things worse on the servers.
Automation solution does not mean increase server load. If 24 hour intervals are to much allow us to schedule it a few days during the week. I just know several people love this addon and dont like running it manually every time something is added and that can also increase the frequency its being run. Tonight i added for instance 2 movies and a tv shows later, i ran it manually 3 times today. If it ran once at scheduled time I would just leave it to run then. Everything in xbmc can be automated except this addon unfortunately since last year. I am just offering a balance between some form automation and not taxing the servers.
Again just my opinion.
RE: [Release] Artwork Downloader - Milhouse - 2013-05-25
@Ghostdivision - do you have cron?
There's probably other scripts that can automate add-on execution but I've added such support to Texture Cache Maintenance utility:
Code:
cron@12:00: ./texturecache.py exec script.artwork.downloader silent=true mediatype=movie
cron@12:30: ./texturecache.py exec script.artwork.downloader silent=true mediatype=tvshowCode:
cron@01:00:./texturecache.py c videoAlternatively, you can write a script that is able to detect when each AD run has finished so that calls to AD can be run serialised without any delay.
Here's an example bash script that I use which initiates a video library scan, followed by calls to AD for movies then tvshows, and finally the texture cache is pre-loaded if any new items have been added (either a full library pre-load or just incremental depending on whether AD has added any new items).
Code:
#!/bin/bash
LOGFILE=/var/log/xbmcscan.log
function _log {
[ -z "$1" ] && echo >> $LOGFILE || $PRINTF "$(date +"%Y-%m-%d %H:%M:%S"): $1\n" >> $LOGFILE
}
function waitForAD {
XBMC="/mnt/share/xbmc/rpi1/Storage/.xbmc"
XBMC_LOG="${XBMC}/temp/xbmc.log"
AD_REPORT="${XBMC}/userdata/addon_data/script.artwork.downloader/downloadreport.txt"
tail -fn0 "${XBMC_LOG}" | \
while read line; do
if [ -n "$(echo "$line" | grep "Artwork Downloader: script stopped")" ]; then
kill $(ps -o pid,args | grep -v grep | grep "tail -fn0 ${XBMC_LOG}" | awk '{ print $1 }') 2>/dev/null
break
fi
done
echo $(grep -e '^ - Total Artwork: 0$' "${AD_REPORT}" | awk '{ print $4 }')
}
NEWITEMS=0
_log "$(/mnt/share/bin/texturecache.py update 2>/dev/null)"
VSCAN="$(/mnt/share/bin/texturecache.py vscan 2>/dev/null)"
RESULT=$?
_log "$(echo "${VSCAN}")"
/mnt/share/bin/texturecache.py exec script.artwork.downloader silent=true mediatype=movie
NEWITEMS=$(($NEWITEMS + $(waitForAD "movies")))
/mnt/share/bin/texturecache.py exec script.artwork.downloader silent=true mediatype=tvshow
NEWITEMS=$(($NEWITEMS + $(waitForAD "tvshows")))
if [ ${NEWITEMS} -ne 0 ]; then
_log "New AD items downloaded - performing a full cache pre-load"
_log "\n$(/mnt/share/bin/texturecache.py c movies 2>/dev/null)"
_log "\n$(/mnt/share/bin/texturecache.py c tvshows 2>/dev/null)"
elif [ ${RESULT} -ne 0 ]; then
_log "No new AD items - performing a partial cache pre-load"
_log "\n$(/mnt/share/bin/texturecache.py lc movies 2>/dev/null)"
_log "\n$(/mnt/share/bin/texturecache.py lc tvshows 2>/dev/null)"
else
_log "Skipping cache pre-load - no new artwork items added"
fi
touch /tmp/texturecache.lastrun/mnt/share/bin/texturecache.cfg:
Code:
xbmc.host=192.168.0.202
userdata=/mnt/share/xbmc/rpi1/Storage/.xbmc/userdata
cache.castthumb = yes
logfile=+/tmp/texturecache.log
lastrunfile=/tmp/texturecache.lastrunI run this script automatically on a remote FreeNAS system that I rsync new media to, so it only runs once the new media has hit the remote server - a Raspberry Pi running OpenELEC with its /storage partition mounted on NFS is connected to the NAS.
RE: [Release] Artwork Downloader - aur0n - 2013-05-26
Is there any way to make Artwork Downloader to search just for missing movies extra fanarts?
Each time I scan, it cycle all my 400+ movies library.
Thanks.
RE: [Release] Artwork Downloader - CaptainKen - 2013-05-27
(2013-05-26, 03:30)aur0n Wrote: Is there any way to make Artwork Downloader to search just for missing movies extra fanarts?
Each time I scan, it cycle all my 400+ movies library.
Thanks.
Sorry no there isn't. However, after the first time you run it, all subsequent runs are pretty fast.I believe the "use local folders" contributed to it running fast.
RE: [Release] Artwork Downloader - Raytestrak - 2013-05-27
(2013-05-27, 06:11)CaptainKen Wrote:(2013-05-26, 03:30)aur0n Wrote: Is there any way to make Artwork Downloader to search just for missing movies extra fanarts?
Each time I scan, it cycle all my 400+ movies library.
Thanks.
Sorry no there isn't. However, after the first time you run it, all subsequent runs are pretty fast.I believe the "use local folders" contributed to it running fast.
Not entirely true. If you run the add-on from the info dialiog of a selected movie or tvshow (in Aeon Nox it's called "get artwork"), it will only run for that selected item.
RE: [Release] Artwork Downloader - CaptainKen - 2013-05-27
(2013-05-27, 06:34)Raytestrak Wrote:(2013-05-27, 06:11)CaptainKen Wrote:(2013-05-26, 03:30)aur0n Wrote: Is there any way to make Artwork Downloader to search just for missing movies extra fanarts?
Each time I scan, it cycle all my 400+ movies library.
Thanks.
Sorry no there isn't. However, after the first time you run it, all subsequent runs are pretty fast.I believe the "use local folders" contributed to it running fast.
Not entirely true. If you run the add-on from the info dialiog of a selected movie or tvshow (in Aeon Nox it's called "get artwork"), it will only run for that selected item.
Yep, thanks Raytestrak, I had thought of that as well, but didn't mention it because it sounded like aur0n wanted to scan all movies for missing artwork. well at least all options are on the table now.
RE: [Release] Artwork Downloader - aur0n - 2013-05-27
(2013-05-27, 06:41)CaptainKen Wrote:(2013-05-27, 06:34)Raytestrak Wrote:(2013-05-27, 06:11)CaptainKen Wrote: Sorry no there isn't. However, after the first time you run it, all subsequent runs are pretty fast.I believe the "use local folders" contributed to it running fast.
Not entirely true. If you run the add-on from the info dialiog of a selected movie or tvshow (in Aeon Nox it's called "get artwork"), it will only run for that selected item.
Yep, thanks Raytestrak, I had thought of that as well, but didn't mention it because it sounded like aur0n wanted to scan all movies for missing artwork. well at least all options are on the table now.
Indeed I wanted to avoid to do that, it's not really handy... I usually add multiple items at once (5/10 movies), this means I have to go to movies, and start the scan for each one of them.
Well, if there's no other option... Thanks anyway.
RE: [Release] Artwork Downloader - Milhouse - 2013-05-27
(2013-05-26, 03:30)aur0n Wrote: Is there any way to make Artwork Downloader to search just for missing movies extra fanarts?
Each time I scan, it cycle all my 400+ movies library.
Thanks.
It has to run through all your movies to build up the list of items to work on - this is unavoidable. Unfortunately it also requires streamdetails, which adds a noticeable overhead to the JSON call.
However, it does appear that AD will then access the remote artwork sites even for movies that already have artwork fulfilled - it will use a 10 day cache to limit access to these remote artwork sites, but eventually it will initiate network access and I'm not really sure why it does that when it's not necessary.
For instance, if I'm only looking for clearart+clearlogos, then AD should:
a) Build a list of all movies and their associated artwork (+ streamdetails to determine HD/SD)
b) Immediately exclude from further processing those movies that already have the required artwork (ie. those with clearart+clearlogo both present)
c) Spend time working only on the remainder
Unfortunately b) doesn't seem to happen, leading to a lot of potentially unnecessary work/delay processing movies for which artwork is no longer required, and I'm not sure why that is the case unless I'm missing something obvious.
(2013-05-27, 20:30)aur0n Wrote: Indeed I wanted to avoid to do that, it's not really handy... I usually add multiple items at once (5/10 movies), this means I have to go to movies, and start the scan for each one of them.
Well, if there's no other option... Thanks anyway.
Writing a script that queries your movie media library, determines which movies have missing artwork items, and then runs AD in turn for each item would be relatively trivial, though not very optimal (but certainly better than running it manually).
Having said it's not very optimal, if almost all of your movies have the required artwork and only a few (new movies?) are missing artwork, then a script that calls AD only for those movies that are missing artwork will be much quicker than running AD against your entire movie library. Though as the number of movies with missing artwork increases, there will rapidly come a point where it makes more sense (in terms of efficiency/elapsed time) to run AD once, in bulk mode, rather than multiple times for each individual movie.
RE: [Release] Artwork Downloader - formula - 2013-05-29
i find this useful but it takes at least half a day to complete , even though i have my disc art png's in the movie folders .. it would be quicker manually doing it but its more of a ball ache , ive set the option to use local folders but takes forever , ive just re scrapped my data base hence missing disc art .png's , shame i have to wait so long just for the program to see whats already there ! is there any other options
RE: [Release] Artwork Downloader - Milhouse - 2013-05-29
(2013-05-29, 21:08)formula Wrote: i find this useful but it takes at least half a day to complete , even though i have my disc art png's in the movie folders .. it would be quicker manually doing it but its more of a ball ache , ive set the option to use local folders but takes forever , ive just re scrapped my data base hence missing disc art .png's , shame i have to wait so long just for the program to see whats already there ! is there any other options
Part of (if not a major) reason for the slow performance is most likely this. If you have cast artwork scraped into your library then the amount of SQL updates that occur when adding artwork increases dramatically, and these updates occur with each new artwork item that is added. That is to say, if 205 SQL updates occur when one new artwork is added (as happens with one of my movies - no exaggeration), and you need to add 6 items of artwork for each movie, that's 6*205 SQL updates per movie - 1200+ updates to add 6 new database items.
It's quite insanely inefficient, really, and not the fault of AD but the JSON processing. I have had no response from the JSON guy when asking if anything could be done to improve performance - eg. short circuiting certain updates to avoid all the unnecessary work when simply adding artwork, or when updating common fields, eg. playcount - so I guess it's head in the sand time.
RE: [Release] Artwork Downloader - formula - 2013-05-29
(2013-05-29, 21:39)MilhouseVH Wrote:(2013-05-29, 21:08)formula Wrote: i find this useful but it takes at least half a day to complete , even though i have my disc art png's in the movie folders .. it would be quicker manually doing it but its more of a ball ache , ive set the option to use local folders but takes forever , ive just re scrapped my data base hence missing disc art .png's , shame i have to wait so long just for the program to see whats already there ! is there any other options
Part of (if not a major) reason for the slow performance is most likely this. If you have cast artwork scraped into your library then the amount of SQL updates that occur when adding artwork increases dramatically, and these updates occur with each new artwork item that is added. That is to say, if 205 SQL updates occur when one new artwork is added (as happens with one of my movies - no exaggeration), and you need to add 6 items of artwork for each movie, that's 6*205 SQL updates per movie - 1200+ updates to add 6 new database items.
It's quite insanely inefficient, really, and not the fault of AD but the JSON processing. I have had no response from the JSON guy when asking if anything could be done to improve performance - eg. short circuiting certain updates to avoid all the unnecessary work when simply adding artwork, or when updating common fields, eg. playcount - so I guess it's head in the sand time.
thanks for the reply ! someone justs needs to make a simple scapper to look for posters, fanart , disc.png's in the movie folders