+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27)
+---- Forum: Service Add-ons (https://forum.kodi.tv/forumdisplay.php?fid=152)
+---- Thread: Library Auto Updater - Version 1.1.0 (/showthread.php?tid=119520)
Library Auto Updater - Version 1.1.0 - robweber - 2012-01-10
One of Kodi's most powerful features are it's Video and Music libraries. With most people adding to their libraries daily, having to restart Kodi or manually re-scan for content can be a chore. While cron jobs or batch scripts can take care of the update process for you, this add-on keeps the library management within Kodi without having to reply on another method. Please note that this is just a fancy timer that calls out to the normal Kodi Library Scanning functions. All of the processes associated with scanning are all handed off to Kodi.
You can download this addon from the official Addon-Repo.
Code: https://github.com/robweber/xbmclibraryautoupdate
Instructions: https://github.com/robweber/xbmclibraryautoupdate/blob/master/README.md
- dynaflash - 2012-01-11
Rob, as per http://forum.xbmc.org/showthread.php?p=985289#post985289 I have ceased working on my implementation of your addon on my own as since then it was updated by yourself. I agree it is a very valuable addition to any decent xbmc implementation!
I mainly reworked it after dharma because at that time it wasn't working with pre - eden. One issue I did have with the new service addon as opposed to as a program is that there is no way I know of to invoke it manually. though maybe I am missing something. Anyway since that post I have gone back to using your new revision. Thanks and please realize I was not attempting to overtake your work but build on it as I was not sure after dharma at that time you would continue on with it.
Once again, Kudos!
- TheStretchedElf - 2012-01-11
Thanks for this, just what I need for my daughters new ATV2.
- vikjon0 - 2012-01-11
Quote:there is no way I know of to invoke it manually
I have though about that. Is it not possible to have multiple extension points?
- Martijn - 2012-01-11
vikjon0 Wrote:I have though about that. Is it not possible to have multiple extension points?
Yes it is
PHP Code:
<extension point="xbmc.python.script" library="default.py">
<provides>executable</provides>
</extension>
<extension point="xbmc.service" library="default.py" start="startup|login"/>
- vikjon0 - 2012-01-11
Quote:Yes it isThanks!
- robweber - 2012-01-11
dynaflash Wrote:I mainly reworked it after dharma because at that time it wasn't working with pre - eden. One issue I did have with the new service addon as opposed to as a program is that there is no way I know of to invoke it manually. though maybe I am missing something. Anyway since that post I have gone back to using your new revision. Thanks and please realize I was not attempting to overtake your work but build on it as I was not sure after dharma at that time you would continue on with it.
Once again, Kudos!
No problem! I wasn't really focused on Eden until late in its development so I can understand wanting something that will work with the nightly builds and such. without a "proper" way for people to even ask for requests there was no way of knowing I would continue doing anything with it. Your idea with the scanning definitely got me thinking of how this could be incorporated into a more functional addon.
Martijn Wrote:Yes it is
Code:<extension point="xbmc.python.script" library="default.py">
<provides>executable</provides>
</extension>
<extension point="xbmc.service" library="default.py" start="startup|login"/>
I had no idea you could do multiple points like that. This is something that should be added to the wiki. I'll try to get this put into the Updater addon soon so it can be launched manually. Since the service will always launch the manual option will be more of a quick way to launch the normal "Scan for Updates" functionality; plus reset the Library Updater timer. Thanks for the info!
- Martijn - 2012-01-11
robweber Wrote:I had no idea you could do multiple points like that. This is something that should be added to the wiki. I'll try to get this put into the Updater addon soon so it can be launched manually. Since the service will always launch the manual option will be more of a quick way to launch the normal "Scan for Updates" functionality; plus reset the Library Updater timer. Thanks for the info!
'Several other add-ons also use it and one even acts like it own repo
The wiki is user driven so you can always add things yourself when you think they are missing
- michaeldecharon - 2012-01-11
What about after updating the library execute clean library.
Also nice, update the library when content is added to a specific folder and clean library when content is deleted in a specific folder.
- robweber - 2012-01-11
michaeldecharon Wrote:What about after updating the library execute clean library.
Also nice, update the library when content is added to a specific folder and clean library when content is deleted in a specific folder.
Regarding the clean library functionality please see this page on the wiki: http://wiki.xbmc.org/index.php?title=Advancedsettings.xml#.3Cvideolibrary.3E You can add a setting to your advancedsettings.xml file so that the library always cleans when it updates. This is the easiest way to make sure this happens without having to separately click the "Clean Library" setting under Video Settings.
As far as monitoring specific folders; the problem implementing that type of functionality is exactly why this addon uses the built in update functionality of XBMC and runs on a timer. Since files can be connected with many different types of protocols (NFS, Samba, local hard disks, etc) it is very hard to develop a cross platform way to monitor for files (event driven updates). The most common way to do this is with some type of folder polling (watchdog services); but those are inefficient. Writing something like that in python would basically duplicate the actions already happening via the XBMC Update command (look through each folder for new files). There are a lot of forum posts on this but the one I find the most interesting is this one: http://forum.xbmc.org/showthread.php?tid=66348 . Several XBMC developers explain the issue with having an event driven system very well.
Generally running the update processes every few hours works fairly well for most people. The scan process is usually pretty quick and most of the time happens when you aren't even using the system. I do agree though that a cross platform, event driven, system would be the best solution eventually.
- Martijn - 2012-01-11
@robweber
I agree that running it every xx hours should suffice.
That aside could something like this work cross platform?:
PHP Code:
try:
date = os.path.getmtime( base_path )
print date
except:
date = 0
# we only refresh if it's been more than a day, 24hr * 60min * 60sec
refresh = ( ( time.time() - ( 24 * 60 * 60 ) ) >= date )
Found it in a script somewhere
- robweber - 2012-01-12
Martijn Wrote:@robweber
I agree that running it every xx hours should suffice.
That aside could something like this work cross platform?:
PHP Code:try:
date = os.path.getmtime( base_path )
print date
except:
date = 0
# we only refresh if it's been more than a day, 24hr * 60min * 60sec
refresh = ( ( time.time() - ( 24 * 60 * 60 ) ) >= date )
Found it in a script somewhere
From what I can see that would flag a folder to be refreshed in the event that its modified date was greater than a day. I'm guessing you could then code the refresh value to trigger if the folder should be scanned or not?
I found the following related to how the "os.path" module in Python functions:
Quote:The os.path module is always the path module suitable for the operating system Python is running on, and therefore usable for local paths.
So while it would probably work for local paths, sources that are connected via samba, nfs, etc probably wouldn't return the right values. The docs I found even said that unc paths on windows systems wouldn't work.
Thinking about it I'm still not sure if doing separate polling of the individual paths and then specifying (via the updatelibrary() function) that path to update is worth the extra effort. The built in xbmc scanner seems to do a good job of detecting new files very quickly; maybe if you had lots of sources it would be worth only scanning folders that have changed.
I think ideally it would be nice if the built in XBMC scanner had a cross platform way of checking for changed files so that it could selectively know what folders to scan, this would put the burden on the core xbmc process rather than having something like an addon do the work. Again, I'm sure the problem is the cross platform piece of it.
Nice find with that piece of code. I'm by no means a Python expert so I'll have to remember that function in case I ever need something similar.
- Martijn - 2012-01-12
That was my guess also that it only works for local path.
I don't find it useless to update on every 'change'. Once or two times a day would be more than sufficient in my case.
Once you made is a service/program they also run it theirselves if they want.
Same as we did with Artwork Downloader.
Maybe you can even trigger it when you finished the library update
On the other hand we offer a one time a day run and they can start it manually
I already looked into it briefly once but couldn't find a way to detect if it finished the update.
- GJones - 2012-01-17
I notice that the service is running an update on startup. Is there any way that we could delay the scan so that it does not run at startup? In particular, I am using the watchlist script and it hits the database heavily at startup. It would be nice to have an option to delay the first run of this script for 1,2 or 5 minutes after startup to let the user get settled into the interface before running the update.
- GJones - 2012-01-19
GJones Wrote:I notice that the service is running an update on startup. Is there any way that we could delay the scan so that it does not run at startup? In particular, I am using the watchlist script and it hits the database heavily at startup. It would be nice to have an option to delay the first run of this script for 1,2 or 5 minutes after startup to let the user get settled into the interface before running the update.
As a suggestion, the last_run could be written to file for comparison so that when the service started (like on an xbmc restart), it could determine if there was a need to update the library on startup.
Additionally, a sleep time after startup could be specified in service.py at the beginning of the first while loop. This would allow the user to specify a period after xbmc startup to wait before updating the library. This would allow less contention between the library update, watchlist and recentlyadded scripts. On large libraries, the impact of all 3 running simultaneously is somewhat disruptive.