Kodi Community Forum
[RELEASE] Library watchdog - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27)
+---- Forum: Service Add-ons (https://forum.kodi.tv/forumdisplay.php?fid=152)
+---- Thread: [RELEASE] Library watchdog (/showthread.php?tid=128896)

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57

RE: [RELEASE] Library watchdog - AlexVallat - 2014-05-02

(2014-05-02, 13:40)takoi Wrote: It hangs because you exit xbmc before watchdog have completely started. Either your network share isn't responding or it is very slow. Give it enough time to start, and it wont hang on exit. It's a known limitation. In the upcoming update for Gotham it is much better handled with a progress bar showing when it's safe to exit.
Hmm. I've tried just starting it and letting it stand, but nothing appears in the log after "Unloading: libImageLib-arm.so" until the automatic shutdown triggers, and then the exact same hanging behaviour occurs (http://xbmclogs.com/show.php?id=191234)

The network share is connected to through ethernet, running on a fast server PC, and I've never had any noticeable delay when browsing it or playing video from it...

Is there any way to tell by watching the log or anything when the watchdog finishes scanning?

Thanks for taking the time to look at this, by the way.

Edit: I also checked playing a video off the network share before shutting down, just to verify that it was accessible and working made no difference, it still hung on exit. If it's hanging because it can't access the network share, then could it be using some other method of accessing it than XMBC itself does?

RE: [RELEASE] Library watchdog - takoi - 2014-05-02

(2014-05-02, 13:59)AlexVallat Wrote: Hmm. I've tried just starting it and letting it stand, but nothing appears in the log after "Unloading: libImageLib-arm.so" until the automatic shutdown triggers, and then the exact same hanging behaviour occurs (http://xbmclogs.com/show.php?id=191234)

The network share is connected to through ethernet, running on a fast server PC, and I've never had any noticeable delay when browsing it or playing video from it...

Is there any way to tell by watching the log or anything when the watchdog finishes scanning?
After line 476 in your log, it should print success ("watching ...") or failure ("failed to watch.." or "does not exist"). After that, it has finished initializing.
It first check if the path exist. If there's a broken connection this can hang up to a minute. Then it will check every file and folder in the path. If it's extremely large, this can take a while, but 5 minutes seems unreasonably high to me. Nevertheless, try leaving it on for a longer time to see if it finally does succeed.

RE: [RELEASE] Library watchdog - AlexVallat - 2014-05-02

OK, absolutely never succeds, no matter how accessible and quick the network share is. So... had to dig a bit deeper on this one, then, and learn a bit of python.

As it turns out, this is actually hanging when trying to construct the observers, well before it even checks for the existence of the network share. It hangs on the line
Code:
libc = ctypes.CDLL(libc_string, use_errno=True)
inside lib\watchdog\observers\inotify.py

libc_string is "libc.so.6" at the time.

I'm afraid I don't know enough about python, Android and C to be able to hazard a guess as to *why* you can't load libc.so.6, but as I don't need inodes to observe network shares, I can solve the issue for me by replacing "if platform.is_linux():" with "if 0:" just to disable inode watching entirely.

RE: [RELEASE] Library watchdog - takoi - 2014-05-02

(2014-05-02, 18:15)AlexVallat Wrote: OK, absolutely never succeds, no matter how accessible and quick the network share is. So... had to dig a bit deeper on this one, then, and learn a bit of python.

As it turns out, this is actually hanging when trying to construct the observers, well before it even checks for the existence of the network share. It hangs on the line
Code:
libc = ctypes.CDLL(libc_string, use_errno=True)
inside lib\watchdog\observers\inotify.py

libc_string is "libc.so.6" at the time.

I'm afraid I don't know enough about python, Android and C to be able to hazard a guess as to *why* you can't load libc.so.6, but as I don't need inodes to observe network shares, I can solve the issue for me by replacing "if platform.is_linux():" with "if 0:" just to disable inode watching entirely.
Great work! Actually, this have been fixed in v0.8.0beta1 (gotham only). Android specific issue (see commit if you're interested). I have an android device myself where I encountered this issue, however, I do not remember it hanging the entire script. It should just continue on normally as the error is handled later.

Easy enough to fix so I can make an update for frodo.

RE: [RELEASE] Library watchdog - AlexVallat - 2014-05-02

(2014-05-02, 19:11)takoi Wrote: Great work! Actually, this have been fixed in v0.8.0beta1 (gotham only). Android specific issue (see commit if you're interested). I have an android device myself where I encountered this issue, however, I do not remember it hanging the entire script. It should just continue on normally as the error is handled later.

Easy enough to fix so I can make an update for frodo.
Nice one! I actually really would appreciate it if you could back-port the fix into a frodo-compatible version, as I'd rather wait for a Gotham-based Beyond build before I upgrade...

RE: [RELEASE] Library watchdog - hellcry - 2014-05-05

(2014-04-06, 16:09)takoi Wrote: New version 0.8.0beta1 for Gotham only. This contains some quite big changes to the code base,
so I would appreciate feedback from platforms i don't have available (Windows, OSX/iOS, Pi and other obscure ones.)

Changes:
- added settings for manually selecting folders to watch
- added setting for changing network polling interval
- added dialog for add-on start up progress
- fix inotify not working on android
- improved startup and shutdown performance
- many minor performance and stability improvements

Download zip


I will test this tonight, thanks I was looking for this addon for Android ARM on XBMC Gotham.

RE: [RELEASE] Library watchdog - TTLucian - 2014-05-08

Just installed 0.8.0beta1 on XBMC 13.0 on Android
I get this error:

Code:
02:57:13 T:1743110336   DEBUG: service.watchdog: video sources ['nfs://192.168.1.222/volume1/Media/Video0/']
02:57:13 T:1743110336   DEBUG: service.watchdog: music sources []
02:57:13 T:1743110336   ERROR: Traceback (most recent call last):
02:57:13 T:1743110336   ERROR:   File "/mnt/sdcard/Android/data/org.xbmc.xbmcZ/files/.xbmc/addons/xbmc-addon-watchdog-0.8.0beta1/core/main.py", line 165, in main
02:57:13 T:1743110336   ERROR:     fs_path, observer = utils.select_observer(path)
02:57:13 T:1743110336   ERROR:   File "/mnt/sdcard/Android/data/org.xbmc.xbmcZ/files/.xbmc/addons/xbmc-addon-watchdog-0.8.0beta1/core/utils.py", line 58, in select_observer
02:57:13 T:1743110336   ERROR:     path_alt = path.decode('utf-8').encode(sys.getfilesystemencoding())
02:57:13 T:1743110336   ERROR: TypeError: encode() argument 1 must be string, not None
02:57:13 T:1743110336   DEBUG: service.watchdog: failed to watch <nfs://192.168.1.222/volume1/Media/Video0/>
02:57:13 T:1743110336   DEBUG: service.watchdog: initialization done

sys.getfilesystemencoding() seems to be returning "None"
Changing line 58 in utils.py from:
Code:
path_alt = path.decode('utf-8').encode(sys.getfilesystemencoding())
to
Code:
path_alt = path.decode('utf-8').encode('utf-8')
Seems to fix the script.

Any ideeas?

RE: [RELEASE] Library watchdog - Chewy74 - 2014-05-08

Hello. On the latest version of XBMC Gotham on Apple TV 2 and library watchdog. Upon boot up it will start scanning library, then stop with error message pop up.

Then at shut down it will lock up and freeze my atv2.

Here is the debug log

http://pastebin.com/xZkKxhMh

RE: [RELEASE] Library watchdog - takoi - 2014-05-08

0.8.0beta2
Download zip

(2014-05-08, 02:04)TTLucian Wrote: Just installed 0.8.0beta1 on XBMC 13.0 on Android
I get this error:
*snip*
This has been fixed. Included in beta2 above.

(2014-05-08, 02:42)Chewy74 Wrote: Hello. On the latest version of XBMC Gotham on Apple TV 2 and library watchdog. Upon boot up it will start scanning library, then stop with error message pop up.

Then at shut down it will lock up and freeze my atv2.
Fixed. Thanks for reporting!

(2014-04-24, 19:27)telsin Wrote: It's an OS mounted NFS share, I was having to much trouble with the beta's built in mount scheme. It's a fast server/network, and the standard polling seems fine, was just curious.
Just though I'd let you know I've added detection of os mounted smb/nfs shares on linux, so polling will be automatically selected, making it a bit more seamless.

Re: RE: [RELEASE] Library watchdog - hellcry - 2014-05-08

(2014-05-08, 21:37)takoi Wrote: 0.8.0beta2
Download zip

(2014-05-08, 02:04)TTLucian Wrote: Just installed 0.8.0beta1 on XBMC 13.0 on Android
I get this error:
*snip*
This has been fixed. Included in beta2 above.

(2014-05-08, 02:42)Chewy74 Wrote: Hello. On the latest version of XBMC Gotham on Apple TV 2 and library watchdog. Upon boot up it will start scanning library, then stop with error message pop up.

Then at shut down it will lock up and freeze my atv2.
Fixed. Thanks for reporting!

(2014-04-24, 19:27)telsin Wrote: It's an OS mounted NFS share, I was having to much trouble with the beta's built in mount scheme. It's a fast server/network, and the standard polling seems fine, was just curious.
Just though I'd let you know I've added detection of os mounted smb/nfs shares on linux, so polling will be automatically selected, making it a bit more seamless.

OMG this actually works, I can't believe this. Is the first time I see this addon working on Android.

Thank you!

RE: [RELEASE] Library watchdog - Scott00007 - 2014-05-09

Watchdog actually worked!!!!!! You do not know how wonderful it is to finally have my library update automatically! I am using Android and it has never worked.......until today!

Thank you so much for all of your hard work on this addon takoi!!!!!

Re: [RELEASE] Library watchdog - Chewy74 - 2014-05-10

OK installed the latest beta 2 and the start up scan error is resolved, but shut down takes forever. I've also set watchdog to clean library upon deletion, but when cleaning starts, a progress window appears and you are stuck in that screen until the it's done cleaning. You can't navigate away from the cleaning. Is there any way to have it run in the background and still be able to browse or navigate to other sections of XBMC? The cleaning process can take up to 5 minutes.

Also I'm getting lock up and atv2 freezes when I tried to uninstall, and or disable watchdog from settings. I had to force a reboot to get it functioning again. As mentioned above, the shut down takes forever and I'm never sure if it's locked up trying to shut down from the earlier bug, or it just needs more time.

Thanks

RE: [RELEASE] Library watchdog - takoi - 2014-05-10

(2014-05-10, 05:51)Chewy74 Wrote: OK installed the latest beta 2 and the start up scan error is resolved, but shut down takes forever. I've also set watchdog to clean library upon deletion, but when cleaning starts, a progress window appears and you are stuck in that screen until the it's done cleaning. You can't navigate away from the cleaning. Is there any way to have it run in the background and still be able to browse or navigate to other sections of XBMC? The cleaning process can take up to 5 minutes.

Also I'm getting lock up and atv2 freezes when I tried to uninstall, and or disable watchdog from settings. I had to force a reboot to get it functioning again. As mentioned above, the shut down takes forever and I'm never sure if it's locked up trying to shut down from the earlier bug, or it just needs more time.

Thanks
The previous error should have been fixed, so I will need a new log. You see, I don't own any osx/ios devices to test on so I'm fixing it blindly. Don't worry about xbmc freezing on exit. It does that on the slightest error. It's usually just the end result of a different issue. Hope it isn't causing too much trouble.

Unfortunately it isn't possible to hide the progress dialog on cleaning. It's xbmc that creates and controls it.

Feature request: pathname exclusion string - thither - 2014-05-13

Thanks for the add-on, I really like it.

For a feature request, I'd like it if I could add a blacklist to the add-on - this would be a regex (or just a string) which would cause Watchdog to skip the file if it appeared in the file's path.

The use case for me is that I have a /Movies directory which SABnzbd is configured to use as the final location for movies that I download. However, after it downloads the files it winds up unpacking them in the same directory, so while this is happening the flie structure looks like:

Movies
+-- New Movie (1987)
+---- _UNPACK
+------ xyzzy.mkv

Once it's finished unpacking, SABnzbd renames the files according to my config, so they wind up more like this:

Movies
+-- New Movie (1987)
+---- New Movie (1987).mkv

The problem is that as soon as Watchdog sees "/Movies/New Movie (1987)/_UNPACK/xyzzy.mkv" it adds it to the library, and then after the rename it adds it again. So for every movie I wind up with two library entries, one which exists and one which doesn't.

Arguably this is more of a bug with SABnzbd than Watchdog, because as far as I can tell there is no way to tell it to unpack its files in a location other than their final directory. I could also work around this by having watchdog clean the library after deletion, but that takes a long time and it seems wasteful to poll my entire library when I already know that just the one new item was added.

But anyways, if I could tell Watchdog to ignore any new files that have "_UNPACK" somewhere in their pathnames, that would solve my problem.

I'm a decent python programmer, so I'm happy to put together a pull request for this feature if the source to Watchdog is available somewhere.

Thanks!

RE: [RELEASE] Library watchdog - Silvaire - 2014-05-13

I'm trying to use this service with the default settings to monitor an NFS-mounted share on OpenELEC 4.0 and I keep getting a "HSTP path not found errorr" notification on start-up when it tries to run. I am using the latest beta posted in this thread.