Kodi Community Forum
[RELEASE] Library watchdog - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27)
+---- Forum: Service Add-ons (https://forum.kodi.tv/forumdisplay.php?fid=152)
+---- Thread: [RELEASE] Library watchdog (/showthread.php?tid=128896)



RE: [RELEASE] Library watchdog - Scott00007 - 2014-04-21

I cannot get Watchdog to find any of my video sources. I have no problem playing the files in XBMC but Watchdog errors out when XBMC starts saying it cannot watch my video sources. I have tried just selecting "watch video sources" and adding the sources directly. I am using the beta for gotham.

I have uploaded a log using the log uploader - 179677

Looking at the log I cannot see what is going wrong, any help would be great.


RE: [RELEASE] Library watchdog - takoi - 2014-04-21

(2014-04-21, 03:54)Scott00007 Wrote: I cannot get Watchdog to find any of my video sources. I have no problem playing the files in XBMC but Watchdog errors out when XBMC starts saying it cannot watch my video sources. I have tried just selecting "watch video sources" and adding the sources directly. I am using the beta for gotham.

I have uploaded a log using the log uploader - 179677

Looking at the log I cannot see what is going wrong, any help would be great.
Fixed. Thanks for reporting. You can get zip from github (version in xbmc repository should not be affected either)


RE: [RELEASE] Library watchdog - telsin - 2014-04-24

I have an issue where it can't monitor my music share, but does fine with my video shares. Looks like it's munging the path internally.

http://xbmclogs.com/show.php?id=184569

This started happening to me on XBMC Gotham Betas sometime after January (no exact version, sorry, screwed up my git versioning) with 0.7.6, and continues happening with latest betas and 0.8.0b1 of watchdog. The logs were from a fresh setup of beta4 with watchdog 0.8.0b1.

Also a question: what unit is the "Network Polling Delay" in? Curious as to what it does to set appropriate for my shares.

Thanks! Let me know if you need something else from the logs, I trimmed them to watchdog only.


RE: [RELEASE] Library watchdog - takoi - 2014-04-24

(2014-04-24, 17:52)telsin Wrote: I have an issue where it can't monitor my music share, but does fine with my video shares. Looks like it's munging the path internally.

http://xbmclogs.com/show.php?id=184569

This started happening to me on XBMC Gotham Betas sometime after January (no exact version, sorry, screwed up my git versioning) with 0.7.6, and continues happening with latest betas and 0.8.0b1 of watchdog. The logs were from a fresh setup of beta4 with watchdog 0.8.0b1.
That log is very odd, the actual error is missing. Are you sure you didn't cut out something?

(2014-04-24, 17:52)telsin Wrote: Also a question: what unit is the "Network Polling Delay" in? Curious as to what it does to set appropriate for my shares.
You mean Network polling interval? In seconds. It's the time it will wait before checking for new files. The default of 4 is fine. You can increase it if you find that it's eating too much resources and don't mind a possible longer delay. But as far as I can see from your log, you aren't using network shares, so it doesn't apply anyway.


RE: [RELEASE] Library watchdog - telsin - 2014-04-24

(2014-04-24, 18:40)takoi Wrote:
(2014-04-24, 17:52)telsin Wrote: I have an issue where it can't monitor my music share, but does fine with my video shares. Looks like it's munging the path internally.
That log is very odd, the actual error is missing. Are you sure you didn't cut out something?

Ok, that's just odd. The grep I ran over it didn't get everything, or it was cut-paste problem. Try this one: http://xbmclogs.com/show.php?id=184923

Should have included the mention that /storage/music is nfs mounted from the same place as video, and has valid permissions.

Code:
telsin@xmbc:~/.xbmc/temp$ ls -al /storage/
total 68
drwxr-xr-x  5 root root  4096 Apr 23 23:19 .
drwxr-xr-x 25 root root  4096 Apr  5 12:23 ..
drwxrwxrwx 38 root users   42 Mar  4 13:34 music
drwxr-xr-x 19 root root    19 Apr 23 23:26 thumbnails
drwxrwxr-x 15 root users   18 Apr 17 18:23 video

I also tried deleting and re-adding the share after this, still getting same error. It is fairly large if that could cause an issue?

(2014-04-24, 18:40)takoi Wrote:
(2014-04-24, 17:52)telsin Wrote: Also a question: what unit is the "Network Polling Delay" in? Curious as to what it does to set appropriate for my shares.
You mean Network polling interval? In seconds. It's the time it will wait before checking for new files. The default of 4 is fine. You can increase it if you find that it's eating too much resources and don't mind a possible longer delay. But as far as I can see from your log, you aren't using network shares, so it doesn't apply anyway.

It's an OS mounted NFS share, I was having to much trouble with the beta's built in mount scheme. It's a fast server/network, and the standard polling seems fine, was just curious.


RE: [RELEASE] Library watchdog - takoi - 2014-04-24

(2014-04-24, 19:27)telsin Wrote: Ok, that's just odd. The grep I ran over it didn't get everything, or it was cut-paste problem. Try this one: http://xbmclogs.com/show.php?id=184923

Should have included the mention that /storage/music is nfs mounted from the same place as video, and has valid permissions.

Code:
telsin@xmbc:~/.xbmc/temp$ ls -al /storage/
total 68
drwxr-xr-x  5 root root  4096 Apr 23 23:19 .
drwxr-xr-x 25 root root  4096 Apr  5 12:23 ..
drwxrwxrwx 38 root users   42 Mar  4 13:34 music
drwxr-xr-x 19 root root    19 Apr 23 23:26 thumbnails
drwxrwxr-x 15 root users   18 Apr 17 18:23 video

I also tried deleting and re-adding the share after this, still getting same error. It is fairly large if that could cause an issue?
File permissions are good, that's not what it means. I'm guessing you have more than 8192 folders in that directory(?). You see, linux has a limit on the number of folders you can monitor simultaneously (Inotify watch limit). The default is usually 8192. It can be adjusted, but either way, if these are remote shares mounted outside of xbmc it will not work unless you select "polling" in add-on settings. (Did you not notice you didn't get any updates for the video sources even though it 'succeeds' to start?). If you do that, this problem shouldn't occur.


RE: [RELEASE] Library watchdog - telsin - 2014-04-24

(2014-04-24, 20:13)takoi Wrote: File permissions are good, that's not what it means. I'm guessing you have more than 8192 folders in that directory(?). You see, linux has a limit on the number of folders you can monitor simultaneously (Inotify watch limit). The default is usually 8192. It can be adjusted, but either way, if these are remote shares mounted outside of xbmc it will not work unless you select "polling" in add-on settings. (Did you not notice you didn't get any updates for the video sources even though it 'succeeds' to start?). If you do that, this problem shouldn't occur.

That makes sense, there's definitely a lot of folders under that tree. I probably didn't notice this start when I switched from xbmc nfs mounts to OS based mounts and the Auto setting tried to use Inotify instead of polling it. I do think it's been finding new video, but haven't been paying enough attention to be sure.

Either way, switched it to polling and it's not complaining any more. Bonus, XBMC now shuts down normally instead of hanging like it was before (trying to unregister a lot of Inotify's maybe?).

Thanks for the help, error message wasn't much use there.


RE: [RELEASE] Library watchdog - takoi - 2014-04-25

(2014-04-24, 21:07)telsin Wrote: That makes sense, there's definitely a lot of folders under that tree. I probably didn't notice this start when I switched from xbmc nfs mounts to OS based mounts and the Auto setting tried to use Inotify instead of polling it. I do think it's been finding new video, but haven't been paying enough attention to be sure.
It will work if you make changes in the mounted directory on the client, otherwise it won't. This is because the kernel can only see changes it makes, i.e on file systems it owns, and there's no communication of sort going on over nfs

I'm always trying to make this more seamless, but I don't know how to detect it reliably yet. When using xbmc's nfs it should work out of the box. I'm hoping that if you are even able to find out how to set up nfs mounts manually, you are also able to find this thread and figure out you have to manually set it to polling;)


RE: [RELEASE] Library watchdog - WhoCares - 2014-04-26

After installing watchdog, I started noticing duplicate episodes under my TV Series. At the time I was busy adding and deleting some video sources. After a while, every single one of my series episodes were duplicated. I tried cleaning the db, but it made no difference. I removed the video sources and re-added them. Same result. Removed one of the series from the db. Re-scan. Same result. Eventually I deleted my db file and re-started xbmc. Changed all the video sources to reflect either movies or series. Scan started. Again, duplicates. Disabled watchdog, deleted db, restarted. Changed video sources to movies and series again. Re-scan.. No more duplicates.

I'm running OpenELEC 3.95.6, Aeon Nox (latest beta), and was running the latest watchdog off the github


RE: [RELEASE] Library watchdog - takoi - 2014-04-26

(2014-04-26, 11:29)WhoCares Wrote: After installing watchdog, I started noticing duplicate episodes under my TV Series. At the time I was busy adding and deleting some video sources. After a while, every single one of my series episodes were duplicated. I tried cleaning the db, but it made no difference. I removed the video sources and re-added them. Same result. Removed one of the series from the db. Re-scan. Same result. Eventually I deleted my db file and re-started xbmc. Changed all the video sources to reflect either movies or series. Scan started. Again, duplicates. Disabled watchdog, deleted db, restarted. Changed video sources to movies and series again. Re-scan.. No more duplicates.

I'm running OpenELEC 3.95.6, Aeon Nox (latest beta), and was running the latest watchdog off the github
Did you know cleaning library on file deletion is off by default?

Just to be clear: watchdog does not touch the database. It does the exact same thing as 'clean library' and 'update library' in the GUI, only automatically. If there's still duplicates after starting clean library manually, the issue comes from something else.


RE: [RELEASE] Library watchdog - WhoCares - 2014-04-27

(2014-04-26, 12:39)takoi Wrote: Did you know cleaning library on file deletion is off by default?

Just to be clear: watchdog does not touch the database. It does the exact same thing as 'clean library' and 'update library' in the GUI, only automatically. If there's still duplicates after starting clean library manually, the issue comes from something else.

Yes, I did know. It was switched on.

After adding a source and changing it's content, xbmc asked if it should refresh - I said yes. Is it possible that xbmc and watchdog each launched a separate instance of refresh, which led to the duplicates?

btw - In the meantime I have re-enabled watchdog (after adding all my sources). No more duplicates since.


RE: [RELEASE] Library watchdog - AlexVallat - 2014-05-02

Hi, I'd really like to get this plugin working, but I'm running into a problem where it hangs XMBC on exit. Could someone help me investigate this, or let me know if there's a known workaround?

I'm running the BeyondXBMC variant, on Android, and monitoring the Video library which has only one source, on an SMB share.

The log file is:
Code:
08:59:52 T:1744289664   DEBUG: : trying mouse action leftclick
08:59:52 T:1744289664   DEBUG:  : Translating XBMC.Quit()
08:59:52 T:1744289664   DEBUG:  : To XBMC.Quit()
08:59:52 T:1744289664   DEBUG: Unfocus WindowID: 10111, ControlID: 2
08:59:52 T:1744289664   DEBUG: Unfocus WindowID: 10000, ControlID: 20
08:59:52 T:1744289664   DEBUG: CAnnouncementManager - Announcement: OnQuit from xbmc
08:59:52 T:1744289664   DEBUG: GOT ANNOUNCEMENT, type: 8, from xbmc, message OnQuit
08:59:52 T:1744289664  NOTICE: Storing total System Uptime
08:59:52 T:1744289664  NOTICE: Saving settings
08:59:52 T:1744289664  NOTICE: stop all
08:59:52 T:1744289664    INFO: stopping PVRManager
08:59:52 T:1744289664   DEBUG:  - Signaling network services to stop
08:59:52 T:1744289664  NOTICE: ES: Stopping event server
08:59:52 T:1744289664   DEBUG:  - Waiting for network services to stop
08:59:52 T:1744289664  NOTICE: stop sap announcement listener
08:59:52 T:1744289664  NOTICE: clean cached files!
08:59:52 T:1744289664  NOTICE: unload skin
08:59:52 T:1744289664    INFO: Unloading old skin ...
08:59:52 T:1744289664   DEBUG: ------ Window Deinit () ------
08:59:52 T:1744289664   DEBUG: ------ Window Deinit (Home.xml) ------
08:59:52 T:1744289664   DEBUG:  - Closed bundle
08:59:52 T:1744289664   DEBUG: ADDON: Stopping service addons.
08:59:52 T:1744289664    INFO: Stopping script with id: 1
08:59:52 T:1818539872    INFO: Scriptresult: Success
08:59:52 T:1744289664   DEBUG: XBPyThread::stop - script termination took 237ms
08:59:52 T:1744289664    INFO: Stopping script with id: 5
08:59:52 T:1828030240  NOTICE: StorageServer-2.5.4 Closed down
08:59:52 T:1828030240    INFO: Scriptresult: Success
08:59:52 T:1744289664   DEBUG: XBPyThread::stop - script termination took 451ms
08:59:52 T:1818539872    INFO: Python script interrupted by user
08:59:52 T:1744289664    INFO: Stopping script with id: 2
08:59:52 T:1827900984   DEBUG: Skin Widgets: script version 0.0.27 stopped
08:59:52 T:1827900984    INFO: Scriptresult: Success
08:59:52 T:1744289664   DEBUG: XBPyThread::stop - script termination took 122ms
08:59:52 T:1744289664    INFO: Stopping script with id: 8
08:59:52 T:1827900984 WARNING: The python script "/data/data/com.flumpster.xbmc/cache/apk/assets/addons/service.skin.widgets/default.py" has left several classes in memory that we couldn't clean up. The classes include: Window,Player,Monitor
08:59:57 T:1744289664   ERROR: XBPyThread::stop - script /mnt/sdcard/Android/data/com.flumpster.xbmc/files/.xbmc/addons/service.watchdog/service.py didn't stop in 5 seconds - let's kill it
08:59:57 T:1828030240    INFO: Python script interrupted by user
08:59:57 T:1744289664    INFO: Previous line repeats 1 times.
08:59:57 T:1744289664    INFO: Stopping script with id: 4
08:59:57 T:1818520392  NOTICE: script.lazytv : 81.217755 :: 80.705691 :::  -  (0, 9, 4) stopped
08:59:57 T:1818520392    INFO: Scriptresult: Success
08:59:57 T:1744289664   DEBUG: XBPyThread::stop - script termination took 59ms
08:59:57 T:1744289664    INFO: Stopping script with id: 6
08:59:58 T:1827960248    INFO: Scriptresult: Success
08:59:58 T:1744289664   DEBUG: XBPyThread::stop - script termination took 278ms
08:59:58 T:1818520392    INFO: Python script interrupted by user
08:59:58 T:1744289664  NOTICE: stop python
08:59:58 T:1744289664   DEBUG: waiting for python thread 1 (/mnt/sdcard/Android/data/com.flumpster.xbmc/files/.xbmc/addons/script.hautopc.after-watch/service.py) to stop
08:59:58 T:1744289664   DEBUG: python thread 1 (/mnt/sdcard/Android/data/com.flumpster.xbmc/files/.xbmc/addons/script.hautopc.after-watch/service.py) destructed
08:59:58 T:1744289664   DEBUG: waiting for python thread 2 (/data/data/com.flumpster.xbmc/cache/apk/assets/addons/service.skin.widgets/default.py) to stop
08:59:58 T:1744289664   DEBUG: python thread 2 (/data/data/com.flumpster.xbmc/cache/apk/assets/addons/service.skin.widgets/default.py) destructed
08:59:58 T:1744289664   DEBUG: waiting for python thread 4 (/mnt/sdcard/Android/data/com.flumpster.xbmc/files/.xbmc/addons/script.lazytv/service.py) to stop
08:59:58 T:1744289664   DEBUG: python thread 4 (/mnt/sdcard/Android/data/com.flumpster.xbmc/files/.xbmc/addons/script.lazytv/service.py) destructed
08:59:58 T:1744289664   DEBUG: waiting for python thread 5 (/data/data/com.flumpster.xbmc/cache/apk/assets/addons/script.common.plugin.cache/default.py) to stop
08:59:58 T:1744289664   DEBUG: python thread 5 (/data/data/com.flumpster.xbmc/cache/apk/assets/addons/script.common.plugin.cache/default.py) destructed
08:59:58 T:1744289664   DEBUG: waiting for python thread 6 (/data/data/com.flumpster.xbmc/cache/apk/assets/addons/script.xbmcbackup/scheduler.py) to stop
08:59:58 T:1744289664   DEBUG: python thread 6 (/data/data/com.flumpster.xbmc/cache/apk/assets/addons/script.xbmcbackup/scheduler.py) destructed
08:59:58 T:1744289664   DEBUG: waiting for python thread 8 (/mnt/sdcard/Android/data/com.flumpster.xbmc/files/.xbmc/addons/service.watchdog/service.py) to stop

To me, the significant lines are:
Code:
08:59:57 T:1744289664   ERROR: XBPyThread::stop - script /mnt/sdcard/Android/data/com.flumpster.xbmc/files/.xbmc/addons/service.watchdog/service.py didn't stop in 5 seconds - let's kill it
and the last line:
Code:
08:59:58 T:1744289664   DEBUG: waiting for python thread 8 (/mnt/sdcard/Android/data/com.flumpster.xbmc/files/.xbmc/addons/service.watchdog/service.py) to stop

This suggests to me that it wasn't able to close the watchdog service gracefully, but then further was unable to kill it forcibly and is therefore stuck waiting for it to die, and it never does.

Does anyone have any suggestions?


RE: [RELEASE] Library watchdog - takoi - 2014-05-02

(2014-05-02, 10:14)AlexVallat Wrote: Does anyone have any suggestions?
Please post the complete log. You have removed all the important parts!


RE: [RELEASE] Library watchdog - AlexVallat - 2014-05-02

(2014-05-02, 12:04)takoi Wrote: Please post the complete log. You have removed all the important parts!
Sorry! I thought I'd posted the important parts. OK, here is a complete log of a clean run, from startup, wait about a minute, then shutdown: http://xbmclogs.com/show.php?id=191185


RE: [RELEASE] Library watchdog - takoi - 2014-05-02

(2014-05-02, 13:22)AlexVallat Wrote: Sorry! I thought I'd posted the important parts. OK, here is a complete log of a clean run, from startup, wait about a minute, then shutdown: http://xbmclogs.com/show.php?id=191185
It hangs because you exit xbmc before watchdog have completely started. Either your network share isn't responding or it is very slow. Give it enough time to start, and it wont hang on exit. It's a known limitation. In the upcoming update for Gotham it is much better handled with a progress bar showing when it's safe to exit.