+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: General Support (https://forum.kodi.tv/forumdisplay.php?fid=111)
+---- Forum: OS independent / Other (https://forum.kodi.tv/forumdisplay.php?fid=228)
+---- Thread: Slow library updates using MySql and clients with different operating systems (/showthread.php?tid=377646)
Slow library updates using MySql and clients with different operating systems - nukeman - 2024-05-23
I'm using mySql shared video libraries and have Kodi installed on my nvidia shield and windows laptop. If I only update the library from the shield, scans go quickly. But, if I then do an update on the windows machine, the scan is much slower. Subsequent library updates on windows go quickly. If I then go back and do an update on the shield, it's slow again. Subsequent updates on the shield are quick.
I reviewed the debug log on a slow scan and find many entries like this:
Code:
2024-05-21 21:42:00.185 T:18564 debug <general>: VideoInfoScanner: Rescanning dir 'smb://nas.home/kids videos/Movies/Shrek (2001)/' due to change (0e1f090b967e89a6dfb9924f435040a8 != 5be2327d846c4035fbe27fa7aad0f864)
2024-05-21 21:42:00.728 T:18564 debug <general>: Mysql execute: update path set strHash='5be2327d846c4035fbe27fa7aad0f864' where idPath=70It looks like the strHash value is changing and requires updating. Is that why the scans are slow? Is this expected behavior? Are shared library updates expected to be performed on only one OS?
BTW, I'm using the The Movie Database Python and TMDb TV Shows providers on both machines.
RE: Slow library updates using MySql and clients with different operating systems - CrystalP - 2024-05-23
This reminds me of a recent similar report when scanning with Kodi clients of different platforms. Yes it sounds like Windows and Android record different hashes for the same directories.
You didn't mention the Kodi version and didn't include a debug log.
I think I vaguely remember @jogal committing a fix for something like that during the v21 dev cycle?
RE: Slow library updates using MySql and clients with different operating systems - crawfish - 2024-05-23
I determined the same thing toward the end of this thread:
https://forum.kodi.tv/showthread.php?tid=374496
User pkscout had some good advice that I've implemented, "Just pick one and have it do the scanning. The other one will then show that same information from the shared databases. Technically, you don't even need the sources defined on the second device at all." I can confirm what he said about removing the sources from the secondary devices, which removes the possibility of the problem at the cost of some potential inconvenience. (ETA: While you can still use the shared library on a device after removing sources, it doesn't help avoid the Update Library slowness problem if you mess up and perform it anyway. See follow-up message below.) Once you've updated library on the master device, there are various triggers that cause the UI to update on the secondary devices, but I implemented the direct method described here, which adds a button to the home screen:
https://forum.kodi.tv/showthread.php?tid=304913&pid=2647874#pid2647874
I did have to make one change, which was to change control_id from 805 to 806, as Kodi 21 Estuary now uses ID 805.
RE: Slow library updates using MySql and clients with different operating systems - nukeman - 2024-05-23
(2024-05-23, 17:40)CrystalP Wrote: You didn't mention the Kodi version and didn't include a debug log.I'm using Omega 21.0 on both devices. I can post a log if necessary but it's pretty long (thousands of TV episodes and hundreds of movies).
RE: Slow library updates using MySql and clients with different operating systems - nukeman - 2024-05-23
(2024-05-23, 19:43)crawfish Wrote: User pkscout had some good advice that I've implemented, "Just pick one and have it do the scanning.
I implemented the direct method described here, which adds a button to the home screen:
Although it's not necessarily intuitive, it's good to know this behavior is by design and not a bug. I like your idea of the button on the home screen. I mapped a custom button on my harmony remote that calls "updatelibrary(video)". That works pretty well too.
RE: Slow library updates using MySql and clients with different operating systems - crawfish - 2024-05-24
(2024-05-23, 22:33)nukeman Wrote: Although it's not necessarily intuitive, it's good to know this behavior is by design and not a bug. I like your idea of the button on the home screen. I mapped a custom button on my harmony remote that calls "updatelibrary(video)". That works pretty well too.
I'd say what's "by design" is that Kodi is primarily designed to use a local database, for which the issue cannot occur.
I don't think calling updatelibrary is the same thing at all as making that skin mod I linked to. The skin mod puts a button on the home screen that just causes the home screen to refresh and get in sync with the shared database WRT things like Currently Watching and Update Library made on a different flavor device, and it works instantly. Calling updatelibrary OTOH is what causes the slowness problem when you invoke it in Kodi running under a different OS (Android and Windows for sure, maybe others) than you last performed the updatelibrary on.
I have to retract what I said about removing sources from the secondary devices making it impossible to cause the problem. I just assumed it would work, and I must never have actually tried performing an Update Library since removing the sources from my secondary devices. I just did Update Library on my Android TV, where the sources are not defined, and to my surprise, it took several minutes, just like when the sources were defined, complete with UI updates in the top right of the screen on its progress. I then switched to my PC, where the sources are defined, and Updated Library, and it took several minutes. So, removing sources doesn't help avoid the problem. Apparently, the only thing that avoids it is the user avoiding invoking Update Library on secondary devices running different OSes (e.g. Android and Windows).
RE: Slow library updates using MySql and clients with different operating systems - jogal - 2024-05-25
(2024-05-23, 17:40)CrystalP Wrote: This reminds me of a recent similar report when scanning with Kodi clients of different platforms. Yes it sounds like Windows and Android record different hashes for the same directories.
You didn't mention the Kodi version and didn't include a debug log.
I think I vaguely remember @jogal committing a fix for something like that during the v21 dev cycle?
The PR is this https://github.com/xbmc/xbmc/pull/23631 but maybe now is other thing or file times/names are "wrong" in Android.
EDIT: I tested this in latest master and not happens in my setups: Android (Shield) / Windows (Intel NUC) / NFS sources on Synology NAS then this time is different issue and is not general.
RE: Slow library updates using MySql and clients with different operating systems - CrystalP - 2024-05-25
(2024-05-25, 09:11)jogal Wrote: EDIT: I tested this in latest master and not happens in my setups: Android (Shield) / Windows (Intel NUC) / NFS sources on Synology NAS then this time is different issue and is not general.
The log extract of OP shows smb, maybe that's the problem if nfs is fine? Don't know if you have the ability to try. I don't have an Android device.
@crawfish, do you share the files with smb, nfs, or other?
RE: Slow library updates using MySql and clients with different operating systems - crawfish - 2024-05-25
(2024-05-25, 19:13)CrystalP Wrote: The log extract of OP shows smb, maybe that's the problem if nfs is fine? Don't know if you have the ability to try. I don't have an Android device.
@crawfish, do you share the files with smb, nfs, or other?
SMB. My file server is a Windows 10 machine.