+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: General Support (https://forum.kodi.tv/forumdisplay.php?fid=111)
+---- Forum: OS independent / Other (https://forum.kodi.tv/forumdisplay.php?fid=228)
+---- Thread: Rebuilding Library, 12 hours and counting (/showthread.php?tid=366451)
Rebuilding Library, 12 hours and counting - sufreak - 2022-01-11
TL;DR - rebuilding library from RPi4 (LibreElec) to MariaDB on Syno 920.
Long version:
I had an old library I imported to MariaDB when I got my NAS. Previously MySQL on Ubuntu. For reasons, I decided to rebuild my library. Exported the NFO files, removed and reinstalled MariaDB, started up Kodi. Had to assign content to the videos, but once I did, it started updating the library. It would take 10-15 seconds per file it seemed. Currently working on my shows, which is 5K files. And it's 12 hours and counting. I enabled debugging in the middle of one of the updates to provide context.
https://paste.kodi.tv/eqanolibif.kodi
I'm not sure if it's because I'm using an RPi, or something on my MariaDB settings. Everything is wired, so there isn't a delay from that. 200/200 on FIOS, so not ISP blocking. I've noticed a slight lag as well from moving through menus.
my Mariadb my.conf file:
Quote:[mysqld]
skip-networking=0
skip-bind-address
skip-name-resolve
optimizer_search_depth=0
RE: Rebuilding Library, 12 hours and counting - Klojum - 2022-01-11
(2022-01-11, 15:24)sufreak Wrote: I'm not sure if it's because I'm using an RPi, or something on my MariaDB settings.
Could be a combo of both.
The RPi4 will store data on local databases for the thumbnail cache, as well as the thumbnail files themselves. If you have a mediocre SDcard, the write speed can be abysmal. Also the RPi4 is relatively slow in creating the thumbnails.
A database server on HDDs is not the slowest solution, but use a SSD with your database server and you have less 'lag'. Also, depending on the version, MariaDB can benefit from doubling cache settings.
Combine both the "troubles" of the RPi and the Syno, and perhaps a lazy internet connection to the information providers, times 5000+, and things can take quite a while. Having nfo files from the occasional video library export will save you so much rescraping time & trouble...
FYI, I run Ubuntu 18.04 LTS server with MySQL 5.7 quite trouble-free. It boots from a USB-SATA SSD, so all internal SATA ports can be used for disks. MariaDB on Raspbian & Ubuntu 20+ and also MySQL 8.0 all have their weirdnesses.
RE: Rebuilding Library, 12 hours and counting - sufreak - 2022-01-11
Thank you for the reply.
I used to run Ubuntu 18.04 LTS server with MySQL and no issues. For reasons, I moved some stuff over to the NAS.
I did have NFO files but they don't seem to be saving any time. I'll look into doubling cache settings.
RE: Rebuilding Library, 12 hours and counting - Klojum - 2022-01-11
ERROR: SQL: [MyMusic72] The table does not exist
BTW, if you also want to rescrape your music collection, something is off on the music database.
RE: Rebuilding Library, 12 hours and counting - sufreak - 2022-01-11
Thank you. That's next on my agenda, once I finish the videos. (When I dumped my DB, the sources lost their content settings)
RE: Rebuilding Library, 12 hours and counting - Klojum - 2022-01-11
(2022-01-11, 17:31)sufreak Wrote: When I dumped my DB, the sources lost their content settings
Because content settings are stored in the database
The sources.xml file is basically just a backup.
RE: Rebuilding Library, 12 hours and counting - sufreak - 2022-01-11
Thanks for that detail. I've been trying to keep sources in sync and realized no need.
On topic, my updated my.cnf file.
Quote:skip-networking=0
skip-bind-address
skip-name-resolve
optimizer_search_depth=0
innodb_buffer_pool_size=2G
innodb_buffer_pool_instances=2
Not a noticeable change really. Looking at this link, I think I covered the settings: https://forum.kodi.tv/showthread.php?tid=329046