+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: General Support (https://forum.kodi.tv/forumdisplay.php?fid=111)
+---- Forum: OS independent / Other (https://forum.kodi.tv/forumdisplay.php?fid=228)
+---- Thread: Clarification of "Clean library..." & help to clean DB (/showthread.php?tid=294011)
RE: Clarification of "Clean library..." & help to clean DB - nickr - 2018-04-09
Yes but Karellen do you have all the junk streaming entries etc that are being complained about?
RE: Clarification of "Clean library..." & help to clean DB - Karellen - 2018-04-09
Nope, not at all.
But I assume that is what they are trying to remove- all the leftover streaming links.
So if they just export their library to NFO to save their valid data, delete the databases and scan the nfo's back into the library, they will rebuild their library, minus all the leftover crud. It will only import valid items.
RE: Clarification of "Clean library..." & help to clean DB - nickr - 2018-04-09
Oh you mean exporting individual files? Who wants all that nfo and picture crud all over their clean video storage?
RE: Clarification of "Clean library..." & help to clean DB - Karellen - 2018-04-09
I did mean Separate Files, but it can work with Separate or Single file. If the mixed media in the movie folder is not wanted, then use Single File and it will store metadata and images in one location of your choosing.
RE: Clarification of "Clean library..." & help to clean DB - DaVu - 2018-04-09
@nickr
or simply have your movies stored in folders matching the moviename. Then those nfo and fanart files will be stored in that folder as well. As clean as possible
RE: Clarification of "Clean library..." & help to clean DB - scott967 - 2018-04-10
(2018-04-08, 21:05)DGenerateKane Wrote: new mysql database so I could go back to the old when this inevitably didn't work) and that just made things worse for me. I exported the entire video library, but when I imported it to the new mysqldatabase I lost 90% of my TV Shows. What's worse, when I tried to go back to the old database, my library is still the new one missing all my shows. I don't know why either, since I was careful which one I had in the advancedsettings.xml. Updating the library only found a few more shows before hanging.
IDK, but just reading this it sounds like you have been maybe doing a lot of adjusting of your MySQL database and ended up with a mess. If it were my system I would probably drop the MySQL database, let Kodi build a new one and start over.
scott s.
.
RE: Clarification of "Clean library..." & help to clean DB - fr1day - 2018-04-10
For me cleaning of libraries is an issue due to the fact that I run a script every night that lower cases all files in the library (Nothing to do with KODI). if something new is scanned into the library after the lower-case script is run, then 24 hours later the file is un-playbable because on Linux XYZ is not the same as xyz. The only workaround I've found is to rename the file and live with the leftover database entries.
I'd love to see some additional work put into database in general, personally I see it as essential to have a single database for all instances of KODI in a house for things like watched status, consistent artwork etc.
RE: Clarification of "Clean library..." & help to clean DB - RuralHunter - 2018-04-13
It's really a shame that no dev looked at this obvious bug for so long time. I read the source and this is what I found:
1. The code is supposed to clean the files but it didn't somehow.
2. The SQL in CleanMediaType to get the files to delete is problematic:
The problem is that, if the file doesn't have a corresponding record in movie, tvshow... tables, it will never get deleted from files table. I'm not sure if this is the root cause but I think we should have a real dev to look at this.cpp:// now grab them media items
std:tring sql = PrepareSQL("SELECT %s.%s, %s.idFile, path.idPath, parentPath.strPath FROM %s "
"JOIN files ON files.idFile = %s.idFile "
"JOIN path ON path.idPath = files.idPath ",
table.c_str(), idField.c_str(), table.c_str(), table.c_str(),
table.c_str());
RE: Clarification of "Clean library..." & help to clean DB - Karellen - 2018-04-13
Yes, a similar observation here... https://forum.kodi.tv/showthread.php?tid=323597&pid=2679032#pid2679032
A lot (not all!) of these problems are the result of users not correctly removing library items from Kodi. The correct way is here... https://kodi.wiki/view/Updating_or_removing_videos
It is not being looked at. If you have some knowledge, why not contribute?
RE: Clarification of "Clean library..." & help to clean DB - RuralHunter - 2018-04-13
No, this is not a problem of users if my findings are correct. Maybe my description is not detailed enough. I believe that a file having a record in files table while not in movie table is normal, if the file has no meta data scraped. This leads the file and the path related to it will not get deleted forever.
Regarding the contribution, I think reading the source and pointing out the possible root cause is one kind of contribution. In most situation, we need a real dev to look at this because I don't have deep enough understanding of the code and my findings maybe not correct, not to think about to correct the source code.
RE: Clarification of "Clean library..." & help to clean DB - fr1day - 2018-04-13
(2018-04-13, 06:31)RuralHunter Wrote: The problem is that, if the file doesn't have a corresponding record in movie, tvshow... tables, it will never get deleted from files table. I'm not sure if this is the root cause but I think we should have a real dev to look at this.I wonder could this be due to the order that clean library does things? If the corresponding records for a movie/TV Show are deleted before it attempts to delete from the file table, then a file will never get deleted.
RE: Clarification of "Clean library..." & help to clean DB - RuralHunter - 2018-04-13
(2018-04-13, 10:23)fr1day Wrote:Nope. The movie/tv deleting happens after the files deleting.(2018-04-13, 06:31)RuralHunter Wrote: The problem is that, if the file doesn't have a corresponding record in movie, tvshow... tables, it will never get deleted from files table. I'm not sure if this is the root cause but I think we should have a real dev to look at this.I wonder could this be due to the order that clean library does things? If the corresponding records for a movie/TV Show are deleted before it attempts to delete from the file table, then a file will never get deleted.
RE: Clarification of "Clean library..." & help to clean DB - scott967 - 2018-04-14
I'm not sure what the requirement is. I think many things such as streamed files never have entries in the library. I always thought "cleaning" was about library items, not file table items.
scott s.
.
RE: Clarification of "Clean library..." & help to clean DB - fr1day - 2018-04-14
(2018-04-14, 00:41)scott967 Wrote: I'm not sure what the requirement is. I think many things such as streamed files never have entries in the library. I always thought "cleaning" was about library items, not file table items.If that's the case, then maybe the clean function needs to be expanded to include the cleaning of all items that are no longer valid. What possible benefit is there to a database keeping information on files that no longer exist?
scott s.
.
RE: Clarification of "Clean library..." & help to clean DB - scott967 - 2018-04-14
(2018-04-14, 13:19)fr1day Wrote:I don't know the history of "files", but it seems to serve as the entry point for storing playcount / last played, stream details, resume point, bookmarks, etc. I assume this was done to allow for keeping info on streamed video across Kodi sessions.(2018-04-14, 00:41)scott967 Wrote: I'm not sure what the requirement is. I think many things such as streamed files never have entries in the library. I always thought "cleaning" was about library items, not file table items.If that's the case, then maybe the clean function needs to be expanded to include the cleaning of all items that are no longer valid. What possible benefit is there to a database keeping information on files that no longer exist?
scott s.
.
scott s.
.