[skylarking]

I apologize if this has been asked before, but I couldn't find anything.

I'm trying to clean up invalid entries that have somehow crept into the 'episode' table - I have rows here for files which have since been deleted, but no combination of this tool nor the traditional clean/update are removing them. Subsequently, my "total/viewed" counts are skewed in the show overview page of Kodi.

Any thoughts on the best method to clean this table out?

As i've understood, the original intent of this database pre wash scrub was to remove the hundreds of streaming entries stored wihin the DB which caused the kodi database clean function to choke and seemingly hang.
So, if you are not having an issue with the database clean function choking, you really don't have an issue with such fragments.

IIRC, these invalid fragments within your database tables are simply dead entries so to speak and occur when one does not remove movies or episodes via kodi itself.
These dead entries should pose no issue as such but i do like a clean database as it satisfies my OCD.

One of the simplest ways to get a clean database is to export your library as separate files (including actor thumbnails and such otehr bits it askes for during the export).
This does result in .nfo, jpg and other metadata files being stored alongside your media files which some don't like.
Also, if you want to preserve your watched state, you need an appropriate entry in the advansedsettings.xml file to force import of watch state from the nfo's.
Once the export is done, simply redefine your sources and then scan for new media at which point all your local media is imported using the local metadata saved during the export process.
At this point you have a clean database without reminants or other such fragments.

The benefit of doing this periodic export is that it makes recovery from database corruption rather simple as all your metadata is stored locally.
If this seems like too much, i believe that you can use database.cleaner to remove these entries but i've not looked into it myself as i prefer the export as a backup so to say.

[black_eagle]

Just to expand a little on @skylarking 's excellent answer, database.cleaner deletes stuff from the 'files' table, whilst optionally pulling info from the 'bookmarks' table.  It doesn't touch any of the other tables in the database as removing items from the files table is enough for Kodi's built in clean to remove any other stuff related to those removed file paths.

Personally, if it were me, I'd export everything as separate files, quit Kodi, drop the database (presuming it's some form of MySQL db - else just delete it !!) then restart Kodi and re-import everything.  Just make sure that all the exported stuff is actually exported and in the right directories before trashing anything !! (and/or, keep a backup of the db just in case, until you're sure everything is imported again correctly).

[MAX404]

Latest (Leia compatible) version can be downloaded from here. V0.6.0 is the Leia version (should work with both MySQL and SQlite databases).

Hi mate

Thanks a lot , i will give it a try right away , any mayor changes worth mentioning?

[nevernow]

I think this tool is not compatible with the latest developments of Kodi's archive support (moved to an add-on, but there are two distinct add-ons providing RAR support). It wants to delete all my RAR'd movies, which appear in the log as

Code:

archive://%2fvar%2fmedia%2fwarez%2fMovies%2fRELEASENAMEHERE%2fRELEASENAMEHERE.rar/

They are multipar RAR archives, scene standard. It would be great to have this add-on support archives.

Archives have nothing to do with anything (or at least, whilst support was in core they didn't - Guess now they are spun out that things may have changed).  However, the addon was originally developed to remove all the crud that is needlessly added to the video database whenever you stream something, be that via netflix, hulu, http(s), rtmp, youtube or whatever.  ALL those videos get added to the database even though they are not in your local library and they all get processed when running a library update with the result that the update is much slower than it ought to be, plus the fact that all that history is retained, possibly for years, through different Kodi versions and library migrations. Imagine the time to migrate those 5000 youtube videos you've watched over the past 3 years, plus all the attempting to look-up art, find a synopsis etc etc [...]

First of all, thanks for your great, detailed answer! And sorry it took me a while to reply. I will try adding an exception for archives, although this will make the cleanup less efficient since all my movies are RAR'd. I understand this tool is designed to clean up streaming stuff that shouldn't end up in the database in the first place and any cleaning of "ghost content" is... collateral benefit. It's a great tool and thanks for making it.
The archive support spin-off was really messy and rushed, I think, but of course this is a discussion for another thread.

[amcfarla]

This script fixed my hung video library cleaning issue.  Thank you for making this available 

[lamster]

Sorry if this is a daft question, but is there a reason why this plug-in would cause an NV shield TV to reboot when trying to install it? I've tested the .zip on Debian's Kodi on a regular x86 PC and the current Kodi from the Windows store on a laptop, both using the same SD card with no problem. But using it on the shield results in the device rebooting before any installation message pops up. There is no message, no warning or exception, it just goes dark until the powering up animation appears.

[black_eagle]

is there a reason why this plug-in would cause an NV shield TV to reboot when trying to install it?

Not that I'm aware of.  Can you get a debug log of you trying to install it ?  If you need to upload the log from within Kodi then

• Start Kodi
  
• Turn on debugging
  
• Attempt to install addon from zip file
  
• Restart Kodi after issue
  
• Upload the kodi-old.log
  

Also, is this the only addon you are having an issue installing ?

[lamster]

is there a reason why this plug-in would cause an NV shield TV to reboot when trying to install it?

Not that I'm aware of.  Can you get a debug log of you trying to install it ?  If you need to upload the log from within Kodi then

• Start Kodi
  
• Turn on debugging
  
• Attempt to install addon from zip file
  
• Restart Kodi after issue
  
• Upload the kodi-old.log
  

Also, is this the only addon you are having an issue installing ? 

Thanks. I'll try that. I don't use plug-ins, other than the basic defaults from the repository for media meta. I've no idea why the clean functionally broke, I don't use removable media and the NFS shares are accessible and work fine.

[lamster]

is there a reason why this plug-in would cause an NV shield TV to reboot when trying to install it?

Not that I'm aware of.  Can you get a debug log of you trying to install it ?  If you need to upload the log from within Kodi then

• Start Kodi
  
• Turn on debugging
  
• Attempt to install addon from zip file
  
• Restart Kodi after issue
  
• Upload the kodi-old.log
  

Also, is this the only addon you are having an issue installing ?  

Thanks. I'll try that. I don't use plug-ins, other than the basic defaults from the repository for media meta. I've no idea why the clean functionally broke, I don't use removable media and the NFS shares are accessible and work fine. 

Seeing as I can't see how to edit the above...

After 30+ mins of faffing about trying to find the log file on the device (I couldn't), I installed the kodi log file plug-in and started over. Instead of trying to install
from the menu options that appear on the left in kodi, I noticed the icon at the top of the page to install from zip, and it worked! Net result, it found 27 problems and the clean functionality can complete. I've no idea what those 27 were or where they came from, which is a pity as it would give clues about the source problem had details been shown... Anyway, the device is happy again, thanks!

[black_eagle]

the device is happy again, thanks!

That's great to hear that you got it all sorted in the end

[strang]

I'm getting the following error when I try to run the add-on: 

Code:

ERROR: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<-- - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
Error Type: <class 'sqlite3.OperationalError'>
Error Contents: near ")": syntax error
Traceback (most recent call last):
    File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.database.cleaner/default.py", line 680, in <module> cleaner_log_file(our_select, False)
      File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.database.cleaner/default.py", line 285, in cleaner_log_file
           cursor.execute(our_select) OperationalError: near ")": syntax error

-->End of Python script error report<--

Here's the full debug log: tovatemumu (paste)

[black_eagle]

That's an old version of the addon.  You're running 0.5.9 and the current version is 0.6.1  The version that you have is not compatible with the Leia database.

sql:
2019-11-17 15:41:25.677 T:5351 NOTICE: Running database version MyVideos116

sql:
2019-11-17 17:47:08.316 T:7655 DEBUG: Video Database Cleaner: script version 0.5.9 started

2019-11-17 17:47:10.317 T:7655 DEBUG: Video Database Cleaner: Checking for local database MyVideos114

2019-11-17 17:47:10.317 T:7655 DEBUG: Video Database Cleaner: Checking for local database MyVideos113

2019-11-17 17:47:10.317 T:7655 DEBUG: Video Database Cleaner: Checking for local database MyVideos112

2019-11-17 17:47:10.317 T:7655 DEBUG: Video Database Cleaner: Database name is MyVideos112

Please update to the latest version and see if the error still occurs.

[HeresJohnny]

Trying to update from within Kodi the latest version your repository reports is actually 0.5.9.

[black_eagle]

Trying to update from within Kodi the latest version your repository reports is actually 0.5.9.

Thanks for the heads up !!  I shall address that as soon as I have time.  In the meantime, the latest version is available here.

[HeresJohnny]

REQUEST for a Kodi M compatible version (update to python3). Thanks in advance!