[black_eagle]

" You'll find it listed under Program Addons as Video Database Cleaner. When you click Run you'll be shown the confirmation page of what will be cleaned based on your settings.. Check it looks right and then hit OK. The clean will then happen, followed by KODI's own Clean Library function.""

Does anyone know if this kind of datbase cleaner still exist?
(i have a full database and can't clean up any more.)

 Yes, it still exists and was recently updated to support Kodi 18 (thanks to @Kib) !! 

You can either download the zipped file from https://github.com/the-black-eagle/scrip...se.cleaner and install (making sure 'use unknown sources' is enabled), or you can download and install the repo (https://github.com/the-black-eagle/repos...blackeagle) and then install the cleaner from there.  

 Probably my ignorance but how can i instal this addon?
Can't find your link or "database Cleaner" or even your repository.blackeagle in "install from repository"
Did enable unknown sources. 

Easy way.  Click this link -> https://github.com/the-black-eagle/scrip...r/releases

Click on the 'script.database.cleaner.zip' to download it to your machine.  Save where you can easily find it.  Start kodi and install via the addon manager -> install from zipfile.

[pjotr69]

Thanks, stupid me, i couldend find the "install form zipfile" tab in kodi. Thought  how do i get to your source using repository if i can't search freely?
looked on the wrong place. its on add-ons; and then inside settingswheel. not in the systems or other add-ons tabs.
It's installed now.

Just out of curiousity a second noob question.(not related to your cleaning add-on)
i loaded a old backup as a test to see if it could change log in viewed or not and if clould remove a entrie (if so sd to small) (kodi 16 in to a kodi17 and it broke backup addon and other program add-ons so i couldend return to the backup i made yesterday to repair this fault.  and i didn't cloned the SD before this action.... stupid i know
So i place a kodi 16 from 042018 sd backup on a sd and worked from there to 17 by opkg upgrade in putty.
let this run its library's. (that's working now.)

But i like to see what add-ons i had installed in my former install, the broken one. So i like to clone this working SD (the one in my device) on a second SD and use just add-onn backup to reinstall my made backup before i broke the settings and such. (mind my back up is one upgrade behind because due of space issue it couldend upgrade) This way i can see what i had installed and which settings i used for the latest date.  Is this possible? place this backup on a higher version of kodi17 or does this create the same effect as i did before? (a broke kodi system)
(and yes the first time i ignored the warning about first save something else before i loaded the backup.. didn'know what that would bring, well it brought a broken kodi. LOL)

[black_eagle]

The easiest way to see what addons etc you had installed before is to mount the SD card in something and then navigate to the kodi addon directory.  In there will be a bunch of directories named, for example, script.artistslideshow, plugin.video.youtube, etc etc

Anything you don't recognise as an addon that you were using will be either a system addon that kodi needs, or a tool or library that one of the other addons requires to work properly.

Addons are not generally interchangeable between major kodi versions so addons for v16 will work on 16.0, 16.1, 16.2 but probably not on 17.0 etc.  If kodi has 'update addons automatically' turn on, then running v17 with v16 addons will cause lots of addons to be updated to the correct version for v17.  After this has happened, you cannot go back to v16 easily as kodi doesn't have a 'downgrade addons' function and therefore it will mark all those upgraded (v17) addons as broken.  This includes skins as well. 

Generally, I would advise making a backup of the entire addons directory and labelling it with the kodi version if you want to try other kodi versions.  Something like "Kodi-v16-addons-directory-02-09-18.zip".  If you want to keep all the settings as well for those addons, they are in a directory called 'addon_data' which itself lives in the userdata directory.  With backups of both those directories, you can 'roll back' to a previous version of kodi by running it once, then exiting and overwriting the appropriate directories with your backup(s).

[pjotr69]

Thus every time i do a putty opkg upgrade the backup of user info by kodi backuptool  is not compatible, if i try to replace it asked for first put advancedsettingsfile back before place back backup.
Even when it is a kodi 17.x version.
What is and does this advancedsettings.xml file?
can i on a win10pc just copy it out of the SD and safe it in a folder with version kodi, name and date and just copied back in case of using the backup of my userdata?
(for now i just go trough with it because its a test SD ( i have a working 17.x SD)
i just want to see the last state of kodi before my fxckup.)
But i want to install the latest distroversion on a larger SD (16Gb) in a wile fully building from scratch (this is still 17.x) and  when kodi 18 is suported by the distro builders and i use putty to upgrade then this question "advangedsettings file needs to be replaced by older one before putting back the backup" is pop up again i think . :-)

So what is the best way to backup the settings and such and use opkg upgrade  or if i want to install kodi 18 and use the back up as a fast install.
- just run your cleaner add-on first, take out SD, make  readcopy of that SD and after that go further in opkg upgrade?
(The opkg upgrate by putty migrates all version issues so that's safe but then the backup made by kodi is obsolete because of the advangedsettings.xml version difference which i never copy/backup apperantly.)

Generally, I would advise making a backup of the entire addons directory and labelling it with the kodi version if you want to try other kodi versions.  Something like "Kodi-v16-addons-directory-02-09-18.zip". 

By using a add-on which can profide that xxxx.zip file? (you see i am not familiar with kodi's advanged managment of backups)

I my experience most backups are older then you mostly want because when you discovering a problem backupping isn't usefull anymore except for returning to that point after pooking around. So version issues are 100% in practise because a upgrade or two is always the case and cloning the versions one by one is easy to forget.
Basic question is "how can i easy and version proof make a (userdata and all its nessecerry's )-backup of my running system?

Sorry for al those questions but every time something go's wrong i discover that the backups don't work the way i thought it worked and diving  in to a lot of non intended time consumming repairwork.
So i try to setup a foolsproof workflow. (i did by using two identical SD's which one was the working one and the other a clean backup from the last clean install. So in a crash i could just put that in and upgrade and update all. (noob version of fast recovering.)

[weak]

Thanks a lot @black_eagle for this great addon!
Cleaning my library now takes seconds instead of hours, like it did pre-cleanup...

[black_eagle]

Thanks a lot @black_eagle for this great addon!
Cleaning my library now takes seconds instead of hours, like it did pre-cleanup...

 Glad it's been of use to you

[HeresJohnny]

I love this add-on, thanks! However, currently it stopped working for me again. I'm on Leia 2018-09-23 nightly. When I start the add-on, it says there are 125 items to be cleaned. They don't seem to get cleaned, though. the next time I start the add-on, it still states there are 125 items to be cleaned. Maybe there was a recent core change or something?

[black_eagle]

I love this add-on, thanks! However, currently it stopped working for me again. I'm on Leia 2018-09-23 nightly. When I start the add-on, it says there are 125 items to be cleaned. They don't seem to get cleaned, though. the next time I start the add-on, it still states there are 125 items to be cleaned. Maybe there was a recent core change or something?

 Core change shouldn't make any difference.  The addon formulates it's own SQL queries based upon the settings you specify, and what it finds in your db.  It then executes those queries to delete stuff from the db and then optionally calls Kodi's own 'clean library' routine. If your problem is repeatable, please turn on debug logging for both kodi and the addon, run it and post a link to the debug log created.  Hopefully that will shed some light on what may be happening.

[tstrul]

Dumb question:
I have an antenna connected to my htpc  kodi and i use PVR.
i also have addon shortcuts in my favorites.
in addition, i use youtube and watch youtube trailers via my skin... but nothing related to library scan...
in my library paths i only have paths for downloaded videos and paths containing strm files to various addons

What would actually happen during the cleaning process... only DB entries for library sources are being deleted? if that's true why would my bookmarked videos be affected? or my youtube watched history/epg recordings? why library scan shpuld go over these things?

Should I exclude these things? what would happen if I won't do it? Will I lose my "watched" history in the youtube app itself?  why favorites should be related to library scan?

[black_eagle]

No, the addon does not remove entries for library sources.  What it will remove are links to streaming videos etc.  This shouldn't affect the Youtube addon as it keeps its history in a different place.  It will also remove the pvr:// links as well but that shouldn't affect any of the pvr functionality - eg, recordings, last played channels etc should all be supplied by your backend and not out of Kodi's video db.

You can always do a 'dry-run' by just using the default settings and hitting abort when it lists what it will remove.  If you are using an Sqlite3 db, the addon will automatically back this up first (in the same directory and append the current date and time to the name) so you can always quit Kodi, delete the current db and rename the backup to 'MyVideos112' or whichever version you have.  Upon restarting Kodi, it would then be using the original database again, with no changes made. Therefore it's possible to 'roll back' at any point, provided you make a backup first.  If you are using MySQL then the addon cannot back that up automatically, and you should do so yourself first.

When writing and testing the addon, it became apparent that some users never scanned anything into the library, preferring to just use the file browser for local files, along with various addons.  The original version of the addon would remove any bookmarks set on these local files as it considered them not part of any library.  We added the option to exclude removing them for these reasons.  Not sure why you are asking about favourites, the addon doesn't utilise that file at all.  It does look in your sources.xml file and excludes any files in those paths.

If you are worried about losing watched status on say, trailers, you can create an 'excludes.xml' file and specify in that what to exclude from cleaning.  See https://github.com/the-black-eagle/scrip...s.txt#L142 onwards for details.

[tstrul]

OK I understand
just ran it and it worked perfectly, library cleanup took seconds!.

Now i see the only thingIi can "mess up" is the watched state in various addons? because I really don't care about it  
thanks a lot!

[black_eagle]

OK I understand
just ran it and it worked perfectly, library cleanup took seconds!.

Nww i see the only thing i can "mess up" is the watched state in various addons? because i really doesnt care about it  
thanks a lot!

 No problem

[AnthonyB]

I'm trying to use this addon (v0.5.8) with Kodi 17.6 on Windows 10 with a MySQL 5.6.41 DB.
When I start the addon I get an error a few seconds later.

Full Kodi.log file here - https://pastebin.com/75atFf5z

Key section seems to be this:

14:04:14.444 T:29132   ERROR: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
                                             - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
                                            Error Type: <class 'xml.etree.ElementTree.ParseError'>
                                            Error Contents: not well-formed (invalid token): line 26, column 6
                                            Traceback (most recent call last):
                                              File "C:\Users\Anthony\AppData\Roaming\Kodi\addons\script.database.cleaner\default.py", line 368, in <module>
                                                advancedsettings = ET.parse(advanced_file)
                                              File "C:\Program Files (x86)\Kodi\system\python\Lib\xml\etree\ElementTree.py", line 1182, in parse
                                                tree.parse(source, parser)
                                              File "C:\Program Files (x86)\Kodi\system\python\Lib\xml\etree\ElementTree.py", line 656, in parse
                                                parser.feed(data)
                                              File "C:\Program Files (x86)\Kodi\system\python\Lib\xml\etree\ElementTree.py", line 1653, in feed
                                                self._raiseerror(v)
                                              File "C:\Program Files (x86)\Kodi\system\python\Lib\xml\etree\ElementTree.py", line 1517, in _raiseerror
                                                raise err
                                            ParseError: not well-formed (invalid token): line 26, column 6
                                            -->End of Python script error report<--

Any ideas why it is failing to load?


EDIT: nevermind. I just remember that this addon has a quirk in that it can't handle comments in the advancedsettings.xml (it's been a while since I used the addon!). As soon as I removed them all the addon worked fine. It would be nice if this could be fixed at some stage.

[black_eagle]

The error is actually in python itself, not specifically in the addon.  AFAICT, comments should be fine, provided they are correctly formed.

Code:

<!--  <network>
    <buffermode>1</buffermode>
   <!-- Value = 0 means use local storage, else, value is RAM cache -->

Whilst it works in an editor, it's not syntactically correct.  There is no closing tag for the first '<!--' and I imagine that is why python errors.  Comments work because the first line of my advancedsettings.xml is a comment and I have never had an error.

[AnthonyB]

Ah! OK. Thanks for clarifying. I'll go and clean it up and see if it works.