[black_eagle]

Cheers matey it all seems to be working as it should here using latest krypton nightly

Great !! Thanks for the feedback

[fernandovg]

why don't you clean "path" table if if strPath doesn't exist anymore OR if strPath begins with "http" ?

I have a bunch of paths in "path" that doesn't exist anymore and a lof of "http" there

UPDATE: seeing your code, you don't delete when protocol is "https://"

[skylarking]

Hmm... as an FYI, i couldn't download script.database.cleaner-0.5.6.zip from dropbox using Firefox 47, oddly it worked as expected using IE11

Any chance Black_Eagle can update his repo or is he M.I.A?

[black_eagle]

why don't you clean "path" table if if strPath doesn't exist anymore OR if strPath begins with "http" ?

I have a bunch of paths in "path" that doesn't exist anymore and a lof of "http" there

UPDATE: seeing your code, you don't delete when protocol is "https://"

No, currently 'https' is not removed however that is trivial to add. Probably it's not in there as no-one originally reported having entries with https in them.

DBC doesn't clean the 'paths' table. It cleans the 'files' table and then calls Kodi's built in cleaning routine, which takes care of the other tables. As it doesn't check anything on disk, it is impossible for the addon to know whether or not a path is valid. The assumption is 'If a path does not exist in sources.xml but does in the database, then it is safe to remove'. The 'just remove streaming links' is there because :-

a) it's faster
b) not everyone adds sources to Kodi

Cheers

[black_eagle]

Hmm... as an FYI, i couldn't download script.database.cleaner-0.5.6.zip from dropbox using Firefox 47, oddly it worked as expected using IE11

Any chance Black_Eagle can update his repo or is he M.I.A?

I don't have access to the repo unfortunately. I dunno what happened with BatterPudding

I'll look into getting a repo hosted on github and then I can update via that. It won't be for a couple of weeks though as RL is very busy currently !!

Regarding dropbox working with one browser and not the other.... No clue I'm afraid. At least you figured a workaround.

[Gerdistan]

Hello sirs!

The addon shown is this thread seems to be exactly what I need, but sadly, I can't get it to work.

My problem: I have a huge library (somewhere between 3 and 4 TB of stuff). I recently switched from SMB to NFS for the shares containing these files, I rescanned the library and now everything shows double. Now I know that I should have removed the old sources and not changed them from SMB to NFS, but I guess that's too late now.

The usual Kodi Database Cleanup results in the system hanging up at 1% or even before.

I tried your tool several times, first with no funky settings, in the second run I tried to remove everything that starts with "smb://", but neither of which did anything. And the worst part is: I can't even find a logfile, nowhere on my machine, not in the tmp nor in the logfiles folder.

Any help is appreciated :-)

[black_eagle]

Hello sirs!

The addon shown is this thread seems to be exactly what I need, but sadly, I can't get it to work.

My problem: I have a huge library (somewhere between 3 and 4 TB of stuff). I recently switched from SMB to NFS for the shares containing these files, I rescanned the library and now everything shows double. Now I know that I should have removed the old sources and not changed them from SMB to NFS, but I guess that's too late now.

The usual Kodi Database Cleanup results in the system hanging up at 1% or even before.

I tried your tool several times, first with no funky settings, in the second run I tried to remove everything that starts with "smb://", but neither of which did anything. And the worst part is: I can't even find a logfile, nowhere on my machine, not in the tmp nor in the logfiles folder.

Any help is appreciated :-)

Which operating system are you using, kodi version and add-on version ?

[Gerdistan]

Kodi 16.1 on a Raspberry Pi 3 (LibreELEC 7.0.2), Addon Version 0.5.6 (Dropbox Version)

[black_eagle]

OK. So, log files (both the kodi log & the add-on's log) should be in '~/.kodi/temp'.

I guess from the 'tmp' and 'logfiles' directories you mentioned, that you are accessing those from another machine on the network. Easiest way in that case to get the logs is to ssh to the pi (you can use putty if on windows) with a username of root and password of libreelec .

Once connected do cp .kodi/temp/*.log .kodi/userdata/ . Note that the spaces and dots are important !! That will copy the logs into the userdata directory which you can then access over smb to pull them off and read them.

If you set the add-on to create the database-cleaner log, then there should be some useful information in that (I hope !!)

[Gerdistan]

Thanks for your answer - had a little trouble following your orders, at least on my RPi, the .kodi folder hides in /storage. But i managed to find the logs, which say....


ideo Database Cleaner V0.5.6 - Running at Friday, 21. October 2016 0808:21:04

There are 0 paths in the database that meet your criteria

The following paths were removed from the database

No paths have been found to remove

---


I think I have a hunch on the problem. I have several MyVideos*.db files in the Database folder, I guess the addon is checking one an kodi is using another. Will look into this tomorrow, gotta run now.

[black_eagle]

Add-on checks the highest db No. it finds. So if you have upgraded or installed a beta or nightly and have a higher db version No, then that's what the add-on will connect to.

It's possible to correct this in the advanced settings of the add-on by specifying the correct name of the db to connect to. This can however lead to issues as the 'clean library' routine called by the add-on will attempt to clean the db connected to by the current version of Kodi. This may not be the same version if you have previously installed or used a beta/nightly version which has upgraded the databases.

[junitodx]

Hi,

I got the add-on to work for me before but for some reason it's no longer functioning.It starts for a second then errors out and says to check the logs.

The database-cleaner.log only has one line in it which is:
Video Database Cleaner V0.5.6 - Running at Saturday, October 22, 2016 04:12:20 PM



I also checked the spmc.log and saw found the following at the very end of the log after the database cleaner failed:

Error Type: <class 'sqlite3.OperationalError'>
Error Contents: Could not decode to UTF-8 column 'strPath' with text 'rtmp://89.248.172.239/live?wmsauthsign=c2vydmvyx3rpbwu9mtavos8ymde2idg6mzm6mzigue0magfzaf92ywx1zt0xm3jztwcxvk00cjivvlzpek1lajdbpt0mdmfsawrtaw51d
Traceback (most recent call last):
File "/storage/emulated/0/Android/data/com.semperpax.spmc16/files/.spmc/addons/script.database.cleaner/default.py", line 742, in <module>
cleaner_log_file(our_select, False)
File "/storage/emulated/0/Android/data/com.semperpax.spmc16/files/.spmc/addons/script.database.cleaner/default.py", line 352, in cleaner_log_file
my_data = cursor.fetchall()
OperationalError: Could not decode to UTF-8 column 'strPath' with text 'rtmp://89.248.172.239/live?wmsauthsign=c2vydmvyx3rpbwu9mtavos8ymde2idg6mzm6mzigue0magfzaf92ywx1zt0xm3jztwcxvk00cjivvlzpek1lajdbpt0mdmfsawrtaw51d
-->End of Python script error report<--
16:12:22 T:1433397552 ERROR: Error loading addon settings to special://profile/addon_data/script.skinshortcuts/settings.xml


I am using SPMC, Metropolis Skin and database cleaner version 0.5.6

The only thing I think I remember installing between the last time I used this add-on when it worked and now is Rating Update v1.4.3 which I could not get to work so I uninstalled it.

Thanks in advance for any assistance

[black_eagle]

Hi junitodx.

That error would appear to be occurring in one of the python modules that the addon uses to connect to the database. I don't have an android device to test with but in any case, I don't really think that it would make a difference.

As a start, if you disable the database-cleaner log file and try to run the addon again. That may let you actually do the clean without causing the error. If not, you should still be able to remove all streaming paths by turning OFF sources.xml in the settings. This should prevent the addon trying to determine which paths to remove or retain and just remove any streaming ones.

In the meantime, I'll try to reproduce your error, although if it is occurring inside the sqlite3 module (which it appears to be) then I doubt that there is much I can really do. I'll check out the SPMC packages to see if there is a version difference.

[junitodx]

Hi junitodx.

That error would appear to be occurring in one of the python modules that the addon uses to connect to the database. I don't have an android device to test with but in any case, I don't really think that it would make a difference.

As a start, if you disable the database-cleaner log file and try to run the addon again. That may let you actually do the clean without causing the error. If not, you should still be able to remove all streaming paths by turning OFF sources.xml in the settings. This should prevent the addon trying to determine which paths to remove or retain and just remove any streaming ones.

In the meantime, I'll try to reproduce your error, although if it is occurring inside the sqlite3 module (which it appears to be) then I doubt that there is much I can really do. I'll check out the SPMC packages to see if there is a version difference.

Thanks for taking a look black_eagle. I tried what you said and after checking and unchecking different variations of what you suggested it was only the database-cleaner.log. After unchecking that I was able to see the Summary screen but I don't have the Abort and Do It options so I still can't actually run it.

[black_eagle]

Hmm, yeah....That would be an issue !!

TBH, the summary window is something I need to address fairly badly. At the moment, it's constructed totally 'on-the-fly' by the add-on. The only things in the actual skin file are the title, icon and the background. Everything else is written by making API calls to some stuff internal to Kodi. Problem with this is that it makes it look slightly different on different platforms and obviously in your case, it's failing to generate the buttons.

Assuming that those paths are correct, then if you turn off the summary window in the add-ons settings, it'll not show that window but instead proceed straight to the clean. This option was added for people who like to fire it once a week on a timer so that it can run without user intervention.

I know this is a bit of a hacky work-around but it should at least let you run it and clean your db up.

As I said before, I don't have a device that can run SPMC to test with, although I'm hoping to change that in a few days time !

Good luck....Please report back whether this works for you or not...And thankyou for the bug reports.