+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27)
+---- Forum: Program Add-ons (https://forum.kodi.tv/forumdisplay.php?fid=151)
+---- Thread: Release Kodi Selective Cleaner addon support (/showthread.php?tid=369984)
RE: Kodi Selective Cleaner addon support - jbinkley60 - 2023-10-30
I had some more time today to do some cleanup and add the progress dialog box. Test version 1.0.3d has been posted here. One question I have is whether folks would want an analyze all tables with a CSV export of the results, in addition to a clean all tables function ? The output would need to be a CSV output. Trying to analyze all tables and then writing to the screen doesn't seem practical nor user friendly.
This 1.0.3d version is pretty close to a next release version, unless folks want the analyze all tables function or someone finds a bug during testing.
Thanks,
Jeff
RE: Kodi Selective Cleaner addon support - HeresJohnny - 2023-10-31
Thank you for your work, as always!
I ran the test version and cleaned all tables without a problem. I am still somewhat surprised that deleteable elements number in the thousands although I curate my db pretty carefully.
Here are some suggestions:
* I think the GUI would look cleaner if the enumeration of the number of elements would move inside of VIDEO DB DATA ANALYER and MUSIC DB DATA ANALYER respectively.
* Personally , I would prefer not a separate log for each cleaning session but a continuous log.
Wishlist for the future:
* Clean links to http:// and https:// video objects that are entered permanently into the video tables after having viewed them (think YouTube and others). Those are just useless bloat.
* Trigger a native "clean library" after manual cleaning as that will also compress the database (not sure if KSC does that by itself).
RE: Kodi Selective Cleaner addon support - jbinkley60 - 2023-11-01
(2023-10-31, 21:23)HeresJohnny Wrote: I think the GUI would look cleaner if the enumeration of the number of elements would move inside of VIDEO DB DATA ANALYER and MUSIC DB DATA ANALYER respectively.
Are you saying that this screen below should show the number of elements in each table along with the table names ? If so, that would be easy to add. The addon would be slightly slower loading while it does the counts but it should still be very quick. If not, please explain further what you are looking for:
(2023-10-31, 21:23)HeresJohnny Wrote: Personally , I would prefer not a separate log for each cleaning session but a continuous log.
Again can you be a little more specific ? Right now the log events are written to a searchable database. Are you wanting a search by table history or are you thinking of a continuous text file ?
(2023-10-31, 21:23)HeresJohnny Wrote: Clean links to http:// and https:// video objects that are entered permanently into the video tables after having viewed them (think YouTube and others). Those are just useless bloat.We'd have to think about this and likely put some parameters around it since other sharing solutions use http/https for local URLs for video playback. Unchecked this would blow away valid library entries.
Thanks for the feedback and testing,
Jeff
RE: Kodi Selective Cleaner addon support - HeresJohnny - 2023-11-03
(2023-11-01, 00:42)jbinkley60 Wrote: Are you saying that this screen below should show the number of elements in each table along with the table names ? If so, that would be easy to add. The addon would be slightly slower loading while it does the counts but it should still be very quick. If not, please explain further what you are looking for:
Sorry for the confusion, I propose to eliminate one level by moving numbered elements inside the next level like this:
(2023-11-01, 00:42)jbinkley60 Wrote: Again can you be a little more specific ? Right now the log events are written to a searchable database. Are you wanting a search by table history or are you thinking of a continuous text file ?
Sorry again. On the machine I tried out the add-in, it seems there was no previous cleaning session so I wrongly assumed the logs would be overwritten each session. I had a look at it on another machine and realized that there actually is a separate log for each cleaning session. Well thought out!
(2023-11-01, 00:42)jbinkley60 Wrote: We'd have to think about this and likely put some parameters around it since other sharing solutions use http/https for local URLs for video playback. Unchecked this would blow away valid library entries.
Yep, it would mean deleting library entries but let's realize it, this tool already carries the danger of making breaking changes and should only be used by adventurous users or diligent backuppers :-)
RE: Kodi Selective Cleaner addon support - jbinkley60 - 2023-11-04
(2023-11-03, 19:50)HeresJohnny Wrote: Sorry for the confusion, I propose to eliminate one level by moving numbered elements inside the next level like this:
So you are thinking of like a submenu under the Video Analyzer ? If so, I'll need to think about that. The original purpose of the three menu items: Movies, TV Shows and Music Videos was to allow a drill down into your database and ultimately delete an individual item in your database table or possibly even allow updating fields in the table for the item. The Video Analyzer portion is more of a bulk analyze/clean tool. I stopped working on the drill down section of the addon (which is why you se it stop when you get to the individual elements) because there was more interest in the Video Analyzer. If folks have suggestions on the drill down section, I'll consider working on it more.
(2023-11-03, 19:50)HeresJohnny Wrote: Sorry again. On the machine I tried out the add-in, it seems there was no previous cleaning session so I wrongly assumed the logs would be overwritten each session. I had a look at it on another machine and realized that there actually is a separate log for each cleaning session. Well thought out!
Currently the logger keeps the newest 10,000 records. When the addon starts it deletes 10,0001+ and higher. It's pretty simple. It doesn't have a logs by table search / query utility right now. I could add that with some work to flag the logs by table and then add the additional queries. The idea would be if you wanted a historical record by table. The more interesting potential option is a verbose detailed log of why something got cleaned (i.e The value X in table Y field Z was value ABC and had no corresponding index entry in table W. )
(2023-11-03, 19:50)HeresJohnny Wrote: Yep, it would mean deleting library entries but let's realize it, this tool already carries the danger of making breaking changes and should only be used by adventurous users or diligent backuppers :-)I understand but I still think there would need to be the ability to add exceptions (i.e. delete You Tube URLs but not My Media Server URLs etc..).
Thanks,
Jeff
RE: Kodi Selective Cleaner addon support - jbinkley60 - 2023-12-17
I have posted a new test version 1.0.3g of Kodi Selective Cleaner. This is hopefully the final test release before 1.0.4. This version has a number of edits and features, including the ability to fix missing music database triggers. This is from a forum thread here.
Here's the complete list of highlights since 1.0.3
• Added detection for Python version
• Added file copy database backups for Python 3.6 and below
• Added single click clean all tables option
• Updated video database to align to Kodi change with Kodi 21 beta 2
• Added option to check and replace missing music database triggers for Local
databases and detection for MySQL with the standard Kodi database naming
Feedback and testing is appreciated. I'll hold this for 2 weeks and then release as 1.0.4.
Thanks,
Jeff
RE: Kodi Selective Cleaner addon support - jbinkley60 - 2024-01-10
A new release version 1.0.4 of Kodi Selective Cleaner has been posted.
Here's the complete list of highlights since 1.0.3
• Added detection for Python version
• Added file copy database backups for Python 3.6 and below
• Added single click clean all tables option
• Updated video database to align to Kodi change with Kodi 21 beta 2 (database 124)
• Added option to check and replace missing music database triggers for Local
databases and detection for MySQL with the standard Kodi database naming
Feedback and testing is appreciated. I am heading out on vacation next week and ran out of time to
add analyzing the uniqueid table into the tool. I have the basic queries written but there are enough
code changes required to add and test a new table, I didn't want to risk breaking something while I
was away. I'll add it as soon as I am back.
Thanks,
Jeff
RE: Kodi Selective Cleaner addon support - jbinkley60 - 2024-01-12
(2023-11-03, 19:50)HeresJohnny Wrote:
Quote:We'd have to think about this and likely put some parameters around it since other sharing solutions use http/https for local URLs for video playback. Unchecked this would blow away valid library entries.Yep, it would mean deleting library entries but let's realize it, this tool already carries the danger of making breaking changes and should only be used by adventurous users or diligent backuppers :-)
I've been thinking about this more and I like your suggestion but have been trying to think of a somewhat safe way to implement it. Here's a thought I have. What if I add a menu option between Music and Video DB Data Analyzer called URLs ? It would operate similarly to Movies, TV Shows etc. whereas you could browse URLs by the main URLs (i.e. https://www.youtube.com, https://otuervideowebsite.com etc..). It would allow you to see what's under each one just for informational purposes prior to doing any deletions. Then add a menu item to allow deletion of all items under the main URLs so you could, as you ask, delete all files associated with https://www.youtube.com or some other high level video URL. This feels like it might strike the right balance and provides some level of safety.
Thoughts ?
Jeff
RE: Kodi Selective Cleaner addon support - PnoT - 2024-01-24
Does the application support using the username and password for the Kodi database in the advancedsettings file?
I can't seem to get it to connect to my database and here is the error: error <general>: EXCEPTION IN (C:Kodi\addons\plugin.program.kscleaner\resources\lib\common.py, LINE 189 "db = mysql.connector.connect(**config)"): connect() argument after ** must be a mapping, not NoneType
I've tried:
default
xbmc_videos
xbmc_videos121
My adv file contains the db name, IP of server, username, and password.
RE: Kodi Selective Cleaner addon support - jbinkley60 - 2024-01-25
(2024-01-24, 22:23)PnoT Wrote: Does the application support using the username and password for the Kodi database in the advancedsettings file?
I can't seem to get it to connect to my database and here is the error: error <general>: EXCEPTION IN (C:Kodi\addons\plugin.program.kscleaner\resources\lib\common.py, LINE 189 "db = mysql.connector.connect(**config)"): connect() argument after ** must be a mapping, not NoneType
I've tried:
default
xbmc_videos
xbmc_videos121
My adv file contains the db name, IP of server, username, and password.
Yes, Kodi Selective Cleaner reads your advancedsettings.xml file to get username, password etc. See here for more information. That error is a generic error which I will fix in the next release of code. It typically means there was a problem parsing your advancedsettings.xml file. Can you post all of the exception messages from your Kodi.log file. ? This is typically the second one. The first one which is likely the XML parsing exception should hold the key. Worst case I may need to get a copy of your advancedsettings.xml file to see the cause. One cause I often see is a space character as the first character on the first line. If there is a space try removing it. That is being fixed in the next release too where the XML parser will ignore it.
Thanks,
Jeff
RE: Kodi Selective Cleaner addon support - PnoT - 2024-01-25
(2024-01-25, 15:01)jbinkley60 Wrote:(2024-01-24, 22:23)PnoT Wrote: Does the application support using the username and password for the Kodi database in the advancedsettings file?
I can't seem to get it to connect to my database and here is the error: error <general>: EXCEPTION IN (C:Kodi\addons\plugin.program.kscleaner\resources\lib\common.py, LINE 189 "db = mysql.connector.connect(**config)"): connect() argument after ** must be a mapping, not NoneType
I've tried:
default
xbmc_videos
xbmc_videos121
My adv file contains the db name, IP of server, username, and password.
Yes, Kodi Selective Cleaner reads your advancedsettings.xml file to get username, password etc. See here for more information. That error is a generic error which I will fix in the next release of code. It typically means there was a problem parsing your advancedsettings.xml file. Can you post all of the exception messages from your Kodi.log file. ? This is typically the second one. The first one which is likely the XML parsing exception should hold the key. Worst case I may need to get a copy of your advancedsettings.xml file to see the cause. One cause I often see is a space character as the first character on the first line. If there is a space try removing it. That is being fixed in the next release too where the XML parser will ignore it.
Thanks,
Jeff
I looked again at the logs, and it was complaining about not being able to parse the port. further inspection of my advancedsettings.xml showed that for the videos database, there was no <port> entry. I modified the settings and changed the names of my databases in the addon, and everything worked!
I noticed that the database name in the addon configuration had to be exact. I tried just xbmc_videos and it wouldn't work without adding "121" which is the version. According to the document, it should append the version number.
RE: Kodi Selective Cleaner addon support - jbinkley60 - 2024-01-26
(2024-01-25, 19:31)PnoT Wrote: looked again at the logs, and it was complaining about not being able to parse the port. further inspection of my advancedsettings.xml showed that for the videos database, there was no <port> entry. I modified the settings and changed the names of my databases in the addon, and everything worked!
I noticed that the database name in the addon configuration had to be exact. I tried just xbmc_videos and it wouldn't work without adding "121" which is the version. According to the document, it should append the version number.
Thanks for the update. I will work on improving the XML parsing code for the next release.
Jeff
RE: Kodi Selective Cleaner addon support - rrrevin - 2024-01-31
(2023-12-17, 12:57)jbinkley60 Wrote: Added single click clean all tables option
Where is this option? I can't find it. Running latest main.zip from github.
RE: Kodi Selective Cleaner addon support - jbinkley60 - 2024-01-31
(2024-01-31, 02:42)rrrevin Wrote: Where is this option? I can't find it. Running latest main.zip from github.
Thanks for making me aware of this. I had posted v1.0.4 in the releases section but had not updated the main branch. I have now. Go ahead and download v1.0.4 and you will see the single click option.
Thanks,
Jeff
RE: Kodi Selective Cleaner addon support - jbinkley60 - 2024-02-05
I am back from vacation now and will beginning working on a test release of Kodi Selective Cleaner which will become version 1.0.5. I've been collecting a list of bug fixes and enhancements. Please review this list and let me know if there is anything else you'd like me to consider for this release. I hope to have an initial version posted later this week or early next for testing:
- Bug fix to handle beginning space characters in advanced settings XML parsing
- Improve overall XML parsing and improved error handling
- Add uniqueid table analysis and cleaning
- Improve support for Kodi profiles
- Improve user information for MySQL backups not being supported
- Wiki updates
Kodi is adding two new tables for movie versions. I am not planning to add these right away due to the amount of changes that continue to occur for this new feature. I am eyeing the Kodi 21 release and what the official / final video database number will be.
Thanks,
Jeff