Help with Movie Scraping
#1
Really hoping someone can help me in reference to kodi Library.

Try to stick with me here, as I will mumble on A LOT!. I use a FireCube 3, changing this is not an option really, as I am fully invested in the amazon eco system.

So I use TMDBhelper to import my Trakt Lists, over time this has created a library of around 5500 movies. I use Movie python scraper to get all the meta. With this going on my video131.db, is currently around 95mb, I know in Kodi terms this is relatively small, but for some reason, after extensive testing once I get over 90mb, it starts to crash etc.

Now, im not looking for help with logs.

My question is as follows, is there a way to reduce this? My thinking is, and correct me if I am wrong, the reason this grows so significantly is due to the fact that, for example when scrapping a movie, lets say Batman….. it will scrape, 15 posters, 15 fanart, 10 clearlogo etc etc.

My questions is as follows, where I have spent a good 9 months, changing some Posters and Fanart to my liking. I would like to keep it how it is.

So is there anyway to export just the 1 poster, 1 fanart, 1 clearlogo and 1 landscape per movies, and not the additional 100 images?

Or would this even make a difference?

I have tried Kodi Export and Import, which does seem to show just the 1 of each imagine, but when I go into artwork settings it shows all images.

 

I really need to get the video131.db down. So please help if you can 😊
Reply
#2
(2024-11-07, 10:21)Adamosborne83 Wrote: So is there anyway to export just the 1 poster, 1 fanart, 1 clearlogo and 1 landscape per movies, and not the additional 100 images?
You won't get 100 images. You will always get 1. Kodi does download numerous links, but that is all they are-URL links. Only the first image is downloaded for each type, which is usually the highest rated image on the TMDB site.
If you install Artwork Dump, rather than the Kodi export, then it will download the original artwork, not the modified artwork from the Kodi cache.
https://forum.kodi.tv/showthread.php?tid=360078

If you are comfortable editing SQL databases, you can delete the image links from the database, but not sure how much of a difference that will make. You can always make a backup of the database and try, then run a clean library in Kodi to have Kodi compress the database after the cleanout.

As for the crashing, no idea without logs. Not familiar with the FireCube3, but guessing it is an older, and low powered bit of hardware?
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#3
Hi Karellen,
I was hoping you'd reply, as you do seem to know what you're talking about 😀

Firstly, hurt, so hurt that you don't know FireCube V3, from Android perspective one of the better bits of kit, beats Shield Pro... but that's an argument for another day.

I am afraid my SQL knowledge is somewhere between 0 and 0, don't get me wrong, probably some people I can sweet talk, but I was hoping for an easy fix as such, but know that's impossible in kodi world haha....

In refernce to a log, here is a part of one

/Sonic the Hedgehog (2020)/Sonic.the.Hedgehog.2020.strm

2024-11-06 19:42:52.281 T:26311    info <general>: Creating InputStream

2024-11-06 19:42:52.705 T:26311    info <general>: Creating Demuxer

2024-11-06 19:42:53.453 T:26311    info <general>: Opening stream: 0 source: 256

2024-11-06 19:42:53.454 T:26311    info <general>: Creating video codec with codec id: 173

2024-11-06 19:42:53.454 T:26311    info <general>: CBitstreamConverter::Open bitstream to annexb init

2024-11-06 19:42:53.455 T:26311    info <general>: CDVDVideoCodecAndroidMediaCodec::Open Testing codec: OMX.amlogic.audio.decoder.ac3

2024-11-06 19:42:53.486 T:26311    info <general>: CDVDVideoCodecAndroidMediaCodec::Open Testing codec: OMX.amlogic.audio.decoder.ac4

2024-11-06 19:42:53.486 T:26311    info <general>: CDVDVideoCodecAndroidMediaCodec::Open Testing codec: OMX.amlogic.audio.decoder.eac3

2024-11-06 19:42:53.486 T:26311    info <general>: CDVDVideoCodecAndroidMediaCodec::Open Testing codec: OMX.amlogic.av1.decoder.awesome

2024-11-06 19:42:53.487 T:26311    info <general>: CDVDVideoCodecAndroidMediaCodec::Open Testing codec: OMX.amlogic.avc.decoder.awesome

2024-11-06 19:42:53.487 T:26311    info <general>: CDVDVideoCodecAndroidMediaCodec::Open Testing codec: OMX.amlogic.dolby-vision.dvhe.decoder

2024-11-06 19:42:53.487 T:26311    info <general>: CDVDVideoCodecAndroidMediaCodec::Open Testing codec: OMX.amlogic.dolby-vision.dvav.decoder

2024-11-06 19:42:53.487 T:26311    info <general>: CDVDVideoCodecAndroidMediaCodec::Open Testing codec: OMX.amlogic.dolby-vision.dav1.decoder

2024-11-06 19:42:53.487 T:26311    info <general>: CDVDVideoCodecAndroidMediaCodec::Open Testing codec: OMX.amlogic.hevc.decoder.awesome

2024-11-06 19:42:54.020 T:26311    info <general>: CDVDVideoCodecAndroidMediaCodec::Open Using codec: OMX.amlogic.hevc.decoder.awesome

2024-11-06 19:42:54.020 T:26311    info <general>: Creating video thread

2024-11-06 19:42:54.021 T:26328    info <general>: running thread: video_thread

2024-11-06 19:42:54.073 T:26311    info <general>: Opening stream: 2 source: 256

2024-11-06 19:42:54.083 T:26311    info <general>: Finding audio codec for: 86060

2024-11-06 19:42:54.088 T:26311    info <general>: CDVDAudioCodecAndroidMediaCodec: Unknown hints.codec(86060)

2024-11-06 19:42:54.088 T:26311    info <general>: CDVDAudioCodecFFmpeg::Open() Successful opened audio decoder truehd

2024-11-06 19:42:54.088 T:26311    info <general>: OpenStream: Allowing max Out-Of-Sync Value of 50 ms

2024-11-06 19:42:54.088 T:26311    info <general>: Creating audio thread

2024-11-06 19:42:54.122 T:26329    info <general>: running thread: CVideoPlayerAudio:Tonguerocess()

2024-11-06 19:42:54.127 T:26311    info <general>: Opening stream: 16 source: 256

2024-11-06 19:42:54.131 T:26311    info <general>: CDVDSubtitlesLibass: Using libass version 1701000

2024-11-06 19:42:54.131 T:26311    info <general>: CDVDSubtitlesLibass: Creating ASS library structure

2024-11-06 19:42:54.131 T:26311    info <general>: CDVDSubtitlesLibass: Initializing ASS Renderer

2024-11-06 19:42:54.139 T:26311    info <general>: CDVDSubtitlesLibass: Initializing ASS library font settings

2024-11-06 19:42:54.204 T:26311    info <general>: CDVDSubtitlesLibass: Creating new ASS track

2024-11-06 19:42:54.211 T:26329    info <general>: Creating audio stream (codec id: 86060, channels: 8, sample rate: 48000, no pass-through)

2024-11-06 19:42:54.296 T:23526    info <general>: CActiveAESink::OpenSink - initialize sink

2024-11-06 19:42:54.912 T:23526    info <general>: Trying to open: samplerate: 48000, channelMask: 12, encoding: 4

2024-11-06 19:42:54.913 T:23526    info <general>: Minimum size we need for stream: 86128 Bytes

2024-11-06 19:42:54.913 T:23526    info <general>: Audiotrack buffer params are: period time = 56.073 ms, period size = 21532 bytes, num periods = 4

2024-11-06 19:42:54.913 T:23526    info <general>: Created Audiotrackbuffer with playing time of 224.291667 ms min buffer size: 86128 bytes

2024-11-06 19:42:55.070 T:23461    info <general>: Instancing CRendererMediaCodecSurface

2024-11-06 19:42:55.070 T:23461    info <general>: CRendererMediaCodecSurface::Configure

2024-11-06 19:42:55.820 T:26329   error <general>: CDVDAudio::AddPacketsRenderer - timeout adding data to renderer
Reply
#4
Yea, I'm a bit of a Windows snob. It's all I really know. Never really got into the Linux and Android thing.

I'd need the whole log that captures the crashing with debug mode enabled, otherwise it is useless.
That snippet does not show any problems.
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#5
Thats ok Karellen, I won't hold the 'windows' thing against you.

Now I had highlight this to our friend Jurial, and he mentioned....The only way to have any sense of the cause of a crash is with a stacktrace with debugging symbols enabled. Otherwise it is just speculation.

But trust me this is a virtual no go on a firecube.

So here is a link for my log, I hope it's 'allowed' i have edited it slightly.

If not just let me know.

https://www.dropbox.com/scl/fi/alf5u5khn...bdkxh&dl=0
Reply
#6
No good. Debug mode is not enabled, and most of the log is missing.
Don't edit anything. Enable debug mode, capture the crash and upload it to Kodi Paste Site and provide the link.
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#7
Thanks again, that was actually the entire log from start to crash.

When I'm home later, I will enable debug, and post back.

Thanks again for the help and support.

PS, was just hoping to make video.db smaller, I will research SQL too, just incase.
Reply
#8
You will need this... https://kodi.wiki/view/Databases/MyVideos
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#9
Hey Karellen,

Hope you're well and still loving Windows...

I got a couple more logs and hope it's what you're after, knowing me, it won't be.

Also, I tried the cleaners and sql and managed to save less than a whole 1mb...

https://www.dropbox.com/scl/fi/34ihms2pb...sibvd&dl=0


https://www.dropbox.com/scl/fi/w8spne0vx...lkybv&dl=0
Reply

Logout Mark Read Team Forum Stats Members Help
Help with Movie Scraping0