2022-06-27, 08:15
Turning off spectrum makes no difference
2022-06-27, 08:20
Turning off spectrum makes no difference ... beside this Leia had worked flawless with spectrum ... after the player for radio stations had switched to the Video player
2022-06-28, 10:36
Just installed the kodi on one of my workstations which is an OpenSUSE Tumbleweed
Quote:rpmqpack rpmquerythen run `kodi https://maggie.torontocast.com:8076/flac` and ... it is eating memory as I see with `top -d 20 | tee -a /tmp/kodi.log`
/suse/werner> rpmqpack | grep kodi
kodi.binary-addons-audiodecoder.qsf
kodi.binary-addons-audiodecoder.modplug
kodi.binary-addons-audiodecoder.snesapu
kodi.binary-addons-audioencoder.wav
kodi.binary-addons
kodi.binary-addons-audiodecoder.2sf
kodi.binary-addons-inputstream.adaptive
kodi.binary-addons-audiodecoder.organya
kodi.binary-addons-debuginfo
kodi.binary-addons-audiodecoder.sacd
kodi.binary-addons-audiodecoder.hvl
kodi.binary-addons-audiodecoder.stsound
kodi.binary-addons-audiodecoder.upse
kodi.binary-addons-audiodecoder.wsr
kodi
kodi.binary-addons-audiodecoder.usf
kodi.binary-addons-audiodecoder.asap
kodi.binary-addons-audioencoder.flac
kodi.binary-addons-audiodecoder.vgmstream
Quote:12581 werner 20 0 3506332 242684 121880 S 25.00 3.000 0:49.85 /usr/lib64/kodi/kodi.bin https://maggie.torontocast.com:8076/flac
12581 werner 20 0 3506332 244732 121880 S 21.72 3.026 0:54.20 /usr/lib64/kodi/kodi.bin https://maggie.torontocast.com:8076/flac
12581 werner 20 0 3506332 246780 121880 R 21.72 3.051 0:58.55 /usr/lib64/kodi/kodi.bin https://maggie.torontocast.com:8076/flac
12581 werner 20 0 3506332 248828 121880 R 22.07 3.076 1:02.97 /usr/lib64/kodi/kodi.bin https://maggie.torontocast.com:8076/flac
12581 werner 20 0 3507848 254468 121880 S 22.60 3.146 1:07.50 /usr/lib64/kodi/kodi.bin https://maggie.torontocast.com:8076/flac
12581 werner 20 0 3507848 257768 121880 S 5.389 3.187 1:08.58 /usr/lib64/kodi/kodi.bin https://maggie.torontocast.com:8076/flac
12581 werner 20 0 3507848 260824 121880 S 3.197 3.224 1:09.22 /usr/lib64/kodi/kodi.bin https://maggie.torontocast.com:8076/flac
12581 werner 20 0 3507848 262872 121880 S 3.345 3.250 1:09.89 /usr/lib64/kodi/kodi.bin https://maggie.torontocast.com:8076/flac
12581 werner 20 0 3507848 266968 121880 S 3.347 3.300 1:10.56 /usr/lib64/kodi/kodi.bin https://maggie.torontocast.com:8076/flac
12581 werner 20 0 3507848 269016 121880 S 3.397 3.326 1:11.24 /usr/lib64/kodi/kodi.bin https://maggie.torontocast.com:8076/flac
12581 werner 20 0 3507848 271064 121880 S 3.343 3.351 1:11.91 /usr/lib64/kodi/kodi.bin https://maggie.torontocast.com:8076/flac
12581 werner 20 0 3507848 273112 121880 S 3.347 3.376 1:12.58 /usr/lib64/kodi/kodi.bin https://maggie.torontocast.com:8076/flac
12581 werner 20 0 3507848 275160 121880 S 3.194 3.402 1:13.22 /usr/lib64/kodi/kodi.bin https://maggie.torontocast.com:8076/flac
12581 werner 20 0 3507848 281280 121880 S 3.345 3.477 1:13.89 /usr/lib64/kodi/kodi.bin https://maggie.torontocast.com:8076/flac
12581 werner 20 0 3507848 283328 121880 S 3.297 3.503 1:14.55 /usr/lib64/kodi/kodi.bin https://maggie.torontocast.com:8076/flac
12581 werner 20 0 3507848 285432 121880 S 3.395 3.529 1:15.23 /usr/lib64/kodi/kodi.bin https://maggie.torontocast.com:8076/flac
2022-06-28, 14:03
The leak is there even without any visualizer and even on https://maggie.torontocast.com:8076/mp3
2022-06-28, 15:37
That is good information for your github issue, although OpenSuse is also not supported by Team Kodi. I personally don't say it a "leak" since it could also be garbage collection, but you still need to find a developer willing to look into in.
Martin
Martin
2022-06-29, 08:37
The gdb shows me that CActiveAESink::OutputSamples() in line 438 of xbmc/cores/AudioEngine/Engines/ActiveAE/ActiveAESink.cpp causes memory allocation and as the commit da04e4a218f seems to change a lot e.g. from pointer uint8_t* memory access to std::vector<uint8_t> it might be that here something is continuously increased but during usage not decreased (as you mentioned a missing garbage collection) ... just tried valgrind but that breaks due timing issues the network connection to the radio station.
2022-06-30, 08:58
(2022-06-28, 15:37)emveepee Wrote: That is good information for your github issue, although OpenSuse is also not supported by Team Kodi. I personally don't say it a "leak" since it could also be garbage collection, but you still need to find a developer willing to look into in.
Martin
Btw: Which Linux based OS *is* supported by the Team Kodi .... and can anyone with such an OS simply try to run
and compare the memory consumption over the time? As I've seen yesterday if I'm using an external player (ffplay 5.0.1) the memory consumption of kodi + ffplay is nearly constant.shell:kodi https://maggie.torontocast.com:8076/flac
2022-06-30, 09:51
(2022-06-30, 08:58)bitstreamout Wrote: Btw: Which Linux based OS *is* supported by the Team Kodi
Kodi has an Ubuntu PPA for x86-64 machines, tested on the original Ubuntu release, but *should* also work on all Ubuntu flavors (although Mint regularly has its hiccups).
There is limited support for Kodi on Debian Buster/Bullseye (Rpi3/4).
Besides that, there is LibreELEC on its own plain vanilla Linux OS, as is CoreELEC.
2022-06-30, 21:40
I only use Kodi Nexus on Ubuntu and Nexus but things seem to work as expected https://pastebin.com/Bk3B8kit memory grows and then shrink over time.
Martin
Martin
2022-07-01, 17:06
I've tried a lot of debugging ... here I see growing during playing the JukeBox Radio Station *and* and Kodi with an always active full screen (with and without spectrum) and this on both QNAP with Container Ubuntu as well as on OpenSUSE Tumbleweed. To reproduce it kodi has to stay with full screen during replay.
2022-07-01, 17:29
For me it looks like PeripBusCEC thread (which uses libces) is very busy and does very often an malloc but less free calls for its scans. At least this what top in thread mode and an attached gdb do show.
2022-07-02, 12:10
Here a shot of a runnig top in the Ubuntu Container where I've currently running a kodi 19.4 with libmemusage.so preloaded
Now a few minutes later
xml:
28491 root 20 0 3328972 268156 79396 S 5.1 13.9 0:26.10 kodi.bin
28565 root 21 1 3328972 268156 79396 S 0.0 13.9 0:00.00 Announce
28566 root 21 1 3328972 268156 7=xml9396 S 0.0 13.9 0:00.00 Lirc
28567 root 20 0 3328972 268156 79396 S 0.0 13.9 0:00.00 kodi.bin
28568 root 20 0 3328972 268156 79396 S 0.0 13.9 0:01.62 ActiveAE
28569 root 19 -1 3328972 268156 79396 S 0.0 13.9 0:00.51 AESink
28570 root 20 0 3328972 268156 79396 S 0.0 13.9 0:00.00 FDEventMonitor
28575 root 39 19 3328972 268156 79396 S 0.0 13.9 0:00.00 kodi.bi:disk$0
28617 root 20 0 3328972 268156 79396 S 0.0 13.9 0:00.01 kodi.bin
28618 root 21 1 3328972 268156 79396 S 0.3 13.9 0:01.20 PeripBusUSBUdev
28619 root 21 1 3328972 268156 79396 S 21.5 13.9 0:50.40 PeripBusCEC
28620 root 21 1 3328972 268156 79396 S 0.0 13.9 0:00.00 PeripBusAddon
28621 root 20 0 3328972 268156 79396 S 0.0 13.9 0:00.80 PeripEventScan
28622 root 20 0 3328972 268156 79396 S 0.0 13.9 0:00.00 Timer
28624 root 20 0 3328972 268156 79396 S 0.3 13.9 0:01.09 LanguageInvoker
28625 root 20 0 3328972 268156 79396 S 0.0 13.9 0:00.29 LanguageInvoker
28628 root 20 0 3328972 268156 79396 S 0.0 13.9 0:00.50 LanguageInvoker
28631 root 20 0 3328972 268156 79396 S 0.0 13.9 0:00.00 kodi.bin
28633 root 20 0 3328972 268156 79396 S 0.0 13.9 0:00.02 kodi.bin
28634 root 20 0 3328972 268156 79396 S 0.0 13.9 0:00.00 kodi.bin
28635 root 20 0 3328972 268156 79396 S 0.0 13.9 0:00.00 kodi.bin
28637 root 20 0 3328972 268156 79396 S 0.0 13.9 0:00.03 kodi.bin
28638 root 20 0 3328972 268156 79396 S 0.0 13.9 0:00.01 EventServer
28642 root 20 0 3328972 268156 79396 S 0.0 13.9 0:00.01 TCPServer
28647 root 20 0 3328972 268156 79396 S 0.0 13.9 0:00.00 MHD-listen
28648 root 20 0 3328972 268156 79396 S 0.0 13.9 0:00.00 MHD-listen
28652 root 21 1 3328972 268156 79396 S 0.0 13.9 0:00.00 JobWorker
28653 root 21 1 3328972 268156 79396 S 0.0 13.9 0:00.75 PVRManager
28658 root 20 0 3328972 268156 79396 S 0.0 13.9 0:00.00 kodi.bin
28697 root 21 1 3328972 268156 79396 S 0.3 13.9 0:00.10 PVRGUIInfo
28698 root 21 1 3328972 268156 79396 S 0.0 13.9 0:02.24 EPGUpdater
28699 root 20 0 3328972 268156 79396 S 0.0 13.9 0:00.00 PVRTimers
28720 root 20 0 3328972 268156 79396 S 20.3 13.9 0:40.08 PAPlayer
28787 root 20 0 3328972 268156 79396 S 0.0 13.9 0:00.00 ShoutcastFile
28788 root 20 0 3328972 268156 79396 S 0.3 13.9 0:02.10 FileCache
Now a few minutes later
xml:
28491 root 20 0 3354048 312816 73864 S 5.4 16.2 0:37.75 kodi.bin
28565 root 21 1 3354048 312816 73864 S 0.0 16.2 0:00.00 Announce
28566 root 21 1 3354048 312816 73864 S 0.0 16.2 0:00.00 Lirc
28567 root 20 0 3354048 312816 73864 S 0.0 16.2 0:00.00 kodi.bin
28568 root 20 0 3354048 312816 73864 S 0.6 16.2 0:02.36 ActiveAE
28569 root 19 -1 3354048 312816 73864 S 0.0 16.2 0:00.76 AESink
28570 root 20 0 3354048 312816 73864 S 0.0 16.2 0:00.00 FDEventMonitor
28575 root 39 19 3354048 312816 73864 S 0.0 16.2 0:00.00 kodi.bi:disk$0
28617 root 20 0 3354048 312816 73864 S 0.0 16.2 0:00.03 kodi.bin
28618 root 21 1 3354048 312816 73864 S 0.0 16.2 0:01.24 PeripBusUSBUdev
28619 root 21 1 3354048 312816 73864 S 21.5 16.2 1:16.90 PeripBusCEC
28620 root 21 1 3354048 312816 73864 S 0.0 16.2 0:00.00 PeripBusAddon
28621 root 20 0 3354048 312816 73864 S 0.0 16.2 0:01.23 PeripEventScan
28622 root 20 0 3354048 312816 73864 S 0.0 16.2 0:00.00 Timer
28624 root 20 0 3354048 312816 73864 S 0.0 16.2 0:01.16 LanguageInvoker
28625 root 20 0 3354048 312816 73864 S 0.0 16.2 0:00.36 LanguageInvoker
28628 root 20 0 3354048 312816 73864 S 0.0 16.2 0:00.57 LanguageInvoker
28631 root 20 0 3354048 312816 73864 S 0.0 16.2 0:00.00 kodi.bin
28633 root 20 0 3354048 312816 73864 S 0.0 16.2 0:00.02 kodi.bin
28634 root 20 0 3354048 312816 73864 S 0.0 16.2 0:00.00 kodi.bin
28635 root 20 0 3354048 312816 73864 S 0.0 16.2 0:00.00 kodi.bin
28637 root 20 0 3354048 312816 73864 S 0.0 16.2 0:00.04 kodi.bin
28638 root 20 0 3354048 312816 73864 S 0.0 16.2 0:00.01 EventServer
28642 root 20 0 3354048 312816 73864 S 0.0 16.2 0:00.01 TCPServer
28647 root 20 0 3354048 312816 73864 S 0.0 16.2 0:00.00 MHD-listen
28648 root 20 0 3354048 312816 73864 S 0.0 16.2 0:00.00 MHD-listen
28652 root 21 1 3354048 312816 73864 S 0.0 16.2 0:00.00 JobWorker
28653 root 21 1 3354048 312816 73864 S 0.0 16.2 0:00.80 PVRManager
28658 root 20 0 3354048 312816 73864 S 0.0 16.2 0:00.00 kodi.bin
28697 root 21 1 3354048 312816 73864 S 0.0 16.2 0:00.15 PVRGUIInfo
28698 root 21 1 3354048 312816 73864 S 0.0 16.2 0:02.49 EPGUpdater
28699 root 20 0 3354048 312816 73864 S 0.0 16.2 0:00.00 PVRTimers
28720 root 20 0 3354048 312816 73864 S 27.5 16.2 1:31.35 PAPlayer
28787 root 20 0 3354048 312816 73864 S 0.0 16.2 0:00.02 ShoutcastFile
28788 root 20 0 3354048 312816 73864 S 0.6 16.2 0:03.26 FileCache
3624 root 21 1 3354048 312816 73864 S 0.0 16.2 0:00.00 JobWorker
2022-07-02, 15:11
For the CEC CPU load: https://github.com/Pulse-Eight/libcec/pull/495 - it's polling for new CEC devices.
For the memory-leak: I remember (it's very long ago), that there was an issue in Leia with libtag, e.g. reading the id3 tags from the streams, which is why it was disabled in Leia. But I can't remember anymore.
As you seem to have quite some good debug skills, maybe valgrind (clicking away all the python stuff) may help?
PS: Sorry that no one replied to your bugreports on github. There is a huge amount of kodi not being maintained anymore ... the reports were perfectly fine ... I sadly could not add any valuable information to it besides the above speculation.
For the memory-leak: I remember (it's very long ago), that there was an issue in Leia with libtag, e.g. reading the id3 tags from the streams, which is why it was disabled in Leia. But I can't remember anymore.
As you seem to have quite some good debug skills, maybe valgrind (clicking away all the python stuff) may help?
PS: Sorry that no one replied to your bugreports on github. There is a huge amount of kodi not being maintained anymore ... the reports were perfectly fine ... I sadly could not add any valuable information to it besides the above speculation.
2022-07-02, 23:03
Thanks for your responce ... meanwhile I have reduces the buffer memory (now only 10MB) ... but after an 45 minutes I see in top
Quote:28348 root 20 0 3697560 627900 8124 S 5.0 32.6 1:58.36 kodi.binand hear stutter ... pressing x and p to stop and restart the radio channel drops this down to 6.1% for each thread ... this morning I have checked the same process or better the CEC process as it shows the same memory
28432 root 21 1 3697560 627900 8124 S 0.0 32.6 0:00.00 Announce
28433 root 21 1 3697560 628164 8124 S 0.0 32.6 0:00.00 Lirc
28435 root 20 0 3697560 628164 8124 S 0.0 32.6 0:00.00 kodi.bin
28436 root 20 0 3697560 628428 8124 S 0.0 32.6 0:08.34 ActiveAE
28437 root 19 -1 3697560 628428 8124 S 0.0 32.6 0:03.47 AESink
28438 root 20 0 3697560 628428 8124 S 0.0 32.6 0:00.00 FDEventMonitor
28443 root 39 19 3697560 628692 8124 S 0.0 32.7 0:00.00 kodi.bi:disk$0
28451 root 20 0 3697560 628692 8124 S 0.0 32.7 0:00.11 kodi.bin
28452 root 21 1 3697560 628956 8124 S 0.0 32.7 0:01.31 PeripBusUSBUdev
28453 root 21 1 3697560 628956 8124 S 0.0 32.7 4:09.61 PeripBusCEC
28454 root 21 1 3697560 628956 8124 S 0.0 32.7 0:00.00 PeripBusAddon
28455 root 20 0 3697560 629220 8124 S 0.0 32.7 0:04.39 PeripEventScan
28456 root 20 0 3697560 629220 8124 S 0.0 32.7 0:00.00 Timer
28458 root 20 0 3697560 629220 8124 S 0.3 32.7 0:01.58 LanguageInvoker
28459 root 20 0 3697560 629484 8124 S 0.0 32.7 0:00.88 LanguageInvoker
28460 root 20 0 3697560 629484 8124 S 0.0 32.7 0:01.06 LanguageInvoker
28462 root 20 0 3697560 629748 8124 S 0.0 32.7 0:00.00 kodi.bin
28464 root 20 0 3697560 629748 8124 S 0.0 32.7 0:00.15 kodi.bin
28465 root 20 0 3697560 629748 8124 S 0.0 32.7 0:00.01 kodi.bin
28466 root 20 0 3697560 630012 8124 S 0.0 32.7 0:00.02 kodi.bin
28467 root 20 0 3697560 630012 8124 S 0.0 32.7 0:00.15 kodi.bin
28468 root 20 0 3697560 630276 8124 S 0.0 32.7 0:00.06 EventServer
28475 root 20 0 3697560 630276 8124 S 0.0 32.7 0:00.05 TCPServer
28479 root 20 0 3697560 630276 8124 S 0.0 32.7 0:00.00 MHD-listen
28481 root 20 0 3697560 630540 8124 S 0.0 32.8 0:00.00 MHD-listen
28483 root 21 1 3697560 630540 8124 S 0.0 32.8 0:00.01 JobWorker
28485 root 21 1 3697560 630540 8124 S 0.0 32.8 0:00.84 PVRManager
28488 root 20 0 3697560 630804 8124 S 0.0 32.8 0:00.00 kodi.bin
28499 root 21 1 3697560 630804 8124 S 0.0 32.8 0:00.56 PVRGUIInfo
28501 root 21 1 3697560 631068 8124 S 0.0 32.8 0:02.28 EPGUpdater
28502 root 20 0 3697560 631068 8124 S 0.0 32.8 0:00.00 PVRTimers
28512 root 20 0 3697560 631068 8124 R 99.1 32.8 19:16.91 PAPlayer
28541 root 20 0 3697560 631332 8124 S 0.0 32.8 0:00.10 ShoutcastFile
28542 root 20 0 3697560 631332 8124 S 0.6 32.8 0:11.85 FileCache
29236 root 21 1 3697560 631596 8124 S 0.0 32.8 0:00.00 JobWorker
29237 root 21 1 3697560 631596 8124 S 0.0 32.8 0:00.00 JobWorker
Quote:28453 root 21 1 3119996 134836 41700 S 19.0 7.0 44:42.65 PeripBusCEClooks like busy any maybe a small memory leak. The temporary memory blob during replay the radion station over HDMI might be caused somewhere else ... maybe valgrind massif tool might show more but I 've no idea how to get this working on/within the QNAP NAS Ubuntu Container.
2022-07-06, 08:56
Now I've an idea what goes wrong here ... it is the difference of virtual memory (even if mmap from (u)dma or memfd ioctls) and real physical memory. In handling of the virtual memory kodi does not leak, regardless if malloc/free or mmap/munmap is used. I've tested this with preloading libmemusage.so on my NAS (as the shell script memusage does not exists) and exporting MEMUSAGE_OUTPUT and MEMUSAGE_TRACE_MMAP and then generating an png on my person laptop with memusagestat script. But also with the KDE tool heaptrack and heaptrack_gui on my Tumblweed workstation.
Indeed kodi 19 is mapping a huge amount of virtual memory (regardless if file or memory caching is used due mmap) but this blob does not increase the used amount of physical memory pages seen in top at %MEM. During accessing the ring buffer (called circular cache in kodi source pointed by heaptrack_gui) those virtual pages get hot which means for the kernel it is a used physical memory pages ... now the difference between Leia and Matrix seems that virtual pages do not get cold anymore even if the data therein had been send as Audio to the AVR receiver (or here on my Tumbleweed workstation to the USB headset).
Depending on the amount of real memory (on my NAS this is 2GB, on my Tumbleweed 8GB, on my big workstation 32GB) and the swappiness factor (in percent of total memory, as given in /proc/sys/vm/swappiness) the system will swap out parts of a program which crosses this limit and this leads to stuttering audio. And kodi Matrix *does* cross this limit on the NAS.
Indeed kodi 19 is mapping a huge amount of virtual memory (regardless if file or memory caching is used due mmap) but this blob does not increase the used amount of physical memory pages seen in top at %MEM. During accessing the ring buffer (called circular cache in kodi source pointed by heaptrack_gui) those virtual pages get hot which means for the kernel it is a used physical memory pages ... now the difference between Leia and Matrix seems that virtual pages do not get cold anymore even if the data therein had been send as Audio to the AVR receiver (or here on my Tumbleweed workstation to the USB headset).
Depending on the amount of real memory (on my NAS this is 2GB, on my Tumbleweed 8GB, on my big workstation 32GB) and the swappiness factor (in percent of total memory, as given in /proc/sys/vm/swappiness) the system will swap out parts of a program which crosses this limit and this leads to stuttering audio. And kodi Matrix *does* cross this limit on the NAS.
