Kodi Community Forum
Bug No cache/buffer is used for m3u8 "playlists", causes stuttering video - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: General Support (https://forum.kodi.tv/forumdisplay.php?fid=111)
+---- Forum: OS independent / Other (https://forum.kodi.tv/forumdisplay.php?fid=228)
+---- Thread: Bug No cache/buffer is used for m3u8 "playlists", causes stuttering video (/showthread.php?tid=154016)

Pages: 1 2 3 4 5

No cache/buffer is used for m3u8 "playlists", causes stuttering video - dataolle - 2013-01-31

When I try to stream television from for example svtplay or other websites where they use m3u8 "playlists" files (HLS streaming), xbmc does not use the cache/buffer (the video information shows 0B 100%). This causes the video playback on my Raspberry PI to stutter badly when playing back video.
If I play single segements within the m3u8 file cache is used (5meg in my case) and playback is perfectly smooth.
Also I have downloaded/reassembled some of the m3u8 files to my webserver (using ffmpeg) and streamed from there, perfectly smooth playback, cache is used.

I also opened a issues on the svtplay plugin github page on this, but since it seems to be a issue with how xbmc handles m3u8 streams I post here instead.

For reference: (svtplay plugin: https://github.com/nilzen/xbmc-svtplay/issues/58 and nrk plugin https://github.com/takoi/xbmc-addon-nrk/issues/8 )
Is there a setting/patch to make xbmc use the cache when playing m3u8 streams?

I use xbmc 12 rc3 on raspbmc.


RE: No cache/buffer is used for m3u8 p"playlists", causes stuttering video - popcornmix - 2013-01-31

The file is cached if:
Code:
if ( (flags & READ_NO_CACHE) == 0 && URIUtils::IsInternetStream(url, true) && !CUtil::IsPicture(strFileName) )
      m_flags |= READ_CACHED;
https://github.com/xbmc/xbmc/blob/master/xbmc/filesystem/File.cpp#L230
IsInternetStream is basically:
Code:
if (strProtocol2 == "http"  || strProtocol2 == "https"  ||
      strProtocol2 == "tcp"   || strProtocol2 == "udp"    ||
      strProtocol2 == "rtp"   || strProtocol2 == "sdp"    ||
      strProtocol2 == "mms"   || strProtocol2 == "mmst"   ||
      strProtocol2 == "mmsh"  || strProtocol2 == "rtsp"   ||
      strProtocol2 == "rtmp"  || strProtocol2 == "rtmpt"  ||
      strProtocol2 == "rtmpe" || strProtocol2 == "rtmpte" ||
      strProtocol2 == "rtmps")

So, what does the m3u8 url look like? Does the addon request READ_NO_CACHE?

RE: No cache/buffer is used for m3u8 "playlists", causes stuttering video - dataolle - 2013-01-31

Example URL:
Code:
"http://svtplay1l-f.akamaihd.net/i/world/open/20130129/1322336-003A/MIN_SANNING_VAR-003A-6ea900962dcad7f4_,900,348,564,1680,2800,.mp4.csmil/index_4_av.m3u8?null=&e=b2a7ace1bf661c32&id="

Here is the contents:
Code:
pi@raspbmc:~$ curl -s "http://svtplay1l-f.akamaihd.net/i/world/open/20130129/1322336-003A/MIN_SANNING_VAR-003A-6ea900962dcad7f4_,900,348,564,1680,2800,.mp4.csmil/index_4_av.m3u8?null=&e=b2a7ace1bf661c32&id=" | head -20
#EXTM3U
#EXT-X-TARGETDURATION:10
#EXT-X-ALLOW-CACHE:YES
#EXT-X-VERSION:2
#EXT-X-MEDIA-SEQUENCE:1
#EXTINF:10,
http://svtplay1l-f.akamaihd.net/i/world/open/20130129/1322336-003A/MIN_SANNING_VAR-003A-6ea900962dcad7f4_,900,348,564,1680,2800,.mp4.csmil/segment1_4_av.ts?null=&e=b2a7ace1bf661c32&id=
#EXTINF:10,
http://svtplay1l-f.akamaihd.net/i/world/open/20130129/1322336-003A/MIN_SANNING_VAR-003A-6ea900962dcad7f4_,900,348,564,1680,2800,.mp4.csmil/segment2_4_av.ts?null=&e=b2a7ace1bf661c32&id=
#EXTINF:10,
http://svtplay1l-f.akamaihd.net/i/world/open/20130129/1322336-003A/MIN_SANNING_VAR-003A-6ea900962dcad7f4_,900,348,564,1680,2800,.mp4.csmil/segment3_4_av.ts?null=&e=b2a7ace1bf661c32&id=
etc etc


I do not belive the problem is with the plugin setting READ_NO_CACHE because i get the same problem when i send the above playlist to xbmc for playback using the json-rpc API, and not use the plugin.

The TS files listed above plays perfectly when i send them one by one using the json-rpc api. also, then the cache is used.

No cache/buffer is used for m3u8 "playlists", causes stuttering video - Memphiz - 2013-01-31

@popcornmix - m3u8 are not fetched via cfile - ffmpeg handles it for us - so the cachng is either missing in ffmpeg or it has to be setup somehow (iirc its dvdinputstreamffmpeg or so which handles m3u8)

RE: No cache/buffer is used for m3u8 "playlists", causes stuttering video - popcornmix - 2013-01-31

(2013-01-31, 01:36)Memphiz Wrote: @popcornmix - m3u8 are not fetched via cfile - ffmpeg handles it for us - so the cachng is either missing in ffmpeg or it has to be setup somehow (iirc its dvdinputstreamffmpeg or so which handles m3u8)

Yes, I discovered CFile::Open is not called for m3u8 files (but is for other http files). However I got lost in the maze of overloading for all the stream/demux handlers and didn't find where the file read occurs.

Pretty sure this is a general xbmc issue, rather than a Pi specific issue.

RE: No cache/buffer is used for m3u8 "playlists", causes stuttering video - dataolle - 2013-01-31

Confirmed the same behavior on a xbmc on a windows box. But the raspberry pi seems to suffer more when there is no cache, the windows box seem to be able to playback the video smoothly without the cache.
Should probably be fixed on all platforms.

RE: No cache/buffer is used for m3u8 "playlists", causes stuttering video - popcornmix - 2013-01-31

(2013-01-31, 09:02)dataolle Wrote: Confirmed the same behavior on a xbmc on a windows box. But the raspberry pi seems to suffer more when there is no cache, the windows box seem to be able to playback the video smoothly without the cache.
Should probably be fixed on all platforms.

Might be worth raising a trac ticket or posting in a non-Pi specific forum.

RE: No cache/buffer is used for m3u8 "playlists", causes stuttering video - flyingrat - 2013-01-31

(2013-01-31, 09:02)dataolle Wrote: Confirmed the same behavior on a xbmc on a windows box. But the raspberry pi seems to suffer more when there is no cache, the windows box seem to be able to playback the video smoothly without the cache.
Should probably be fixed on all platforms.

Hi folks, just got the flu which mean I have some spare time to dig into this issue ;-) I can confirm the same problem exists on both OSX and iOS (Macs and IPads) as well.

Have a look at this video stream that works fine with a browser: "www.svtplay.se" but is staggering with XBMC (SVT Play addon) on OSX, iOS and RPi devices: "SVT.Play.Screen.Recording.mp4". This video sample is recorded with "Quicktime Screen Recorder" on a Mac. I couldn't find a simular tool for iPad but it's the same behavior.

Some stream info: M3U8 info:

Code:
#EXTM3U
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=985000,RESOLUTION=768x432,CODECS="avc1.77.30, mp4a.40.5"
http://svtplay10l-f.akamaihd.net/i/se/open/20130127/1331352-003A/TEST_YOUR_BRAIN-003A-1989ce6a99f2fce2_,900,348,564,1680,2800,.mp4.csmil/index_0_av.m3u8?null=&e=b2a7ace1bf661c32&id=
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=346000,RESOLUTION=512x288,CODECS="avc1.66.30, mp4a.40.5"
http://svtplay10l-f.akamaihd.net/i/se/open/20130127/1331352-003A/TEST_YOUR_BRAIN-003A-1989ce6a99f2fce2_,900,348,564,1680,2800,.mp4.csmil/index_1_av.m3u8?null=&e=b2a7ace1bf661c32&id=
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=562000,RESOLUTION=512x288,CODECS="avc1.66.30, mp4a.40.5"
http://svtplay10l-f.akamaihd.net/i/se/open/20130127/1331352-003A/TEST_YOUR_BRAIN-003A-1989ce6a99f2fce2_,900,348,564,1680,2800,.mp4.csmil/index_2_av.m3u8?null=&e=b2a7ace1bf661c32&id=
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1678000,RESOLUTION=1024x576,CODECS="avc1.77.30, mp4a.40.5"
http://svtplay10l-f.akamaihd.net/i/se/open/20130127/1331352-003A/TEST_YOUR_BRAIN-003A-1989ce6a99f2fce2_,900,348,564,1680,2800,.mp4.csmil/index_3_av.m3u8?null=&e=b2a7ace1bf661c32&id=
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2793000,RESOLUTION=1280x720,CODECS="avc1.77.30, mp4a.40.5"
http://svtplay10l-f.akamaihd.net/i/se/open/20130127/1331352-003A/TEST_YOUR_BRAIN-003A-1989ce6a99f2fce2_,900,348,564,1680,2800,.mp4.csmil/index_4_av.m3u8?null=&e=b2a7ace1bf661c32&id=

(2013-01-31, 13:00)popcornmix Wrote: Might be worth raising a trac ticket or posting in a non-Pi specific forum.

That's probably a good idea. Anyone who knows what forum is best suited to handle this type of issues?


RE: No cache/buffer is used for m3u8 "playlists", causes stuttering video - zoltan - 2013-01-31

I have the same problem streaming live from my dreambox (.m3u stream) hd channels...

RE: No cache/buffer is used for m3u8 "playlists", causes stuttering video - Ned Scott - 2013-01-31

Thread moved...

RE: No cache/buffer is used for m3u8 "playlists", causes stuttering video - flyingrat - 2013-01-31

(2013-01-31, 02:22)popcornmix Wrote:
(2013-01-31, 01:36)Memphiz Wrote: @popcornmix - m3u8 are not fetched via cfile - ffmpeg handles it for us - so the cachng is either missing in ffmpeg or it has to be setup somehow (iirc its dvdinputstreamffmpeg or so which handles m3u8)

Yes, I discovered CFile::Open is not called for m3u8 files (but is for other http files). However I got lost in the maze of overloading for all the stream/demux handlers and didn't find where the file read occurs.

Pretty sure this is a general xbmc issue, rather than a Pi specific issue.


I did a search for ".m3u8" (link) in the repro. Every method name below is linked to the repro src.

Folder "https://github.com/xbmc/xbmc/tree/master/xbmc/cores/dvdplayer/DVDInputStreams"

This is the call path I found so far:

1. CDVDPlayer::OpenInputStream() -> CDVDFactoryInputStream::CreateInputStream()
2. CDVDFactoryInputStream::CreateInputStream() -> CDVDInputStreamFFmpeg()
3. CDVDInputStreamFFmpeg::CDVDInputStreamFFmpeg() -> CDVDInputStream(DVDSTREAM_TYPE_FFMPEG)



1. DVDPlayer.cpp - Line 547 - CDVDPlayer::OpenInputStream()

Code:
bool CDVDPlayer::OpenInputStream()
{
  if(m_pInputStream)
    SAFE_DELETE(m_pInputStream);

  CLog::Log(LOGNOTICE, "Creating InputStream");

  // correct the filename if needed
  CStdString filename(m_filename);
  if (filename.Find("dvd://") == 0
  ||  filename.CompareNoCase("iso9660://video_ts/video_ts.ifo") == 0)
  {
    m_filename = g_mediaManager.TranslateDevicePath("");
  }

  // before creating the input stream, if this is an HLS playlist then get the
  // most appropriate bitrate based on our network settings
  if (filename.Left(7) == "http://" && filename.Right(5) == ".m3u8")
  {
    // get the available bandwidth (as per user settings)
    int maxrate = g_guiSettings.GetInt("network.bandwidth");
    if(maxrate <= 0)
      maxrate = INT_MAX;

    // determine the most appropriate stream
    m_filename = PLAYLIST::CPlayListM3U::GetBestBandwidthStream(m_filename, (size_t)maxrate);
  }
  m_pInputStream = CDVDFactoryInputStream::CreateInputStream(this, m_filename, m_mimetype);



2. "DVDFactoryInputStream.cpp - Line 92 - CDVDFactoryInputStream::CreateInputStream"

Code:
|| (item.IsInternetStream() && item.IsType(".m3u8")))
    return new CDVDInputStreamFFmpeg();



3. "DVDInputStreamFFmpeg.cpp - Line 41 - CDVDInputStreamFFmpeg::Open"

Code:
bool CDVDInputStreamFFmpeg::Open(const char* strFile, const std::string& content)
{
  if (!CDVDInputStream::Open(strFile, content))
    return false;

  return true;
}


Where in the call path is the cache supposed to be implemented ??
--







RE: No cache/buffer is used for m3u8 "playlists", causes stuttering video - Memphiz - 2013-01-31

Nope it has to be inside of ffmpeg somewhere. Cause ffmpeg handles the whole http connection for those streams. Someone needs to find out if ffmpeg has an av_option (or something else) for this to configure/enable caching.

RE: No cache/buffer is used for m3u8 "playlists", causes stuttering video - flyingrat - 2013-01-31

(2013-01-31, 16:13)Memphiz Wrote: Nope it has to be inside of ffmpeg somewhere. Cause ffmpeg handles the whole http connection for those streams. Someone needs to find out if ffmpeg has an av_option (or something else) for this to configure/enable caching.
Ok, doing a new dig into the ffmpeg repro...


RE: No cache/buffer is used for m3u8 "playlists", causes stuttering video - flyingrat - 2013-02-01

Hi folks, a brief status report whats currently going on...

First the bad news (sort of ;-). I'm getting better from my flu which means I'm probably getting back to work next week, thus I wont have as much time as I would like on the subject but I'll do my best.

The good news is that the ffmpeg libraries seems ok. My plan was to use a standalone player based on ffmpeg to verify the libraries. In this case I got the osx mplayer work on my mac. Mplayer was able to play the svt-play m3u8 streams without staggering. When I lowered the stream cache it occasionally complained about "Your system is too SLOW" which is a good problem indicator. Btw, anyone knows what the "SPS 32 referenced in buffering period" is meaning?

I also managed (which is another subject) to built the lastest stable release (xbmc-12.0.tar.gz) from scratch to make sure we track the current problem on the right release level. My current plan is to debug/trace the mplayer to find out how the cache is used m3u8 streams. Any other suggestions or ideas are welcome!

Trace output from playing a m3u8 stream with a low (32k) cache:

$ mplayer2 -cache 32 "http://svtplay7m-f.akamaihd.net/i/se//open/20130129/1230799-001A/DIVING_WITH_CRO-001A-23b378251ee65366_,900,348,564,1680,2800,.mp4.csmil/master.m3u8"

Code:
MPlayer2 UNKNOWN (C) 2000-2011 MPlayer Team
Can't init Apple Remote.

Playing http://svtplay7m-f.akamaihd.net/i/se//open/20130129/1230799-001A/DIVING_WITH_CRO-001A-23b378251ee65366_,900,348,564,1680,2800,.mp4.csmil/master.m3u8.
Resolving svtplay7m-f.akamaihd.net for AF_INET6...

Couldn't resolve name for AF_INET6: svtplay7m-f.akamaihd.net
Resolving svtplay7m-f.akamaihd.net for AF_INET...
Connecting to server svtplay7m-f.akamaihd.net[92.33.33.152]: 80...

Cache size set to 32 KBytes
Cache fill:  4.29% (1407 bytes)  

Detected file format: Apple HTTP Live Streaming format (libavformat)
Using network protocols without global network initialization. Please use avformat_network_init(), this will become mandatory later.
[NULL @ 0x7fdebb255800] non-existing SPS 0 referenced in buffering period
[NULL @ 0x7fdebb255800] non-existing SPS 32 referenced in buffering period
[NULL @ 0x7fdebbc95000] non-existing SPS 0 referenced in buffering period
[NULL @ 0x7fdebbc95000] non-existing SPS 32 referenced in buffering period
[NULL @ 0x7fdebcaf3000] non-existing SPS 0 referenced in buffering period
[NULL @ 0x7fdebcaf3000] non-existing SPS 32 referenced in buffering period
[NULL @ 0x7fdebcafaa00] non-existing SPS 0 referenced in buffering period
[NULL @ 0x7fdebcafaa00] non-existing SPS 32 referenced in buffering period
[NULL @ 0x7fdebcb1bc00] non-existing SPS 0 referenced in buffering period
[NULL @ 0x7fdebcb1bc00] non-existing SPS 32 referenced in buffering period
[h264 @ 0x7fdebb25a200] non-existing SPS 0 referenced in buffering period
[h264 @ 0x7fdebb25a200] non-existing SPS 32 referenced in buffering period
[h264 @ 0x7fdebcb6ac00] non-existing SPS 0 referenced in buffering period
[h264 @ 0x7fdebcb6ac00] non-existing SPS 32 referenced in buffering period
[h264 @ 0x7fdebcb45c00] non-existing SPS 0 referenced in buffering period
[h264 @ 0x7fdebcb45c00] non-existing SPS 32 referenced in buffering period
[h264 @ 0x7fdebcb58000] non-existing SPS 0 referenced in buffering period
[h264 @ 0x7fdebcb58000] non-existing SPS 32 referenced in buffering period
[h264 @ 0x7fdebcb7d800] non-existing SPS 0 referenced in buffering period
[h264 @ 0x7fdebcb7d800] non-existing SPS 32 referenced in buffering period
[applehttp @ 0x7fdebb041a00] Estimating duration from bitrate, this may be inaccurate
[lavf] stream 0: video (h264), -vid 0
[lavf] stream 1: audio (aac), -aid 0
[lavf] stream 2: video (h264), -vid 1
[lavf] stream 3: audio (aac), -aid 1
[lavf] stream 4: video (h264), -vid 2
[lavf] stream 5: audio (aac), -aid 2
[lavf] stream 6: video (h264), -vid 3
[lavf] stream 7: audio (aac), -aid 3
[lavf] stream 8: video (h264), -vid 4
[lavf] stream 9: audio (aac), -aid 4
VIDEO:  [H264]  768x432  0bpp  25.000 fps    0.0 kbps ( 0.0 kbyte/s)
Opening video filter: [scale]
[ass] auto-open
==========================================================================
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
Asking decoder to use 2 threads if supported.
[h264 @ 0x7fdebd8a6a00] non-existing SPS 0 referenced in buffering period
[h264 @ 0x7fdebd8a6a00] non-existing SPS 32 referenced in buffering period
Selected video codec: [ffh264] vfm: ffmpeg (FFmpeg H.264)
==========================================================================
==========================================================================
Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders
AUDIO: 48000 Hz, 2 ch, s16le, 0.0 kbit/0.00% (ratio: 0->192000)
Selected audio codec: [ffaac] afm: ffmpeg (FFmpeg AAC (MPEG-2/MPEG-4 Audio))
==========================================================================
AO: [coreaudio] 48000Hz 2ch s16le (2 bytes per sample)
Starting playback...
A:  10.1 V:   0.0 A-V:  0.000 ct:  0.000   0/  0 ??% ??% ??,?% 0 0 0%
Movie-Aspect is 1.78:1 - prescaling to correct movie aspect.
[swscaler @ 0x7fdebd0fa800] BICUBIC scaler, from yuv420p to yuyv422 using MMX2
VO: [corevideo] 768x432 => 768x432 Packed YUY2
A:  10.2 V:  10.2 A-V: -0.040 ct:  0.000   0/  0 ??% ??% ??,?% 0 0 0%
Decreasing video pts: 10.181000 < 10.221000
A:  10.3 V:  10.3 A-V:  0.001 ct:  0.000   0/  0 ??% ??% ??,?% 0 0 0%
Decreasing video pts: 10.141000 < 10.261000
A:  10.4 V:  10.4 A-V: -0.003 ct:  0.000   0/  0 ??% ??% ??,?% 0 0 0%
Decreasing video pts: 10.341000 < 10.381000
Using network protocols without global network initialization. Please use avformat_network_init(), this will become mandatory later.
Using network protocols without global network initialization. Please use avformat_network_init(), this will become mandatory later.
[applehttp @ 0x7fdebb041a00] No longer receiving variant 1
[applehttp @ 0x7fdebb041a00] No longer receiving variant 2
[applehttp @ 0x7fdebb041a00] No longer receiving variant 3
[applehttp @ 0x7fdebb041a00] No longer receiving variant 4
Using network protocols without global network initialization. Please use avformat_network_init(), this will become mandatory later.
Using network protocols without global network initialization. Please use avformat_network_init(), this will become mandatory later.
Using network protocols without global network initialization. Please use avformat_network_init(), this will become mandatory later.
Using network protocols without global network initialization. Please use avformat_network_init(), this will become mandatory later.
A:  47.7 V:  47.7 A-V: -0.000 ct:  0.001   0/  0  3%  8%  3.9% 59 0 0%


           ************************************************
           **** Your system is too SLOW to play this!  ****
           ************************************************

Possible reasons, problems, workarounds:
- Most common: broken/buggy _audio_ driver
  - Try -ao sdl or use the OSS emulation of ALSA.
  - Experiment with different values for -autosync, 30 is a good start.
- Slow video output
  - Try a different -vo driver (-vo help for a list) or try -framedrop!
- Slow CPU
  - Don't try to play a big DVD/DivX on a slow CPU! Try some of the lavdopts,
    e.g. -vfm ffmpeg -lavdopts lowres=1:fast:skiploopfilter=all.
- Broken file
  - Try various combinations of -nobps -ni -forceidx -mc 0.
- Slow media (NFS/SMB mounts, DVD, VCD etc)
  - Try -cache 8192.
- Are you using -cache to play a non-interleaved AVI file?
  - Try -nocache.
Read DOCS/HTML/en/video.html for tuning/speedup tips.
If none of this helps you, read DOCS/HTML/en/bugreports.html.

Using network protocols without global network initialization. Please use avformat_network_init(), this will become mandatory later.

A: 181.3 V: 181.3 A-V: -0.002 ct:  0.001   0/  0  1%  7%  5.1% 294 0 0%
A: 182.1 V: 182.1 A-V:  0.000 ct:  0.001   0/  0  1%  7%  5.1% 294 0 0%
A: 182.8 V: 182.7 A-V:  0.137 ct:  0.001   0/  0  1%  7%  5.1% 294 0 0%
A: 184.5 V: 184.5 A-V: -0.003 ct:  0.001   0/  0  1%  7%  5.1% 295 0 0%
A: 185.3 V: 185.3 A-V:  0.010 ct:  0.001   0/  0  1%  7%  5.0% 295 0 0%
A: 185.8 V: 185.8 A-V:  0.000 ct:  0.001   0/  0  1%  7%  5.0% 295 0 0%

MPlayer interrupted by signal 2 in module: decode_audio
MPlayer interrupted by signal 2 in module: enable_cache
A: 190.0 V: 189.2 A-V:  0.832 ct:  0.001   0/  0  1%  7%  5.4% 296 0 0%

Exiting... (Quit)
--


RE: No cache/buffer is used for m3u8 "playlists", causes stuttering video - flyingrat - 2013-02-01

Arrghh, hate xcode 4. Where has the freaking bookmark function (Meta-D) gone now then? Anyone??