Login at Kodi Home

dataolle · (This post was last modified: 2013-01-31, 00:06 by dataolle.)

When I try to stream television from for example svtplay or other websites where they use m3u8 "playlists" files (HLS streaming), xbmc does not use the cache/buffer (the video information shows 0B 100%). This causes the video playback on my Raspberry PI to stutter badly when playing back video.
If I play single segements within the m3u8 file cache is used (5meg in my case) and playback is perfectly smooth.
Also I have downloaded/reassembled some of the m3u8 files to my webserver (using ffmpeg) and streamed from there, perfectly smooth playback, cache is used.

I also opened a issues on the svtplay plugin github page on this, but since it seems to be a issue with how xbmc handles m3u8 streams I post here instead.

For reference: (svtplay plugin: https://github.com/nilzen/xbmc-svtplay/issues/58 and nrk plugin https://github.com/takoi/xbmc-addon-nrk/issues/8 )
Is there a setting/patch to make xbmc use the cache when playing m3u8 streams?

I use xbmc 12 rc3 on raspbmc.

**popcornmix** · 2013-01-31, 00:15

The file is cached if:

Code:
if ( (flags & READ_NO_CACHE) == 0 && URIUtils::IsInternetStream(url, true) && !CUtil::IsPicture(strFileName) )

      m_flags |= READ_CACHED;

https://github.com/xbmc/xbmc/blob/master...e.cpp#L230
IsInternetStream is basically:

Code:
if (strProtocol2 == "http"  || strProtocol2 == "https"  ||

      strProtocol2 == "tcp"   || strProtocol2 == "udp"    ||

      strProtocol2 == "rtp"   || strProtocol2 == "sdp"    ||

      strProtocol2 == "mms"   || strProtocol2 == "mmst"   ||

      strProtocol2 == "mmsh"  || strProtocol2 == "rtsp"   ||

      strProtocol2 == "rtmp"  || strProtocol2 == "rtmpt"  ||

      strProtocol2 == "rtmpe" || strProtocol2 == "rtmpte" ||

      strProtocol2 == "rtmps")

So, what does the m3u8 url look like? Does the addon request READ_NO_CACHE?

dataolle · 2013-01-31, 00:29

Example URL:

Code:
"http://svtplay1l-f.akamaihd.net/i/world/open/20130129/1322336-003A/MIN_SANNING_VAR-003A-6ea900962dcad7f4_,900,348,564,1680,2800,.mp4.csmil/index_4_av.m3u8?null=&e=b2a7ace1bf661c32&id="

Here is the contents:

Code:
pi@raspbmc:~$ curl -s "http://svtplay1l-f.akamaihd.net/i/world/open/20130129/1322336-003A/MIN_SANNING_VAR-003A-6ea900962dcad7f4_,900,348,564,1680,2800,.mp4.csmil/index_4_av.m3u8?null=&e=b2a7ace1bf661c32&id=" | head -20

#EXTM3U

#EXT-X-TARGETDURATION:10

#EXT-X-ALLOW-CACHE:YES

#EXT-X-VERSION:2

#EXT-X-MEDIA-SEQUENCE:1

#EXTINF:10,

http://svtplay1l-f.akamaihd.net/i/world/open/20130129/1322336-003A/MIN_SANNING_VAR-003A-6ea900962dcad7f4_,900,348,564,1680,2800,.mp4.csmil/segment1_4_av.ts?null=&e=b2a7ace1bf661c32&id=

#EXTINF:10,

http://svtplay1l-f.akamaihd.net/i/world/open/20130129/1322336-003A/MIN_SANNING_VAR-003A-6ea900962dcad7f4_,900,348,564,1680,2800,.mp4.csmil/segment2_4_av.ts?null=&e=b2a7ace1bf661c32&id=

#EXTINF:10,

http://svtplay1l-f.akamaihd.net/i/world/open/20130129/1322336-003A/MIN_SANNING_VAR-003A-6ea900962dcad7f4_,900,348,564,1680,2800,.mp4.csmil/segment3_4_av.ts?null=&e=b2a7ace1bf661c32&id=

etc etc

I do not belive the problem is with the plugin setting READ_NO_CACHE because i get the same problem when i send the above playlist to xbmc for playback using the json-rpc API, and not use the plugin.

The TS files listed above plays perfectly when i send them one by one using the json-rpc api. also, then the cache is used.

**Memphiz** · 2013-01-31, 01:36

@popcornmix - m3u8 are not fetched via cfile - ffmpeg handles it for us - so the cachng is either missing in ffmpeg or it has to be setup somehow (iirc its dvdinputstreamffmpeg or so which handles m3u8)

**popcornmix** · 2013-01-31, 02:22

(2013-01-31, 01:36)Memphiz Wrote: @popcornmix - m3u8 are not fetched via cfile - ffmpeg handles it for us - so the cachng is either missing in ffmpeg or it has to be setup somehow (iirc its dvdinputstreamffmpeg or so which handles m3u8)

Yes, I discovered CFile::Open is not called for m3u8 files (but is for other http files). However I got lost in the maze of overloading for all the stream/demux handlers and didn't find where the file read occurs.

Pretty sure this is a general xbmc issue, rather than a Pi specific issue.

dataolle · 2013-01-31, 09:02

Confirmed the same behavior on a xbmc on a windows box. But the raspberry pi seems to suffer more when there is no cache, the windows box seem to be able to playback the video smoothly without the cache.
Should probably be fixed on all platforms.

**popcornmix** · 2013-01-31, 13:00

(2013-01-31, 09:02)dataolle Wrote: Confirmed the same behavior on a xbmc on a windows box. But the raspberry pi seems to suffer more when there is no cache, the windows box seem to be able to playback the video smoothly without the cache.
Should probably be fixed on all platforms.

Might be worth raising a trac ticket or posting in a non-Pi specific forum.

flyingrat · (This post was last modified: 2013-01-31, 15:07 by flyingrat.)

(2013-01-31, 09:02)dataolle Wrote: Confirmed the same behavior on a xbmc on a windows box. But the raspberry pi seems to suffer more when there is no cache, the windows box seem to be able to playback the video smoothly without the cache.
Should probably be fixed on all platforms.

Hi folks, just got the flu which mean I have some spare time to dig into this issue ;-) I can confirm the same problem exists on both OSX and iOS (Macs and IPads) as well.

Have a look at this video stream that works fine with a browser: "www.svtplay.se" but is staggering with XBMC (SVT Play addon) on OSX, iOS and RPi devices: "SVT.Play.Screen.Recording.mp4". This video sample is recorded with "Quicktime Screen Recorder" on a Mac. I couldn't find a simular tool for iPad but it's the same behavior.

Some stream info:

http://www.svtplay.se/video/986850?output=json (Flash swf params)
"manifest.f4m"
"master.m3u8"

M3U8 info:

Code:
#EXTM3U

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=985000,RESOLUTION=768x432,CODECS="avc1.77.30, mp4a.40.5"

http://svtplay10l-f.akamaihd.net/i/se/open/20130127/1331352-003A/TEST_YOUR_BRAIN-003A-1989ce6a99f2fce2_,900,348,564,1680,2800,.mp4.csmil/index_0_av.m3u8?null=&e=b2a7ace1bf661c32&id=

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=346000,RESOLUTION=512x288,CODECS="avc1.66.30, mp4a.40.5"

http://svtplay10l-f.akamaihd.net/i/se/open/20130127/1331352-003A/TEST_YOUR_BRAIN-003A-1989ce6a99f2fce2_,900,348,564,1680,2800,.mp4.csmil/index_1_av.m3u8?null=&e=b2a7ace1bf661c32&id=

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=562000,RESOLUTION=512x288,CODECS="avc1.66.30, mp4a.40.5"

http://svtplay10l-f.akamaihd.net/i/se/open/20130127/1331352-003A/TEST_YOUR_BRAIN-003A-1989ce6a99f2fce2_,900,348,564,1680,2800,.mp4.csmil/index_2_av.m3u8?null=&e=b2a7ace1bf661c32&id=

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1678000,RESOLUTION=1024x576,CODECS="avc1.77.30, mp4a.40.5"

http://svtplay10l-f.akamaihd.net/i/se/open/20130127/1331352-003A/TEST_YOUR_BRAIN-003A-1989ce6a99f2fce2_,900,348,564,1680,2800,.mp4.csmil/index_3_av.m3u8?null=&e=b2a7ace1bf661c32&id=

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2793000,RESOLUTION=1280x720,CODECS="avc1.77.30, mp4a.40.5"

http://svtplay10l-f.akamaihd.net/i/se/open/20130127/1331352-003A/TEST_YOUR_BRAIN-003A-1989ce6a99f2fce2_,900,348,564,1680,2800,.mp4.csmil/index_4_av.m3u8?null=&e=b2a7ace1bf661c32&id=

(2013-01-31, 13:00)popcornmix Wrote: Might be worth raising a trac ticket or posting in a non-Pi specific forum.

That's probably a good idea. Anyone who knows what forum is best suited to handle this type of issues?

zoltan · 2013-01-31, 15:18

I have the same problem streaming live from my dreambox (.m3u stream) hd channels...

~~Ned Scott~~ · 2013-01-31, 15:25

Thread moved...

flyingrat · (This post was last modified: 2013-01-31, 17:12 by flyingrat.)

(2013-01-31, 02:22)popcornmix Wrote:
(2013-01-31, 01:36)Memphiz Wrote: @popcornmix - m3u8 are not fetched via cfile - ffmpeg handles it for us - so the cachng is either missing in ffmpeg or it has to be setup somehow (iirc its dvdinputstreamffmpeg or so which handles m3u8)

Yes, I discovered CFile::Open is not called for m3u8 files (but is for other http files). However I got lost in the maze of overloading for all the stream/demux handlers and didn't find where the file read occurs.

Pretty sure this is a general xbmc issue, rather than a Pi specific issue.

I did a search for ".m3u8" (link) in the repro. Every method name below is linked to the repro src.

Folder "https://github.com/xbmc/xbmc/tree/master/xbmc/cores/dvdplayer/DVDInputStreams"

This is the call path I found so far:

1. CDVDPlayer::OpenInputStream() -> CDVDFactoryInputStream::CreateInputStream()
2. CDVDFactoryInputStream::CreateInputStream() -> CDVDInputStreamFFmpeg()
3. CDVDInputStreamFFmpeg::CDVDInputStreamFFmpeg() -> CDVDInputStream(DVDSTREAM_TYPE_FFMPEG)

1. DVDPlayer.cpp - Line 547 - CDVDPlayer::OpenInputStream()

Code:
bool CDVDPlayer::OpenInputStream()

{

  if(m_pInputStream)

    SAFE_DELETE(m_pInputStream);

  CLog::Log(LOGNOTICE, "Creating InputStream");

  // correct the filename if needed

  CStdString filename(m_filename);

  if (filename.Find("dvd://") == 0

  ||  filename.CompareNoCase("iso9660://video_ts/video_ts.ifo") == 0)

  {

    m_filename = g_mediaManager.TranslateDevicePath("");

  }

  // before creating the input stream, if this is an HLS playlist then get the

  // most appropriate bitrate based on our network settings

  if (filename.Left(7) == "http://" && filename.Right(5) == ".m3u8")

  {

    // get the available bandwidth (as per user settings)

    int maxrate = g_guiSettings.GetInt("network.bandwidth");

    if(maxrate <= 0)

      maxrate = INT_MAX;

    // determine the most appropriate stream

    m_filename = PLAYLIST::CPlayListM3U::GetBestBandwidthStream(m_filename, (size_t)maxrate);

  }

  m_pInputStream = CDVDFactoryInputStream::CreateInputStream(this, m_filename, m_mimetype);

2. "DVDFactoryInputStream.cpp - Line 92 - CDVDFactoryInputStream::CreateInputStream"

Code:
|| (item.IsInternetStream() && item.IsType(".m3u8")))

    return new CDVDInputStreamFFmpeg();

3. "DVDInputStreamFFmpeg.cpp - Line 41 - CDVDInputStreamFFmpeg::Open"

Code:
bool CDVDInputStreamFFmpeg::Open(const char* strFile, const std::string& content)

{

  if (!CDVDInputStream::Open(strFile, content))

    return false;

  return true;

}

Where in the call path is the cache supposed to be implemented ??
--

**Memphiz** · 2013-01-31, 16:13

Nope it has to be inside of ffmpeg somewhere. Cause ffmpeg handles the whole http connection for those streams. Someone needs to find out if ffmpeg has an av_option (or something else) for this to configure/enable caching.

flyingrat · 2013-01-31, 17:15

(2013-01-31, 16:13)Memphiz Wrote: Nope it has to be inside of ffmpeg somewhere. Cause ffmpeg handles the whole http connection for those streams. Someone needs to find out if ffmpeg has an av_option (or something else) for this to configure/enable caching.

Ok, doing a new dig into the ffmpeg repro...

flyingrat · 2013-02-01, 15:35

Hi folks, a brief status report whats currently going on...

First the bad news (sort of ;-). I'm getting better from my flu which means I'm probably getting back to work next week, thus I wont have as much time as I would like on the subject but I'll do my best.

The good news is that the ffmpeg libraries seems ok. My plan was to use a standalone player based on ffmpeg to verify the libraries. In this case I got the osx mplayer work on my mac. Mplayer was able to play the svt-play m3u8 streams without staggering. When I lowered the stream cache it occasionally complained about "Your system is too SLOW" which is a good problem indicator. Btw, anyone knows what the "SPS 32 referenced in buffering period" is meaning?

I also managed (which is another subject) to built the lastest stable release (xbmc-12.0.tar.gz) from scratch to make sure we track the current problem on the right release level. My current plan is to debug/trace the mplayer to find out how the cache is used m3u8 streams. Any other suggestions or ideas are welcome!

Trace output from playing a m3u8 stream with a low (32k) cache:

$ mplayer2 -cache 32 "http://svtplay7m-f.akamaihd.net/i/se//open/20130129/1230799-001A/DIVING_WITH_CRO-001A-23b378251ee65366_,900,348,564,1680,2800,.mp4.csmil/master.m3u8"

Code:
MPlayer2 UNKNOWN (C) 2000-2011 MPlayer Team

Can't init Apple Remote.

Playing http://svtplay7m-f.akamaihd.net/i/se//open/20130129/1230799-001A/DIVING_WITH_CRO-001A-23b378251ee65366_,900,348,564,1680,2800,.mp4.csmil/master.m3u8.

Resolving svtplay7m-f.akamaihd.net for AF_INET6...

Couldn't resolve name for AF_INET6: svtplay7m-f.akamaihd.net

Resolving svtplay7m-f.akamaihd.net for AF_INET...

Connecting to server svtplay7m-f.akamaihd.net[92.33.33.152]: 80...

Cache size set to 32 KBytes

Cache fill:  4.29% (1407 bytes)   

Detected file format: Apple HTTP Live Streaming format (libavformat)

Using network protocols without global network initialization. Please use avformat_network_init(), this will become mandatory later.

[NULL @ 0x7fdebb255800] non-existing SPS 0 referenced in buffering period

[NULL @ 0x7fdebb255800] non-existing SPS 32 referenced in buffering period

[NULL @ 0x7fdebbc95000] non-existing SPS 0 referenced in buffering period

[NULL @ 0x7fdebbc95000] non-existing SPS 32 referenced in buffering period

[NULL @ 0x7fdebcaf3000] non-existing SPS 0 referenced in buffering period

[NULL @ 0x7fdebcaf3000] non-existing SPS 32 referenced in buffering period

[NULL @ 0x7fdebcafaa00] non-existing SPS 0 referenced in buffering period

[NULL @ 0x7fdebcafaa00] non-existing SPS 32 referenced in buffering period

[NULL @ 0x7fdebcb1bc00] non-existing SPS 0 referenced in buffering period

[NULL @ 0x7fdebcb1bc00] non-existing SPS 32 referenced in buffering period

[h264 @ 0x7fdebb25a200] non-existing SPS 0 referenced in buffering period

[h264 @ 0x7fdebb25a200] non-existing SPS 32 referenced in buffering period

[h264 @ 0x7fdebcb6ac00] non-existing SPS 0 referenced in buffering period

[h264 @ 0x7fdebcb6ac00] non-existing SPS 32 referenced in buffering period

[h264 @ 0x7fdebcb45c00] non-existing SPS 0 referenced in buffering period

[h264 @ 0x7fdebcb45c00] non-existing SPS 32 referenced in buffering period

[h264 @ 0x7fdebcb58000] non-existing SPS 0 referenced in buffering period

[h264 @ 0x7fdebcb58000] non-existing SPS 32 referenced in buffering period

[h264 @ 0x7fdebcb7d800] non-existing SPS 0 referenced in buffering period

[h264 @ 0x7fdebcb7d800] non-existing SPS 32 referenced in buffering period

[applehttp @ 0x7fdebb041a00] Estimating duration from bitrate, this may be inaccurate

[lavf] stream 0: video (h264), -vid 0

[lavf] stream 1: audio (aac), -aid 0

[lavf] stream 2: video (h264), -vid 1

[lavf] stream 3: audio (aac), -aid 1

[lavf] stream 4: video (h264), -vid 2

[lavf] stream 5: audio (aac), -aid 2

[lavf] stream 6: video (h264), -vid 3

[lavf] stream 7: audio (aac), -aid 3

[lavf] stream 8: video (h264), -vid 4

[lavf] stream 9: audio (aac), -aid 4

VIDEO:  [H264]  768x432  0bpp  25.000 fps    0.0 kbps ( 0.0 kbyte/s)

Opening video filter: [scale]

[ass] auto-open

==========================================================================

Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family

Asking decoder to use 2 threads if supported.

[h264 @ 0x7fdebd8a6a00] non-existing SPS 0 referenced in buffering period

[h264 @ 0x7fdebd8a6a00] non-existing SPS 32 referenced in buffering period

Selected video codec: [ffh264] vfm: ffmpeg (FFmpeg H.264)

==========================================================================

==========================================================================

Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders

AUDIO: 48000 Hz, 2 ch, s16le, 0.0 kbit/0.00% (ratio: 0->192000)

Selected audio codec: [ffaac] afm: ffmpeg (FFmpeg AAC (MPEG-2/MPEG-4 Audio))

==========================================================================

AO: [coreaudio] 48000Hz 2ch s16le (2 bytes per sample)

Starting playback...

A:  10.1 V:   0.0 A-V:  0.000 ct:  0.000   0/  0 ??% ??% ??,?% 0 0 0% 

Movie-Aspect is 1.78:1 - prescaling to correct movie aspect.

[swscaler @ 0x7fdebd0fa800] BICUBIC scaler, from yuv420p to yuyv422 using MMX2

VO: [corevideo] 768x432 => 768x432 Packed YUY2 

A:  10.2 V:  10.2 A-V: -0.040 ct:  0.000   0/  0 ??% ??% ??,?% 0 0 0% 

Decreasing video pts: 10.181000 < 10.221000

A:  10.3 V:  10.3 A-V:  0.001 ct:  0.000   0/  0 ??% ??% ??,?% 0 0 0% 

Decreasing video pts: 10.141000 < 10.261000

A:  10.4 V:  10.4 A-V: -0.003 ct:  0.000   0/  0 ??% ??% ??,?% 0 0 0% 

Decreasing video pts: 10.341000 < 10.381000

Using network protocols without global network initialization. Please use avformat_network_init(), this will become mandatory later.

Using network protocols without global network initialization. Please use avformat_network_init(), this will become mandatory later.

[applehttp @ 0x7fdebb041a00] No longer receiving variant 1

[applehttp @ 0x7fdebb041a00] No longer receiving variant 2

[applehttp @ 0x7fdebb041a00] No longer receiving variant 3

[applehttp @ 0x7fdebb041a00] No longer receiving variant 4

Using network protocols without global network initialization. Please use avformat_network_init(), this will become mandatory later.

Using network protocols without global network initialization. Please use avformat_network_init(), this will become mandatory later.

Using network protocols without global network initialization. Please use avformat_network_init(), this will become mandatory later.

Using network protocols without global network initialization. Please use avformat_network_init(), this will become mandatory later.

A:  47.7 V:  47.7 A-V: -0.000 ct:  0.001   0/  0  3%  8%  3.9% 59 0 0% 

           ************************************************

           **** Your system is too SLOW to play this!  ****

           ************************************************

Possible reasons, problems, workarounds:

- Most common: broken/buggy _audio_ driver

  - Try -ao sdl or use the OSS emulation of ALSA.

  - Experiment with different values for -autosync, 30 is a good start.

- Slow video output

  - Try a different -vo driver (-vo help for a list) or try -framedrop!

- Slow CPU

  - Don't try to play a big DVD/DivX on a slow CPU! Try some of the lavdopts,

    e.g. -vfm ffmpeg -lavdopts lowres=1:fast:skiploopfilter=all.

- Broken file

  - Try various combinations of -nobps -ni -forceidx -mc 0.

- Slow media (NFS/SMB mounts, DVD, VCD etc)

  - Try -cache 8192.

- Are you using -cache to play a non-interleaved AVI file?

  - Try -nocache.

Read DOCS/HTML/en/video.html for tuning/speedup tips.

If none of this helps you, read DOCS/HTML/en/bugreports.html.

Using network protocols without global network initialization. Please use avformat_network_init(), this will become mandatory later.

A: 181.3 V: 181.3 A-V: -0.002 ct:  0.001   0/  0  1%  7%  5.1% 294 0 0% 

A: 182.1 V: 182.1 A-V:  0.000 ct:  0.001   0/  0  1%  7%  5.1% 294 0 0% 

A: 182.8 V: 182.7 A-V:  0.137 ct:  0.001   0/  0  1%  7%  5.1% 294 0 0% 

A: 184.5 V: 184.5 A-V: -0.003 ct:  0.001   0/  0  1%  7%  5.1% 295 0 0% 

A: 185.3 V: 185.3 A-V:  0.010 ct:  0.001   0/  0  1%  7%  5.0% 295 0 0% 

A: 185.8 V: 185.8 A-V:  0.000 ct:  0.001   0/  0  1%  7%  5.0% 295 0 0% 

MPlayer interrupted by signal 2 in module: decode_audio

MPlayer interrupted by signal 2 in module: enable_cache

A: 190.0 V: 189.2 A-V:  0.832 ct:  0.001   0/  0  1%  7%  5.4% 296 0 0% 

Exiting... (Quit)

--

flyingrat · 2013-02-01, 16:11

Arrghh, hate xcode 4. Where has the freaking bookmark function (Meta-D) gone now then? Anyone??