> As compared to a WebSocket where the server pushes segments as soon as they're available.
This didn’t become a requirement for LL-HLS according to this blog post, but you could build the same push technique using HLS and HTTP/2 to push data to the client before it needs to ask for it: https://www.theoplayer.com/blog/impact-of-apple-ll-hls-updat... (or as suggested in the post, provide hint URLs for where the next block will be…)
I expect folks are looking at this very closely, but the current protocol obviously lets the client be in full control over what gets sent, which would be somewhat lost if relying exclusively on HTTP2 server-initiated push… For example, clients can request separate caches of low and high bitrate media to always ensure they have something to play back in time…
Oh, I missed EXT-X-PRELOAD-HINT, [1] thanks. That seems like a significant improvement, in that the client can do a hanging GET and have the server respond when the media segment exists, avoiding that transit latency. Still has to send extra requests. (I guess the silver lining there is that this is a flow control mechanism, so the server won't keep sending more data if the client isn't keeping up with previous data. My custom WebSocket protocol doesn't really do flow control right now, beyond what TCP does.)
[edit: another problem: multiple simultaneous hanging GETS are really awkward with HTTP/1.1 due to connection limits. You have to shard the domain name (which causes problems with cookie auth). Or you really need to use HTTP/2 or HTTP/3.]
I'm really confused by the HTTP/2 push thing that article mentioned. My general understanding of HTTP/2 push is that the clients implement it by adding the pushed data to their cache and nothing more. (Or don't implement it at all. IIRC Chrome dropped support.) If the client later needs that URL, it loads more quickly. But there's no JS API for doing something as soon as a push is received. Maybe Safari has a special hook in their HLS implementation that bypasses the polling delay if the media segment is pushed, but if so, I don't know where that's documented, and it's a surprising behavior. Or maybe they gave up on the idea? Dunno.
The whole HLS spec seems arcane compared to a protocol in which the server pushes data over a WebSocket. Even MSE is more complicated than I'd like. I'd rather try WebCodecs, but only Chrome supports it AFAIK. [2]
I suppose the cool kids use WebRTC for live video instead, but my use case is an NVR, where I'd like a uniform way of handling playback and live. I don't think WebRTC really does playback.
> where I'd like a uniform way of handling playback and live
Seems to me that WebRTC and HLS solve two different problems though. WebRTC largely prefers dropping packets to stay real-time while HLS buffers and preserves every frame by default rather than drop frames to stay real-time. One is designed for calls (real-time) and one is not (streaming).
That’s why HLS seems over-complicated. It’s not designed around real-time signalling, instead it’s designed around making requests for frames/bytes effectively sequentially.
Now, if you’re not distributing requests at scale, HLS is indeed overkill. But if you might have latency and want to stream uninterrupted video footage, it’s a necessary evil. If you pick WebRTC, you’ve no easy way to ask to pick up where you left off, because the default is just a stream of “real-time now” packets and dropped packets are lost forever.
MSE would be a way of capturing packets, but if the protocol doesn’t let you sequentially access bytes starting from a timestamp, you’re stuck when trying to resume a stream, no?
I might have misunderstood something, but they do seem like they serve different purposes. :)
> WebRTC largely prefers dropping packets to stay real-time
In my experience, this doesn't work as well as people say. I haven't played with WebRTC yet, but my understanding is it's based on RTP over an unreliable transport, which I am familiar with. Dropping packet by packet (no mechanism to either retransmit the dropped packet or skip sending the rest of the packets in the frame) isn't great. When you lose one packet, you lose a full frame but waste bandwidth sending the data anyway. Worse, it's usually a reference frame, so all the following frames are suspect until the next IDR frame. RTP over TCP (interleaved RTSP channels) can be better, by retransmitting lost packets belonging to a frame once you've decided to send that frame, and by skipping whole frames at once when things fall behind (observed within the application as insufficient buffer space). TCP has more rapid feedback, too. (ACKs are far more frequent than RTCP RRs.)
> while HLS buffers and preserves every frame by default rather than drop frames to stay real-time. One is designed for calls (real-time) and one is not (streaming). That’s why HLS seems over-complicated. It’s not designed around real-time signalling, instead it’s designed around making requests for frames/bytes effectively sequentially.
Sure, and the WebSocket protocol I mentioned also can preserve every frame, more simply.
The most justifiable complexity IMHO in HLS is multiple encodings (variants) of different quality so you can switch between them. But not everyone needs/wants that, or trusts the user agent to do a good job with the selection.
> MSE would be a way of capturing packets, but if the protocol doesn’t let you sequentially access bytes starting from a timestamp, you’re stuck when trying to resume a stream, no?
MSE lets you specify your own protocol. Mine [1] lets you seek to arbitrary timestamps. It has its own complexity (around clock uncertainty/drift due to cheap/unreliable hardware setups) but will be more complex if I have to deal with HLS's complexity also.
This didn’t become a requirement for LL-HLS according to this blog post, but you could build the same push technique using HLS and HTTP/2 to push data to the client before it needs to ask for it: https://www.theoplayer.com/blog/impact-of-apple-ll-hls-updat... (or as suggested in the post, provide hint URLs for where the next block will be…)
Apple’s official docs on LL-HLS: https://developer.apple.com/documentation/http_live_streamin...
Edit: the full spec is at https://datatracker.ietf.org/doc/html/draft-pantos-hls-rfc82... and doesn’t appear to mention push directly but does require HTTP2. https://www.akamai.com/blog/performance/-using-ll-hls-with-b... Mentions more details but I don’t have time to read them now.
I expect folks are looking at this very closely, but the current protocol obviously lets the client be in full control over what gets sent, which would be somewhat lost if relying exclusively on HTTP2 server-initiated push… For example, clients can request separate caches of low and high bitrate media to always ensure they have something to play back in time…