Understanding Live Video Streaming
Often, I’m getting requests like “we just need video in a standard format that we will give to all our users so that they can watch it in any browser and mobile devices”. I’ll try to answer some of the basic questions in this article with the hopes that it will help someone to save time on research.
First, you need to know what formats are already involved and what formats are required in your solution. If you work with legacy IP cameras then your solution will have to be built around the cameras formats. If you are a camera manufacturer you can consider supporting formats required by your player software. And if you are a web developer then you’ll first consider the formats supported by web browsers. Is there one single format that would work in all situations? A quick answer is no. However, there is a chance that such format will eventually emerge. As of today, the most poplar live video formats are RTSP, RTMP, HLS and WebRTC.
Let’s start from video players. On desktops, it’s a standard to watch video in browsers whereas on mobile devices, it’s more common to have dedicated video applications. A player working in a desktop browser is called web player where the main browsers are Chrome, Edge, Firefox and Safari. Mobile players are apps running on mobile devices, most mobile devices are either Android or iOS.
A web player works in a browser and as a result it should follow all the rules that a browser dictates. Unfortunately, those rules are changing and vary from browser to browser. The majority of browsers restrict the use of video plug-ins. For example, VLC offered an extremely popular plugin that browsers have stopped supporting. Today, developers need to use formats which are supported by browsers natively. Almost any browser supports HLS. The bad thing about HLS is a big latency. While it’s fine to watch event broadcast (think IPTV) it’s not suitable when you need to control your drone remotely. RTMP is great in terms of latency but it works through Adobe Flash plug-in (as of today one of few plug-ins supported by browsers) and you must enable it in browser’s settings. RTSP is not supported by any of the browsers. WebRTC is the most promising format and it’s supported by majority of browsers except Safari. What to do if you want your video to be played in any browser? As of today, the answer is to support multiple formats. Fortunately, very often you don’t need to re-encode video/audio for that, it’s simply repackaging from one format to another which is a relatively light operation even when it’s done on-the-fly.
You have much more flexibility with mobile players. It’s up to developer to incorporate support of different formats sometimes including proprietary. RTSP and RTMP are most popular for mobile apps because of low latency and relatively simple implementation. A more complex task on mobiles is to interface with mobile hardware decoders. On desktops decoding and interface with h/w acceleration is done by browsers.
If you have to support video playback in browsers then you have to look into RTMP, HLS and WebRTC. If you are OK with big latency then HLS can serve the purpose. If latency is important, then in addition to HLS you must support either RTMP or WebRTC. If you don’t need to play video in browsers then just stick to the format of your video source and make sure that it’s supported by your mobile app. For example, if your video source format is RTSP and you want to watch this stream in all major browsers as well as in mobile apps, then you’ll have to reformat video on-the-fly and stream it in HLS, RTMP and WebRTC formats.
I hope that it was helpful and not very boring. If you think the same then you are welcome to read my next article where I am going to answer basic questions about video streaming servers and how they work in different network scenarios.
Yaro Lisitsyn Co-founder | VXG (Video Experts Group)