Real Time Messaging Protocol

The world of video on the internet has a complex history behind it and has been lead by a few major players. At the forefront of most things we do with technology are protocols. The earliest widespread protocol for transferring incremental chunks of video over the internet was RTMP. This protocol empowered the transfer of data over a persistent connection with relatively low latency communication.

A lot of major platforms depend on RTMP to power the delivery of media to their users. Although the succession of ordered images has long since become the standard for entertainment, education and news, it often is not considered critical for consumers or even content creators to be privy to the technology itself.

What is RTMP?

RTMP is the necessary technology that allows for efficient real time transfer of packets in a format that is readable as Video data. It allows for a stream of bytes to be transferred over a persistent connection using TCP.

What is TCP?

TCP utilizes the 3-way handshake process to establish connection between computers. When you send an email or access a website you are almost always using TCP. Due to the prevelance of HTTP, even though the use case of streaming video of UDP would be ideal, most media platforms use TCP for media transfer.

What is UDP?

UDP is an alternate protocol for transferring data from computer to computer. It does not require a handshake. A simple example would be VOIP voice calls. It is not feasible or so important - in most cases - to ensure that every packet of audio is transferred and the other side has received it. The other major streaming protocol besides RTMP is WebRTC. WebRTC uses UDP based connections for transferring packet data during calls. WebRTC allows for faster time-sensitive transfer of video between clients and sends smaller packet sizes.

If UDP is faster why use TCP?

It seems fair that if UDP is often faster and lighter, why not use it for all use cases? You can have a higher throughput, get less delay on data transferred and transmit fixed size packets without worrying about the receipt of all the bytes. It seems that UDP is superior.

Here's the thing: because HTTP is the standard it makes it simple for developers to work with TCP based requests.

The internet largely relies on TCP because of the reasons above, even though in some ways UDP might seem more ideal.

MPEG-DASH and HLS

While RTMP is the commonly used protocol for livestreaming video to millions of clients over the internet, the schema of the data must be defined for major Media Platforms. As much as we may think of a video as a simple file to be shared in chunks, modern requirements necessitate that we explicitly define information such as video resolution, audio files and captions.

For this we require a playlist file. This is not necessarily a playlist that plays multiple unique videos, rather it encapsulates data such that the video player or application can accomodate different viewer contexts. First there is MPEG-DASH (Dynamic Streaming over HTTP). Once the server receives the input for the livestream from the streamer it is transcoded and referenced in this playlist file.

HLS (HTTP Live Streaming) is a protocol created by Apple which achieves the same use case as MPEG-DASH but for Apple products primarily. Currently the most common and optimal video encoding for streaming over the internet is H.264.

As a provider of Media Content over the web you must support both. Ontop of being redundant, having concurrency and being positioned to process millions of streams of incoming video at a time, we must create both MPEG-DASH and HLS formats because we don't know the device the client is using.

VAST and VMAP

The final aspect we will cover in this article is Advertising. In order to satisfy the needs of Advertisers and Streaming over the internet, the web required a standard for delivering advertisements to clients. With this we use Video Ad Serving Templates and Video Multiple Ad Playlists.

What is VAST?

With Ad delivery it is ultimately up to the client player to decide how to handle these files but these files help in describing when and how ads should play along with what endpoints should be notified when certain actions occur. With a VAST tag we define tracking events for a single advertisement or video such as the start, first quartile, midpoint, pause, meta, fullscreen and etc to keep track of how users are interacting with advertisements. By doing this we can keep a statistical reference for advertisers on how their videos are performing and we of course can act on this information and make judgement calls programmatically to decide what ads are best to share with users.

What is VMAP?

Using VMAP we can define a series of advertisements that should be played during any video. We use this to empower dynamic ad insertion on the client side. The tag should provide a timeline with the unique advertisement video files to play at parts in the video. All of this requires the Ad server to understand the video it is preparing advertisements for and either guessing what points to place the advertisements or using a timeline model on a video.

We can define pre-rolls and post-rolls such that advertisements play before or after the video playback. With this level of control your business can determine exactly when and how videos are played. You may hear the schema "VPAID" but the Interactive Advertising Buerau has opted to deprecate VPAID which should be better for the community. The complexity of operating an Ad server to manage VAST and VMAP ontop of a Video Transcoding server is more than enough. Due to error rates with the interface and the difficulty in applying VPAID to mobile it has not been so favoured by publishers.

To be brief, VPAID allowed for the definition of rich interactive in-stream advertisements. You would be able to define interactive ads say for example a form or a survey or maybe multiple sections of the same ad. It would be fired using a line pointing to a javascript file. These functionalities are slowly being adopted in more secure methods to VAST and at Tycoon we are committed to ensuring our Advertising servers are compliant with IAB standards so that your advertisers can trust the implementation of their campaigns on your platform.