EXERCISE: Using Apple HTTP Live Streaming Tools and FFMPEG to create and validate HLS solutions

As mentioned in a previous post, HLS is one of the pre-dominate standards for delivering video and audio online, serving contents on a variety of devices and browser environments.

An HLS stream works on sequence of MPEG-2 TS file segments, listed in a manifest index file. The TS media segments are typically of 10 seconds long. The manifest file provides an ordered list of the URLs of the media segment files and is saved with an .m3u8 extension (Developer.apple.com, 2019).

The receiver or client downloads the manifest file, parses the URLs and downloads media segments in the sequence for playback. The URLs for segments can be a remote HTTP resource or local file on a web server, accessed using the standard http file protocol (Developer.apple.com, 2019).

Manifest index file contains details on sequence number to associate chunks from different profiles, time information, type of stream, information about the chunk duration etc (Developer.apple.com, 2019).

This post entry discusses how to create and validate an HLS stream file using Apple’s HLS tools and FFmpeg.

Apple’s HLS Tools available for download via developer.apple.com, is a series of command-line tools that can be used to create and validate HLS solutions.

They include:

• Media Stream Segmenter
• Media File Segmenter 
• Media Stream Validator 

The Media Stream Segmenter (mediastreamsegmenter) receives an MPEG-2 transport stream over an UDP network connection and divides it into a series of small media segments of equal duration. It then creates an index file containing references to the individual media segments. The index file and media segments are then deployed via web server infrastructure for streaming. The Media Stream Segmenter can produce either live or Video-on-Demand streams (Developer.apple.com, 2019).

The Media File Segmenter (mediafilesegmenter) divides a .mov, .mp4, .m4v, .m4a, or .mp3 file into small media segments and creates an index file. The index file and media segments are then deployed via web server infrastructure for streaming. The Media File Segmenter is only used for VOD streams (Developer.apple.com, 2019).

The Media Stream Validator (mediastreamvalidator) simulates an HTTP Live Streaming session and verifies that the index file and media segments conform to the HTTP Live Streaming specification. It checks for several “best practices” to ensure reliable streaming. If any errors or problems are found, a detailed diagnostic report is displayed. Validation data can be written to a JSON file using --validation-data argument. The HLS Report (hlsreport) uses the JSON file generated by Media Stream Validator to create a report for the validated stream (Developer.apple.com, 2019).


These tools provided by Apple are relatively easy to use, to create an HLS stream file from a source mp4 file simply use the following command.

mediafilesegmenter example1.mp4 -B segment -i output.m3u8

The command above created a series of .ts segments and a manifest file named output.m3u8, see figure 1.

Figure 1 - Creating an HLS stream file using Apple's Media File Segmenter.

To playback the example streams please check https://cryptogist.co.uk/stream/example1/output.m3u8

Some mediafilesegmenter command line options:

fileSequence is the default prefix for the generated .ts files. It can be changed using the -B option. Similarly the manifest file is named program_index.m3u8 by default and can be changed using -i option, duration of each segment is 10 seconds by default and can be changed using -t sec option, to specify a base url for the media file in the manifest file use the -b option.

To validate the HLS stream simply use the following command.

mediastreamvalidator output.m3u8

Figure 2 shows the result of the validation.

Figure 2 - Validation results using the Media Stream Validator

FFmpeg

FFmpeg can also be used to create an HLS stream file. Simply type the following command.
ffmpeg -i example2.mp4 -hls_time 10 output.m3u8

-hls_time defines the chunk interval in seconds

Structure of manifest file

Figure 3 - HLS Manifest file

M3U file format defines two key tags: EXTM3U and EXTINF.

• The EXTM3U tag indicates that the playlist is an extended M3U file. All HLS playlists must start with this tag.
• The EXTINF tag is a record marker, has a unique sequence number, specific the duration and describes the media file by the URL that follows it. In Figure 3 the first TS segment (segment-0.ts) refers to the URL which is local to the server and 10.00000 denotes the duration of first segment in seconds.
• The EXT-X-BITRATE tag provides information about the bitrate of the TS segment
• The EXT-X-PLAYLIST-TYPE tag provides mutability information about the event type e.g VOD.
• The EXT-X-MEDIA-SEQUENCE tag indicates the sequence number of the first URL that appears in a playlist file.

Other important tags are

• The EXT-X-TARGETDURATION tag specifies the maximum media file duration.
• The EXT-X-VERSION tag indicates the compatibility version of the playlist file.
• The EXT-X-ENDLIST tag indicates that no more media files will be added to the playlist file.

References

Developer.apple.com. (2019). Using HTTP Live Streaming. [online] Available at: https://developer.apple.com/library/archive/documentation/NetworkingInternet/Conceptual/StreamingMediaGuide/UsingHTTPLiveStreaming/UsingHTTPLiveStreaming.html [Accessed 20 May 2019].