Article

JavaScript Object Notation (JSON) Chapter Format for HTTP Live Streaming

Understand the JSON chapter formatting required for HTTP Live Streaming.

Overview

This document describes how to provide chapter markers and other per-chapter metadata used by iOS, tvOS, and macOS for HTTP Live Streaming (HLS). This metadata is provided by the content provider in JSON format, and is referenced through the use of the EXT-X-SESSION-DATA tag in the HLS Master Playlist.

This document specifies:

  • What type of chapter metadata can be provided.

  • How to define the chapter metadata in JSON format.

  • How to declare the location of the chapter data in the HLS Master Playlist.

  • How to validate that the JSON data is in the correct format.

  • How the metadata appears in the API.

Chapter Metadata

Chapter data is information that is provided to describe a chapter. Four different types of data can be provided for each chapter: timing, titles, images, and general metadata. Keep the following in mind when designing your chapter data:

  • Each chapter is required to have a start time and may have an optional duration.

  • Each chapter may have multiple titles, multiple images, and multiple metadata items.

  • Each title in a chapter has a unique BCP-47 language tag.

  • Each metadata item in a chapter has a unique key and language.

JSON Format

JSON is a format that uses human-readable text to define data objects. JSON is described in RFC7159. Here is a simple example of chapter data formatted as JSON:

[
    {
        "chapter": 1,
        "start-time": 0,
        "titles": [
            {
                "language": "en",
                "title": "birth"
            },
            {
                "language": "es",
                "title": "nacimiento"
            }
        ]
    },
    {
        "chapter": 2,
        "start-time": 500.1,
        "titles": [
            {
                "language": "en",
                "title": "life"
            },
            {
                "language": "es",
                "title": "vida"
            }
        ]
    },
    {
        "chapter": 3,
        "start-time": 1200.2,
        "titles": [
            {
                "language": "en",
                "title": "death"
            },
            {
                "language": "es",
                "title": "muerte"
            }
        ]
    }
]

The above example has three chapters. Only chapter titles and start-times are present. Each chapter title is presented in two different languages.

Here is the general layout of the JSON chapter data format. (Note that JavaScript comments are not legal JSON.)

[
    {
        "chapter": number,
        "start-time": number,
        "duration": number,
        "titles": [
            {
                "language": string,
                "title": string
            },
            // … (for each language for this chapter)
        ],
        "images": [
            {
                "image-category": string,
                "pixel-width": number,
                "pixel-height": number,
                "url": string
            },
            // … (for each image type for this chapter)
        ],
        "metadata": [
            {
                "key": string,
                "value": object | number | string | array,
                "language": string,
            },
            // … for each metadata value for this chapter
        ],
    },
    // … for each chapter
]

Chapter metadata is presented as an array of chapters. Each element in the array contains information about that chapter. The first array element describes the first chapter, the second element describes the second chapter, and so on.

Chapter Entries

Each element in the chapter array is a JSON object called a chapter entry. The chapter entry must contain a start-time. The chapter entry should contain titles, images, and possibly metadata.

The chapter entry may contain an item named chapter to promote human readability of the JSON. The chapter entry may contain duration. Chapter entries are assumed to have a duration from start-time to the start-time of the next chapter entry, unless duration is specified. Chapter timing can overlap and nest, in which case both start-time and duration must be specified.

Titles

Chapter entries may contain a titles JSON array. Each element of a titles array is a JSON object that must contain both a language BCP-47 string and a title string containing the title for that chapter, written in the language specified by the language string. All strings must be encoded using UTF-8. Set the language to "und" if the title strings are language-neutral, such as a numeric string.

Images

Chapter entries may contain an images JSON array. Each element of a images array is a JSON object containing information about images for each chapter. For example, an images array may contain an element for a thumbnail image and an element for an HD image.

Each element in an images array is a JSON object and must contain an image-category string, a pixel-width number, a pixel-height number and an url string.

The image-category string should be the same across chapters for images that are similar. In the case mentioned above you would use one string for the thumbnails (e.g. ‘thumbnail’) and a different string for the HD images (e.g. ‘hd’).

The url string must be an absolute or relative URL to the image data associated with the chapter. Relative URLs are relative to the path that contained the JSON Chapter document.

The images files themselves may be in a variety of image formats. For example, JPEG, PNG, and TIFF are all supported.

Metadata

Chapter entries may also contain a metadata JSON array. Each element of a metadata array is a JSON object containing metadata for that chapter. Each metadata array element contains a mandatory key and value along with an optional language BCP-47 string. The key element must be a string. The value element can be a string, number, array or object. If any value is a URI, it is passed as is. The system has no way to interpret a relative URI in that context.

Master Playlist

For use in HTTP Live Streaming, JSON formatted chapter data must be specified in a Master Playlist using the EXT-X-SESSION-DATA tag.

The DATA-ID attribute of the EXT-X-SESSION-DATA must be "com.apple.hls.chapters". The URI attribute must point the the JSON-formatted chapter data. The URI may be absolute, or relative to the path that contained the master playlist.

For example: #EXT-X-SESSION-DATA:DATA-ID="com.apple.hls.chapters",URI="http://meta.example.com/movie403/chapters.json"

Validation

The following JSON schema can be used with a JSON schema validator to validate your chapter data:

{
    "$schema": "http://json-schema.org/draft-04/schema#",
    "title": "HLS Chapter Data",
    "description": "HLS chapter data format",
    "type": "array",
    "items": {
        "description": "chapter entry",
        "type": "object",
        "properties": {
            "chapter": {
                "description": "Chapter number (optional)",
                "type": "number",
                "minimum": 1
            },
            "start-time": {
                "description": "Chapter start time",
                "type": "number",
                "minimum": 0
            },
            "duration": {
                "description": "Chapter duration (optional)",
                "type": "number",
                "minimum": 0,
                "exclusiveMinimum": true
            },
            "titles": {
                "description": "List of titles by language for chapter (optional)",
                "type": "array",
                "items": {
                    "description": "Title object",
                    "type": "object",
                    "properties": {
                        "language": {
                            "description": "BCP 47 language code; und for undefined",
                            "type": "string"
                        },
                        "title": {
                            "description": "Chapter title string",
                            "type": "string"
                        }
                    },
                    "required": ["language", "title"]
                }
            },
            "images": {
                "description": "List of images for chapter (optional)",
                "type": "array",
                "items": {
                    "description": "Image object",
                    "type": "object",
                    "properties": {
                        "image-category": {
                            "description": "Image category",
                            "type": "string"
                        },
                        "pixel-width": {
                            "description": "Pixel width",
                            "type": "integer",
                            "minimum": 0,
                            "exclusiveMinimum": true
                        },
                        "pixel-height": {
                            "description": "Pixel height",
                            "type": "integer",
                            "minimum": 0,
                            "exclusiveMinimum": true
                        },
                        "url": {
                            "description": "URL to image (relative or absolute)",
                            "type": "string"
                        }
                    },
                    "required": ["image-category", "pixel-width", "pixel-height", "url"]
                }
            },
            "metadata": {
                "description": "List of metadata entries for chapter (optional)",
                "type": "array",
                "items": {
                    "description": "Metadata object",
                    "type": "object",
                    "properties": {
                        "key": {
                            "description": "Key value name",
                            "type": "string"
                        },
                        "value": {
                            "description": "Metadata value",
                            "type": ["string", "number", "boolean", "array", "object"]
                        },
                        "language": {
                            "description": "BCP 47 language code (optional)",
                            "type": "string"
                        }
                    },
                    "required": ["key", "value"]
                }
            }
        },
        "required": ["start-time"]
    }
}

Using Chapter Data in Your App

Testing

You can use QuickTime Player to quickly test your HLS streams with chapter data. QuickTime Player will display a chapter pop-up control (with images if you have them). In QuickTime Player, use File > Open Location… or ⌘L to open a URL.

QuickTime Player will display your chapters in the order they are in the JSON file, it does no sorting or rearrangement.

Accessing the data

The Media Playback Programming Guide (in the chapter “Refining The User Experience”) has a description of how to access chapter data. The methods described there return an array of AVTimedMetadataGroup objects, one object for each chapter. The order of the groups matches the order of the JSON file.

Each AVTimedMetadataGroup object has a start time, end time, and a list of AVMetadataItem. Every item from the titles, images, and metadata arrays in the JSON will be in the list of metadata items.

The images will have an extraAttributes dictionary. This dictionary will contain a key "iTunesImageResolution" whose value is a dictionary that contains the pixel-width, pixel-height, and image-category from the JSON entry.

The metadata item keys will be placed in the key space quickTimeMetadata. This key space defines its key values to be expressed as reverse-DNS strings. This allows you to define your own keys in a well established way that avoids collisions.

See Also

Metadata Manipulation

Finding Metadata Values

Retrieve and filter all of the metadata associated with an asset.

class AVMetadataItem

A metadata item associated with an audiovisual asset or one of its tracks.

class AVMetadataGroup

A collection of metadata items associated with a timeline segment.

class AVMetadataItemFilter

An object that filters selected information from a metadata item.

class AVMetadataItemValueRequest

An object used to respond to a request to load the value of a metadata item.

class AVMutableMetadataItem

A mutable metadata item associated with an audiovisual asset or with one of its tracks.

class AVDateRangeMetadataGroup

A collection of metadata items that are valid for use within a specific date range.

class AVMutableDateRangeMetadataGroup

A mutable collection of metadata items that are valid for use within a specific range of dates.

class AVTimedMetadataGroup

A collection of metadata items that are valid for use during a specific time range.

class AVMutableTimedMetadataGroup

A mutable collection of metadata items that are valid for use during a specific time range.

struct AVMetadataFormat

A value that defines a metadata format.