Live FairPlay and WebVTT

Question

Created Jan ’22

Replies 3

Boosts 0

Views 1.7k

Participants 2

I've been maintaining adaptive HLS+FairPlay streams with audio and video for years. My implementation works great. However, I'm now also implementing captions or subtitles, and I'm having trouble with the later.

I'm actually able to generate my HLS streams with WebVTT subtitles, and they work great. But as soon as I encrypt the streams, Apple players stop working (forever stall). And my fairplay implementation works perfectly when no subtitles are involved.

I'm not encrypting the webvtt chunks: they travel as plain text, as stated in Apple guidelines. I believe this may be the issue: encrypted a/v streams with an unencrypted subtitles stream. However, encrypting plain text subtitles with SAMPLE-AES has no sense to me, so far I was unable to find a single HLS example online with subtitles that also happen to have FairPlay encryption. All documents I have about FairPlay also say nothing about this,

I've also tried applying CEA-608 closed captions in the video stream, and this actually works great with FairPlay. But CEA-608 has its own issues, and so I would like to migrate that tech to WebVTT: which also works great, except when FairPlay is involved.

I understand that Apple also establishes that I could use TTML (ISMC1) inside fMP4, which I suspect may be SAMPLE-AES encryptable. However, given my customers use cases, I need to use TS format for HLS, and so I can't use fMP4.

With all this in mind, do anybody know how to properly configure HLS+FairPlay with a plain text WebVTT subtitles stream? Please note this is about live streaming, and not VOD nor offline playback.

Thanks.

Answered by kindaforgot in 705819022

I've eventually found the problem. It wasn't fairplay.

The thing was that, for different reasons, our encoders created medialist with nont matching "EXT-X-DISCONTINUITY-SEQUENCE" tag between A/V and text streams. My tests without fairplay were working fine because, after restarting the streams without encription, EXT-X-DISCONTINUITY-SEQUENCE was 0 in all medialists.

After this details was found, I was able to fix our encoders, and now webvtt works fine with fairplay enabled for audio and video in our LIVE streams.

Boost

Answer 1

DTS Engineer OP

Apple

Jan ’22

Could you file a bug report via Feedback Assistant (https://developer.apple.com/bug-reporting/) and post the feedback number in this thread? Please include a sample stream URL in your bug report, and indicate what playback mechanism you're using (AVPlayer, Safari, etc).

0

Answer 2

kindaforgot OP

Jan ’22

About the playback mechanism, the problem is the same in both IOs apps using AVFoundation and MacOS using safari:

When no fairplay, webvtt plays fine in both.
When fairplay encryption is added to audio and video (but no webvtt), player stalls forever.
Fairplay encrypted streams work fine without the added webvtt stream.

This document says that streams other than audio and video are not encrypted: https://developer.apple.com/library/archive/documentation/AudioVideo/Conceptual/HLS_Sample_Encryption/Encryption/Encryption.html#//apple_ref/doc/uid/TP40012862-CH2-SW18

This other document says that "WebVTT subtitles MUST be in text files, with an X-TIMESTAMP-MAP according to the HLS specification": https://developer.apple.com/documentation/http_live_streaming/hls_authoring_specification_for_apple_devices

So, I understand my text files with no encryption should work with fairplay. And, again, my webvtts chunks work fine when disabling encryption. May I be missing something?

0

Answer 3

kindaforgot OP

Feb ’22

Accepted Answer

I've eventually found the problem. It wasn't fairplay.

The thing was that, for different reasons, our encoders created medialist with nont matching "EXT-X-DISCONTINUITY-SEQUENCE" tag between A/V and text streams. My tests without fairplay were working fine because, after restarting the streams without encription, EXT-X-DISCONTINUITY-SEQUENCE was 0 in all medialists.

After this details was found, I was able to fix our encoders, and now webvtt works fine with fairplay enabled for audio and video in our LIVE streams.

0