Synchronizing Live Video Multiple AVPlayers

Question

Created Jun ’15

Replies 7

Boosts 0

Views 7.4k

Participants 3

Hello, I am currently trying to synchronize two or more AVPlayers, each with a different player item (camera angle), but I am unable to get all of the player to start playing at the same time. I am also working with live video, so I don't think I can use AVCompositions.

Current Procedure:

Pause all players
Seek all players to the desired time
If all seeks completed then Preroll all players
If all prerolls completed then tell all players to Play

Code:

- (void)syncAtTime:(CMTime)time {
    __block NSArray *players = self.players;

    const NSUInteger prerollsNeeded = players.count;
    __block NSUInteger prerollsTried = 0;
    __block NSUInteger prerollsCompleted = 0;

    void (^prerollHandler)(BOOL) = ^(BOOL complete) {
          prerollsTried++;
          if (complete) {
              prerollsCompleted++;
          }
          if (prerollsTried >= prerollsNeeded) {
              // all preroll completion handlers have ran

              // play all the players now that they SHOULD be ready
              if (prerollsCompleted >= prerollsNeeded) {
                    for (AVPlayer *player in players) {
                        [player play];
                    }
              }
          }
    };

    const NSUInteger seeksNeeded = players.count;
    __block NSUInteger seeksTried = 0;
    __block NSUInteger seeksCompleted = 0;

    void (^seekHandler)(BOOL) = ^(BOOL complete) {
          seeksTried++;
          if (complete) {
              seeksCompleted++;
          }

          if (seeksTried >= seeksNeeded) {
              // all seek completion handlers have ran
        
              // if all the seeks completed preroll and play
              if (seeksComplete >= seeksNeeded) {
                    for (AVPlayer *player in players) {
                        [player prerollAtRate:1.0 completionHandler:prerollHandler];
                    }
              }
        
          }
    };

    // pause seek all the players
    for (AVPlayer *player in players) {
          [player pause];
          [player seekToTime:time toleranceBefore:kCMTimeZero toleranceAfter:kCMTimeZero completionHandler:seekHandler];
    }

}

If anyone has any tips or suggestions please let me know!

(Yes I know I should be using Swift) 🙂

Boost

Answer 1

idrori OP

Jun ’15

hi

AVPlayer has a property called masterClock. it used to sync video only streams with external audio source.

Maybe you can use it to sync your AVPlayer instances.

To create a master clock object:

CMTimebaseRef controlTimebase;
CMTimebaseCreateWithMasterClock( CFAllocatorGetDefault(), CMClockGetHostTimeClock(), &controlTimebase );

if you have a timestamp of a specific frame/time in the movie you want to sync to, use:

CMTimebaseSetTime(controlTimebase, CMTimeMake(timebase, time_base_units));
CMTimebaseSetRate(controlTimebase, 1.0);

Then assign controlTimebase to your AVPlayer(s)

Good luck !

0

Answer 2

ncvitak OP

Jun ’15

Thanks!

I'll check this out and post results 😀

0

Answer 3

ncvitak OP

Jun ’15

It appears AVPlayer does not have timebase property, and the player's currentItem timebase is read only 😟

0

Answer 4

ncvitak OP

Jun ’15

Update:

Ok, I managed to get the players synced up with local content by first initially setting the them all to use the same clock

CMClockRef syncClock;
CMAudioClockCreate(kCFAllocatorDefault, &syncClock);

for (AVPlayer *player in self.players) {
    player.masterClock = syncClock;
}

Then after each of players finish prerolling in my previous implementation, instead of using setRate:

for (AVPlayer *player in players) {
    [player setRate:1.0 time:kCMTimeInvalid atHostTime:CMClockGetTime(syncClock)];
}

This works very well with local content on the device, but with live video the method "setRate time: atHostTime:" seems to be doing absolutely nothing.

Please leave any tips or suggestions you may have regarding using "setRate time: atHostTime:" with live video. Thanks!

0

Answer 5

ncvitak OP

Jun ’15

Accepted Answer

Update:

So I ended up using one of my technical support credits on this question, but the results weren't very pleasing regarding HTTP Live Streaming (HLS).

"Our engineers have reviewed your request and have concluded that there is no supported way to achieve the desired functionality given the currently shipping system configurations" - Apple Developer Technical Support

I guess I'll start a new post under HTTP Live Streaming, since that is the only issue now.

0

Answer 6

jlott1 OP

Aug ’15

Hey ncvitak,

I am too trying to solve this mystery. I was trying to play multiple videos from http sources and was using the same approach you mentioned, with the addition of setting on of my avplayers as a master player and added a time observer to track how in sync the other players were to the master.

Still had trouble keeping them in sync after starting them in sync. I did have a bit of success playing multiple videos using an AVComposition. I had to first make sure that certain keys were loaded from the individual asset tracks first before adding it to the composition, but the syncing was much better than when I was trying to play multiple AVPlayers. The only caveat is that I have no control over the caching policy of the AVPlayer while it is playing a AVComposition. And I know that you can't use an AVAssetResourceLoaderDelegate for managing asset resources while they are part of an AVComposition, (the AVPlayer just freezes)

So that's my 2 cents. I hope this was info you could use. Please keep me posted on your findings as well.

1

Answer 7

ncvitak OP

Aug ’15

Thanks for the AVComposition tip, I'll definitely try it out. Currently I have my players synced to a master player, and every 5 seconds the other players attempt to sync with the master unitl they are about 0.25s apart. This method is currently not very reliable, so now I am extremely interested in using the AVComposition. What keys need to be loaded in order to play HLS content in an AVComposition?

0