Performance issues with `AVAssetWriter`

Question

mrousavy OP

Created Nov ’23

Replies 1

Boosts 1

Participants 2

Hey all!

I'm trying to build a Camera app that records Video and Audio buffers (AVCaptureVideoDataOutput and AVCaptureAudioDataOutput) to an mp4/mov file using AVAssetWriter.

When creating the Recording Session, I noticed that it blocks for around 5-7 seconds before starting the recording, so I dug deeper to find out why.

This is how I create my AVAssetWriter:

let assetWriter = try AVAssetWriter(outputURL: tempURL, fileType: .mov)
let videoWriter = self.createVideoWriter(...)
assetWriter.add(videoWriter)
let audioWriter = self.createAudioWriter(...)
assetWriter.add(audioWriter)
assetWriter.startWriting()

There's two slow parts here in that code:

The createAudioWriter(...) function takes ages!

This is how I create the audio AVAssetWriterInput:

// audioOutput is my AVCaptureAudioDataOutput, audioInput is the microphone
let settings = audioOutput.recommendedAudioSettingsForAssetWriter(writingTo: .mov)
let format = audioInput.device.activeFormat.formatDescription

let audioWriter = AVAssetWriterInput(mediaType: .audio,
                                     outputSettings: settings,
                                     sourceFormatHint: format)
audioWriter.expectsMediaDataInRealTime = true

The above code takes up to 3000ms on an iPhone 11 Pro!

When I remove the recommended settings and just pass nil as outputSettings:

audioWriter = AVAssetWriterInput(mediaType: .audio,
                                 outputSettings: nil)
audioWriter.expectsMediaDataInRealTime = true

...It initializes almost instantly - something like 30 to 50ms.

Starting the AVAssetWriter takes ages!

Calling this method:
```
assetWriter.startWriting()
```
...takes takes 3000 to 5000ms on my iPhone 11 Pro!

Does anyone have any ideas why this is so slow? Am I doing something wrong?

It feels like passing nil as the outputSettings is not a good idea, and recommendedAudioSettingsForAssetWriter should be the way to go, but 3 seconds initialization time is not acceptable.

Here's the full code: RecordingSession.swift from react-native-vision-camera. This gets called from here.

I'd appreciate any help, thanks!

Boost

Answer 1

mrousavy OP

Nov ’23

Some additional information:

Regarding the audio AVAssetWriter.init(...) call taking 3 seconds:

These are the recommendedVideoSettingsForAssetWriter settings I get:

["AVVideoCompressionPropertiesKey": {
  AllowFrameReordering = 1;
  AllowOpenGOP = 1;
  AverageBitRate = 47848572;
  BaseLayerFrameRate = 15;
  ExpectedFrameRate = 60;
  MaxAllowedFrameQP = 41;
  MaxKeyFrameIntervalDuration = 1;
  MinAllowedFrameQP = 15;
  MinimizeMemoryUsage = 1;
  Priority = 80;
  ProfileLevel = "HEVC_Main_AutoLevel";
  RealTime = 1;
  RelaxAverageBitRateTarget = 1;
 },
 "AVVideoWidthKey": 2160,
 "AVVideoCodecKey": hvc1,
 "AVVideoHeightKey": 3840]

(video initializes in ~10ms)

And these are the recommendedAudioSettingsForAssetWriter(..) I get:

["AVEncoderQualityForVBRKey": 91,
 "AVEncoderBitRatePerChannelKey": 96000,
 "AVEncoderBitRateStrategyKey": AVAudioBitRateStrategy_Variable,
 "AVFormatIDKey": 1633772320,
 "AVNumberOfChannelsKey": 1,
 "AVSampleRateKey": 44100]

(audio initializes in ~1500-3000ms)

Note that subsequent calls are faster. I am also changing the AVAudioSessionCategory while configuring the audio AVAssetWriter, not sure if that has anything to do with it.

Regarding the AVAssetWriter.startWriting() call taking 3-5 seconds: I still have no idea why that takes so long.

0