AVASSETREADER and AVAssetWriter: ideal settings

Hello,

To create a test project, I want to understand how the video and audio settings would look for a destination video whose content comes from a source video.

I obtained the output from the source video in the audio and video tracks as follows:

let audioSettings = [
    AVFormatIDKey: kAudioFormatLinearPCM,
    AVSampleRateKey: 44100,
    AVNumberOfChannelsKey: 2
] as [String : Any]
var audioOutput = AVAssetReaderTrackOutput(track: audioTrack!,
                                           outputSettings: audioSettings)

// Video
let videoSettings = [
    kCVPixelBufferPixelFormatTypeKey: kCVPixelFormatType_32BGRA,
    kCVPixelBufferWidthKey: videoTrack!.naturalSize.width,
    kCVPixelBufferHeightKey: videoTrack!.naturalSize.height
] as [String: Any]
var videoOutput = AVAssetReaderTrackOutput(track: videoTrack!, outputSettings: videoSettings)

With this, I'm obtaining the CMSampleBuffer using AVAssetReader.copyNextSampleBuffer .

How can I add it to the destination video? Should I use a while loop, considering I already have the AVAssetWriter set up?

Something like this:

while let buffer = videoOutput.copyNextSampleBuffer() {
    if let imgBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) {
        let frame = imgBuffer as CVPixelBuffer
        let time = CMSampleBufferGetPresentationTimeStamp(sampleBuffer)
        adaptor.append(frame, withMediaTime: time)
    }
}

Lastly, regarding the destination video. How should the AVAssetWriterInput for audio and PixelBuffer of the destination video be set up?

Provide an example, something like:

let audioSettings = […] as [String: Any]

Looking forward to your response.

Hello @Paulo_DEV01,

Have you tried using AVOutputSettingsAssistant?

It produces audioSettings and videoSettings based on the preset that you initialize it with.

Best regards,

Greg

Hello,

Yes, I’ve already tried that. It’s giving an error when I try to write with "crase"AVASSETWRITER"crase". How would a while loop look when writing the video, placing the content from the source video into an output video?

Hello,

Here’s the translation:

I actually want an example of two while loops after setting up the configuration: one while loop for the AVAssetReader to handle the image and audio, and then a while loop for the output video using AVAssetWriter. I understood the configuration part; I just need the part with the while loops for the source and destination videos. An example of both, handling both image and audio.

Hello @Paulo_DEV01,

You might want to take a look at this sample code: https://developer.apple.com/documentation/avfoundation/media_reading_and_writing/converting_side-by-side_3d_video_to_multiview_hevc_and_spatial_video

It does not align perfectly with your use case, but it does show how you can read a video, and write a new video, so the principles are similar to your case.

Best regards,

Greg

Hello,

Here's a detailed and formatted translation of your text into English:


I was able to add frames using the side-by-side eye example.
However, when I tried to add audio, the app crashed.
I tried debugging it, but I couldn't figure it out.

Below are the audio configurations I used.


Audio Configuration for AVAssetReader

// Track to audio
guard let audioTrack = try await asset.loadTracks(withMediaType: .audio).first else {
    fatalError("Track not loaded.")
}

let audioReaderSettings: [String: Any] = [
    AVFormatIDKey: kAudioFormatLinearPCM
]

let audioOutput = AVAssetReaderTrackOutput(track: audioTrack, outputSettings: audioReaderSettings)

if reader.canAdd(audioOutput) {
    reader.add(audioOutput)
}

Audio Configuration for AVAssetWriterInput

// Audio settings for video input
guard let loadFormat = try await audioTrack.load(.formatDescriptions).first,
      let format = loadFormat as? CMFormatDescription,
      let asbd = format.audioStreamBasicDescription else {
    fatalError("Audio format not loaded.")
}

let audioSettings: [String: Any] = [
    AVFormatIDKey: kAudioFormatMPEG4AAC,
    AVSampleRateKey: asbd.mSampleRate,
    AVEncoderBitRatePerChannelKey: 64000,
    AVNumberOfChannelsKey: asbd.mChannelsPerFrame
]

let audioInput = AVAssetWriterInput(mediaType: .audio, outputSettings: audioSettings)

guard assetWriter.canAdd(audioInput) else {
    fatalError("Error adding audio as input")
}

assetWriter.add(audioInput)

requestMediaDataWhenReady Implementation

audioInput.requestMediaDataWhenReady(on: DispatchQueue(label: "Add audio")) {
    while audioInput.isReadyForMoreMediaData {
        autoreleasepool {
            if let sampleBuffer = audioOutput.copyNextSampleBuffer() {
                audioInput.append(sampleBuffer)
            } else {
                assetWriter.finishWriting {
                    if assetWriter.status == .completed {
                        continuation.resume()
                        print("Operation completed successfully: \(url.absoluteString)")
                        msg("Operation completed successfully: \(url.absoluteString)")
                        self.canExport = true
                    } else {
                        print(assetWriter.error?.localizedDescription ?? "Error not found.")
                        msg(assetWriter.error?.localizedDescription ?? "Error not found.")
                    }
                }
            }
        }
        
        if assetWriter.status != .writing {
            break
        }
    }
}

Issue

The app keeps crashing when I try to add audio.

Please help me. There aren't good examples of audio processing with AVAssetReader.
I look forward to your response.

AVASSETREADER and AVAssetWriter: ideal settings
 
 
Q