I’m using AVFoundation in my iPhone application to encode a video in MP4 format with H.264, which can then be shared or exported.
Do I need to pay a license for using the H.264 format to MPEG LA? Or are these fees already covered by Apple?
I’ve read articles suggesting that Apple covers these fees when encoding is done through its native APIs (or via its dedicated encoding hardware components), but I haven’t found any explicit confirmation of this point in the various documentation or contracts... Did I miss something?
Video
RSS for tagDive into the world of video on Apple platforms, exploring ways to integrate video functionalities within your iOS,iPadOS, macOS, tvOS, visionOS or watchOS app.
Selecting any option will automatically load the page
Post
Replies
Boosts
Views
Activity
We have had the same video player in our app for at least 5 years with few issues but the iOS 18 updated has now resulted in video playback for our users who have downloaded the video for offline viewing is now played at 2x speed.
Hello,
To create a test project, I want to understand how the video and audio settings would look for a destination video whose content comes from a source video.
I obtained the output from the source video in the audio and video tracks as follows:
let audioSettings = [
AVFormatIDKey: kAudioFormatLinearPCM,
AVSampleRateKey: 44100,
AVNumberOfChannelsKey: 2
] as [String : Any]
var audioOutput = AVAssetReaderTrackOutput(track: audioTrack!,
outputSettings: audioSettings)
// Video
let videoSettings = [
kCVPixelBufferPixelFormatTypeKey: kCVPixelFormatType_32BGRA,
kCVPixelBufferWidthKey: videoTrack!.naturalSize.width,
kCVPixelBufferHeightKey: videoTrack!.naturalSize.height
] as [String: Any]
var videoOutput = AVAssetReaderTrackOutput(track: videoTrack!, outputSettings: videoSettings)
With this, I'm obtaining the
CMSampleBuffer
using
AVAssetReader.copyNextSampleBuffer
.
How can I add it to the destination video?
Should I use a while loop, considering I already have the
AVAssetWriter
set up?
Something like this:
while let buffer = videoOutput.copyNextSampleBuffer() {
if let imgBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) {
let frame = imgBuffer as CVPixelBuffer
let time = CMSampleBufferGetPresentationTimeStamp(sampleBuffer)
adaptor.append(frame, withMediaTime: time)
}
}
Lastly, regarding the destination video.
How should the
AVAssetWriterInput
for audio and PixelBuffer of the destination video be set up?
Provide an example, something like:
let audioSettings = […] as [String: Any]
Looking forward to your response.
Since iOS/iPadOs/tvOS 18 then we have run into a new problem with streaming of FairPlay encrypted video. On the affected streams then the audio plays perfectly but the video freezes for periods of a few seconds, so it will freeze for 5s or so, then be OK for a few seconds then freeze again.
It is entirely reproducible when all the following are true
the video streams were produced by a particular encoder (or particular settings, not sure on that)
the video must be encrypted
device is running some variety of iOS 18 (or iPadOS or tvOS)
the device is an affected device
Known devices are
AppleTV 4K 2nd Gen
iPad Pro 11" 1st and 2nd gen
Devices known not to show the problem are
all other AppleTV models
iPhone 13 Pro and 16 Pro
If we stream the same content, but unencrypted, then it plays perfectly, or if you play the encrypted stream on, say, tvOS 17.
When the freezing occurs then we can see in the console logs repeating blocks of lines like the following
default 18:08:46.578582+0000 videocodecd AppleAVD: AppleAVDDecodeFrameResponse(): Frame# 5771 DecodeFrame failed with error 0x0000013c
default 18:08:46.578756+0000 videocodecd AppleAVD: AppleAVDDecodeFrameInternal(): failed - error: 316
default 18:08:46.579018+0000 videocodecd AppleAVD: AppleAVDDecodeFrameInternal(): avdDec - Frame# 5771, DecodeFrame failed with error: 0x13c
default 18:08:46.579169+0000 videocodecd AppleAVD: AppleAVDDisplayCallback(): Asking fig to drop frame # 5771 with err -12909 - internalStatus: 315
also more relevant looking lines:
default 18:17:39.122019+0000 kernel AppleAVD: avdOutbox0ISR(): FRM DONE (cid: 2.0, fno: 10970, codecT: 1) FAILED!!
default 18:17:39.122155+0000 videocodecd AppleAVD: AppleAVDDisplayCallback(): Asking fig to drop frame # 10970 with err -12909 - internalStatus: 315
default 18:17:39.122221+0000 kernel AppleAVD: ## client[ 2.0] @ frm 10970, errStatus: 0x10
default 18:17:39.122338+0000 kernel AppleAVD: decodeFailIdentify(): VP error bit 4 has EP3B0 error
default 18:17:39.122401+0000 kernel AppleAVD: processHWResponse(): clientID 2.0 frameNumber 10970 error 315, offsetIndex 10, isHwErr 1
So it would seem to me that one of the following must be happening:
When these particular HLS files are encrypted then the data is being corrupted in some way that played back on iOS 17 and earlier but now won't on 18+, or
There's a regression in iOS 18 that means that this particular format of video data is corrupted on decryption
If anyone has seen similar behaviour, or has any ideas how to identify which of the two scenarios it is, please say.
Unfortunately we don't have control of the servers so can't make changes there unless we can identify they are definitely the cause of the problem.
Thanks, Simon.
Topic:
Media Technologies
SubTopic:
Video
My app currently captures video using an AVCaptureSession set with the AVCaptureSessionPreset1920x1080 preset. However, I'd like to update this behavior, such that video can be recorded at a range of different resolutions.
There isn't a preset aligning to each desired resolution, so I thought I'd instead directly set the AVCaptureDeviceFormat. For any desired resolution, I would find the format that is closest without going under the desired resolution, and then crop it down as a post-processing step.
However, what I've observed is that there can be a range of available formats for a device at each resolution, with various differing settings. Presumably there is logic within AVCaptureSession that selects a reasonable default based on all these different settings, but since I am applying the format directly, I think I don't have a way to make use of that default logic? And it is undocumented?
Does this mean that the only way to select a format is to implement a comparison function that considers all different values of all different properties on AVCaptureDeviceFormat, and then sort the formats according to this comparator?
If so, what if some new property is added to AVCaptureDeviceFormat in the future? The sort would not take this new property into account, and the function might select a format with some new undesired property.
Are there any guarantees about what types for formats will be supported on a device? For example, can I take for granted that a '420v' format will exist at each resolution? If so I could filter the formats down only to those with this setting without risking filtering out all of the supported formats.
I suspect I may be missing something obvious. Any help would be greatly appreciated!
I'm trying to add metadata every second during video capture in the Swift sample App "AVMultiCamPiP". A simple string that changes every second with a write function triggered by a Timer. Can't get it to work, no matter how I arrange it, always ends up with the error "Cannot create a new metadata adaptor with an asset writer input that has already started writing".
This is the setup section:
// Add a metadata input
let assetWriterMetaDataInput = AVAssetWriterInput(mediaType: .metadata, outputSettings: nil, sourceFormatHint: AVTimedMetadataGroup().copyFormatDescription())
assetWriterMetaDataInput.expectsMediaDataInRealTime = true
assetWriter.add(assetWriterMetaDataInput)
self.assetWriterMetaDataInput = assetWriterMetaDataInput
This is the timed metadata creation which gets triggered every second:
let newNoteMetadataItem = AVMutableMetadataItem()
newNoteMetadataItem.value = "Some string" as (NSCopying & NSObjectProtocol)?
let metadataItemGroup = AVTimedMetadataGroup.init(items: [newNoteMetadataItem], timeRange: CMTimeRangeMake( start: CMClockGetTime( CMClockGetHostTimeClock() ), duration: CMTime.invalid ))
movieRecorder?.recordMetaData(meta: metadataItemGroup)
This function is supposed to add the metadata to the track:
func recordMetaData(meta: AVTimedMetadataGroup) {
guard isRecording,
let assetWriter = assetWriter,
assetWriter.status == .writing,
let input = assetWriterMetaDataInput,
input.isReadyForMoreMediaData else {
return
}
let metadataAdaptor = AVAssetWriterInputMetadataAdaptor(assetWriterInput: input)
metadataAdaptor.append(meta)
}
I have an older code example in objc which works OK, but it uses "AVCaptureMetadataInput appendTimedMetadataGroup" and writes to an identifier called "quickTimeMetadataLocationNote". I'd like to do something similar in the above Swift code ...
All suggestions are appreciated !
We have developed a simple video player Swift application for macOS, which uses the AVFoundation Framework. A special feature of this app is the ability to play the video backward with speeds like -0.25x, -0.5x, and -1.0x. MP4 video file is played directly from the local file system, video codec is h.264, and audio AAC. Video files are huge, like 10 GB, and a length of 3 hours.
Playing video in reverse direction works well on a Macbook Air with M1 or M2 chip. When we run the same app with the same video on a Macbook Air with M3 chip the reverse playback is much worse. Playback might stutter badly, especially in the latter part of the video. This same behavior also happens in Apple's Quicktime video player when playing in the reverse direction with -1x speed. What's even more strange is that at one point of a time, the video playback is totally smooth, but again, after a while, the playback is stuttering. For example, this morning reverse playback worked 100 % smoothly, then I rebooted the Mac and tried again: the result was stuttering. After this the Mac stayed idle for several hours and I tried to reverse play video again: smooth performance! My conclusion: M3 playback works fine if the stars in the sky are aligned correctly. :-)
So it's not only our app, but also Quicktime player is having exactly the same behavior. And only with the M3 chip. The same symptom appears with another similar M3 Mac, so it can't be a single fault. At the same time, open-source video player iina can reverse play the video well on the same Mac.
All Macs have otherwise identical configuration: 16 GB RAM and macOS 15.1.1.
Have you experienced the same problem? Any chance to solve this problem?
I really hope that the M4 chip Mac is behaving better here.
I am creating an AVComposition and using it with an AVPlayer. The player works fine and doesn't consume much memory when I do not set playerItem.videoComposition. Here is the code that works without excessive memory usage:
func configurePlayer(composition: AVMutableComposition, videoComposition: AVVideoComposition) {
player.pause()
player.replaceCurrentItem(with: nil)
let playerItem = AVPlayerItem(asset: composition)
player.play()
}
However, when I add playerItem.videoComposition = videoComposition, as in the code below, the memory usage becomes excessive:
func configurePlayer(composition: AVMutableComposition, videoComposition: AVVideoComposition) {
player.pause()
player.replaceCurrentItem(with: nil)
let playerItem = AVPlayerItem(asset: composition)
playerItem.videoComposition = videoComposition
player.play()
}
Issue Details:
The memory usage seems to depend on the number of video tracks in the composition, rather than their duration. For instance, two videos of 30 minutes each consume less memory than 40 videos of just 2 seconds each.
The excessive memory usage is showing up in the Other Processes section of Xcode's debug panel.
For reference, 42 videos, each less than 30 seconds, are using around 1.4 GB of memory.
I'm struggling to understand why adding videoComposition causes such high memory consumption, especially since it happens even when no layer instructions are applied. Any insights on how to address this would be greatly appreciated. Before After
I initially thought the problem might be due to having too many layer instructions in the video composition, but this doesn't seem to be the case. Even when I set a videoComposition without any layer instructions, the memory consumption remains high.
Hello!
I am building a video camera app and trying to implement Apple log for iPhone 15 Pro and 16 Pro.
I am not seeing a lot of documentation on it and notice the amount of apps that use it on the app is rather limited. Less an 5 to be exact.
Is Apple Log recording a feature that is accessible to developers?
Here is a link to documentation: https://developer.apple.com/documentation/avfoundation/avcapturecolorspace/applelog
I see in most of the old sample codes from Apple that when using AVAssetWriter to append audio, video, and metadata samples in a real time camera recording setup, calls to .append(sampleBuffer) are either synchronised using an NSLock or all the samples are sent to the asset writer on the same dispatch queue thereby preventing concurrent writes. However I can't find any documentation that calls to assetWriterInput.append(sampleBuffer) for different media samples such as Audio and Video should not be done concurrently. Is it not valid for these methods to be executed in parallel for instance?
`videoSamplesAssetWriterInput.append(videoSampleBuffer)` from DispatchQueue 1
`audioSamplesAssetWriterInput.append(audioSampleBuffer)` from DispatchQueue 2
Can anyone explain how AVAssetExportSession works in iOS 18 and earlier versions?
How do we export a spatial video at a higher resolution than 2200 x 2200?
For example, I want to export a video that I edited in a spatial timeline as 4096 x 4096 resolution with spatial metadata to output as an MV-HEVC file.
I looked under both Final Cut and Compressor export settings, but couldn't find a way to do this.
Thanks for your help!
Hello,
I work on a video streaming app, and I have been working on this crash that we are seeing quite frequently (it is our #2 crasher at the moment). The stack trace indicates that an AVPictureInPictureController is being deallocated on a background thread. This leads to dangling AutoLayout constraints getting cleaned up, further resulting in an exception being thrown from the framework about the layout engine being accessed from a background thread.
We have internal analytics which indicate that the crash occurs after the user comes back to the app after putting it in the background, and switches playback from Picture in Picture mode back to the app's regular playback UI.
What has me puzzled here is that, as I'm sure you know, we have no control over the PIP UI. It is entirely system-provided, and there is none of our app's code in the stack trace. The whole process is even initiated by a KVO on an AVPlayerController property, and our app doesn't use that class directly anywhere. So how did we manage to cause this process to happen on a background thread?
Add to this the fact that the bug only appeared once we switched to compiling with Xcode 16, and is overwhelmingly present only on devices running iOS 18.
These factors lead me to believe that this is probably an OS issue. But before I go to file a feedback, I thought I would post here in case anyone has any ideas. I have attached an instance of the crash log to this post.
2025-01-08_17-32-45.7003_-0800-ea8d5c3323e0f1fc059cf83f6ec86377bdae1788.crash
I have AVPlayer with AVPictureInPictureController. Play video in app and picture In Picture works except one situation. Issue is: I pause video in application and during switch to background is not PiP activate. What do I wrong?
import UIKit
import AVKit
import AVFoundation
class ViewControllerSec: UIViewController,AVPictureInPictureControllerDelegate {
var pipPlayer: AVPlayer!
var avCanvas : UIView!
var pipCanvas: AVPlayerLayer?
var pipController: AVPictureInPictureController!
var mainViewControler : UIViewController!
var playerItem : AVPlayerItem!
var videoAvasset : AVAsset!
public func link(to parentViewController : UIViewController) {
mainViewControler = parentViewController
setup()
}
@objc func appWillResignActiveNotification(application: UIApplication) {
guard let pipController = pipController else {
print("PiP not supported")
return
}
print("PIP isSuspend: \(pipController.isPictureInPictureSuspended)")
print("PIP isPossible: \(pipController.isPictureInPicturePossible)"
if playerItem.status == .readyToPlay {
if pipPlayer.rate == 0 {
pipPlayer.play()
}
pipController.startPictureInPicture(). ---> Errorin log: Failed to start picture in picture.
} else {
print("Player not ready for PiP.")
}
}
private func setupAudio() {
do {
let session = AVAudioSession.sharedInstance()
try session.setCategory(.playback, mode: .moviePlayback)
try session.setActive(true)
} catch {
print("Audio session setup failed: \(error.localizedDescription)")
}
}
@objc func playerItemDidFailToPlayToEnd(_ notification: Notification) {
if let error = notification.userInfo?[AVPlayerItemFailedToPlayToEndTimeErrorKey] as? Error {
print("Failed to play to end: \(error.localizedDescription)")
}
}
func setup() {
setupAudio()
guard let videoURL = URL(string: "https://demo.unified-streaming.com/k8s/features/stable/video/tears-of-steel/tears-of-steel.mp4/.m3u8") else { return }
videoAvasset = AVAsset(url: videoURL)
playerItem = AVPlayerItem(asset: videoAvasset)
addPlayerObservers()
pipPlayer = AVPlayer(playerItem: playerItem)
avCanvas = UIView(frame: view.bounds)
pipCanvas = AVPlayerLayer(player: pipPlayer)
guard let pipCanvas else { return }
pipCanvas.frame = avCanvas.bounds
//pipCanvas.videoGravity = .resizeAspectFill
mainViewControler.view.addSubview(avCanvas)
avCanvas.layer.addSublayer(pipCanvas)
if AVPictureInPictureController.isPictureInPictureSupported() {
pipController = AVPictureInPictureController(playerLayer: pipCanvas)
pipController?.delegate = self
pipController?.canStartPictureInPictureAutomaticallyFromInline = true
}
let playButton = UIButton(frame: CGRect(x: 20, y: 50, width: 100, height: 50))
playButton.setTitle("Play", for: .normal)
playButton.backgroundColor = .blue
playButton.addTarget(self, action: #selector(playTapped), for: .touchUpInside)
mainViewControler.view.addSubview(playButton)
let pauseButton = UIButton(frame: CGRect(x: 140, y: 50, width: 100, height: 50))
pauseButton.setTitle("Pause", for: .normal)
pauseButton.backgroundColor = .red
pauseButton.addTarget(self, action: #selector(pauseTapped), for: .touchUpInside)
mainViewControler.view.addSubview(pauseButton)
let pipButton = UIButton(frame: CGRect(x: 260, y: 50, width: 150, height: 50))
pipButton.setTitle("Start PiP", for: .normal)
pipButton.backgroundColor = .green
pipButton.addTarget(self, action: #selector(startPictureInPicture), for: .touchUpInside)
mainViewControler.view.addSubview(pipButton)
print("Error:\(String(describing: pipPlayer.error?.localizedDescription))")
NotificationCenter.default.addObserver(forName: UIApplication.didEnterBackgroundNotification, object: nil, queue: nil) { [weak self] _ in
guard let self = self else { return }
if self.pipPlayer.rate == 0 {
self.pipPlayer.play()
pipController?.startPictureInPicture()
}
}
func addPlayerObservers() {
playerItem?.addObserver(self, forKeyPath: "status", options: [.old, .new], context: nil)
NotificationCenter.default.addObserver(self, selector: #selector(playerDidFinishPlaying(_:)), name: .AVPlayerItemDidPlayToEndTime, object: playerItem)
}
override func observeValue(forKeyPath keyPath: String?, of object: Any?, change: [NSKeyValueChangeKey : Any]?, context: UnsafeMutableRawPointer?) {
if keyPath == "status" {
if let statusNumber = change?[.newKey] as? NSNumber {
let status = AVPlayer.Status(rawValue: statusNumber.intValue)!
switch status {
case .readyToPlay:
print("Player is ready to play")
case .failed:
print("Player failed: \(String(describing: playerItem?.error))")
case .unknown:
print("Player status is unknown")
@unknown default:
fatalError()
}
}
}
}
@objc func playerDidFinishPlaying(_ notification: Notification) {
print("Video finished playing.")
}
deinit {
playerItem?.removeObserver(self, forKeyPath: "status")
NotificationCenter.default.removeObserver(self)
}
@objc func playTapped() {
pipPlayer.play()
}
@objc func pauseTapped() {
pipPlayer.pause()
}
@objc func startPictureInPicture() {
if let pipController = pipController, !pipController.isPictureInPictureActive {
pipController.startPictureInPicture()
}
}
@objc func stopPictureInPicture() {
if let pipController = pipController, pipController.isPictureInPictureActive {
pipController.stopPictureInPicture()
}
}
func pictureInPictureController(_ pictureInPictureController: AVPictureInPictureController, failedToStartPictureInPictureWithError error: Error) {
print("Failed to start PiP: \(error.localizedDescription)")
if let underlyingError = (error as NSError).userInfo[NSUnderlyingErrorKey] {
print("Underlying error: \(underlyingError)")
}
}
}
In my project, i want to add emoji to my video but emoji image becomes dark when add in hdr video, im trying to convert my emoji image to hdr format or create with hdr format but seems not work. I hope someone has experiences in this case could help me.
Hello there,
I need to move through video loaded in an AVPlayer one frame at a time back or forth. For that I tried to use AVPlayerItem's method step(byCount:) and it works just fine.
However I need to know when stepping happened and as far as I observed it is not immediate using the method. If I check the currentTime() just after calling the method it's the same and if I do it slightly later (depending of the video itself) it shows the correct "jumped" time.
To achieve my goal I tried subclassing AVPlayerItem and implement my own async method utilizing NotificationCenter and the timeJumpedNotification assuming it would deliver it as the time actually jumps but it's not the case.
Here is my "stripped" and simplified version of the custom Player Item:
import AVFoundation
final class PlayerItem: AVPlayerItem {
private var jumpCompletion: ( (CMTime) -> () )?
override init(asset: AVAsset, automaticallyLoadedAssetKeys: [String]?) {
super .init(asset: asset, automaticallyLoadedAssetKeys: automaticallyLoadedAssetKeys)
NotificationCenter.default.addObserver(self, selector: #selector(timeDidChange(_:)), name: AVPlayerItem.timeJumpedNotification, object: self)
}
deinit {
NotificationCenter.default.removeObserver(self, name: AVPlayerItem.timeJumpedNotification, object: self)
jumpCompletion = nil
}
@discardableResult func step(by count: Int) async -> CMTime {
await withCheckedContinuation { continuation in
step(by: count) { time in
continuation.resume(returning: time)
}
}
}
func step(by count: Int, completion: @escaping ( (CMTime) -> () )) {
guard jumpCompletion == nil else {
completion(currentTime())
return
}
jumpCompletion = completion
step(byCount: count)
}
@objc private func timeDidChange(_ notification: Notification) {
switch notification.name {
case AVPlayerItem.timeJumpedNotification where notification.object as? AVPlayerItem [==](https://www.example.com/) self:
jumpCompletion?(currentTime())
jumpCompletion = nil
default: return
}
}
}
In short the notification never gets called thus the above is not working.
I guess the key there is that in the docs about the timeJumpedNotification: is said:
"A notification the system posts when a player item’s time changes discontinuously."
so the step(byCount:) is not considered as discontinuous operation and doesn't trigger it.
I'd be really helpful if somebody can help as I don't want to use seek(to:toleranceBefore:toleranceAfter:) mainly cause it's not accurate in terms of the exact next/previous frame as the video might have VFR and that causes repeating frames sometimes or even skipping one or another.
Thanks a lot
Hello, I am trying to get the new iPhone 16 pro to achieve 4k 120fps encoding when we are getting the video feed from the default, wide angle camera on the back. We are using the apple API to capture the individual frames from the camera as they are processed and we get them in this callback:
// this is the main callback function to handle video frames captured
func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
We are then taking these frames as they come in and encoding them using VideoToolBox. After they are encoded, they are added to a ring buffer so we can access them after they have been encoded.
The problem is that when we are encoding these frames on an iPhone 16 Pro, we are only reaching 80-90fps instead of 120fps. We have removed as much processing as we can. We get some small attributes about the frame when it comes in, encode the frame, and then add it to our ring buffer.
I have attached a sample project that is broken down as much as possible to the basic task of encoding 4k 120fps footage. Inside the sample app, there is an fps and pps display showing how many frames we are encoding per second. FPS represents how many frames we are coming in per second from the camera, and PPS represents how many frames we are processing (encoding) per second.
Link to sample project: https://github.com/jake-fishtech/EncoderPerformance
Thanks you for any help or suggestions.
I am creating an app that decodes H.265 elementary streams on iOS.
I use VideoToolBox to decode from H.265 to NV12.
The decoded data is enqueued in the CMSampleBufferDisplayLayer as a CMSampleBuffer.
However, nothing is displayed in the VideoPlayerView. It remains black.
The decoding in VideoToolBox is successful. I confirmed this by saving the NV12 data in the CMSampleBuffer to a file and displaying it using a tool.
Why is nothing displayed in the VideoPlayerView?
I can provide other source code as well.
//
// ContentView.swift
// H265Decoder
//
// Created by Kohshin Tokunaga on 2025/02/15.
//
import SwiftUI
struct ContentView: View {
var body: some View {
VStack {
Text("H.265 Player (temp.h265)")
.font(.headline)
VideoPlayerView()
.frame(width: 360, height: 640) // Adjust or make it responsive for iOS
}
.padding()
}
}
#Preview {
ContentView()
}
//
// VideoPlayerView.swift
// H265Decoder
//
// Created by Kohshin Tokunaga on 2025/02/15.
//
import SwiftUI
import AVFoundation
struct VideoPlayerView: UIViewRepresentable {
// Return an H265Player as the coordinator, and start playback there.
func makeCoordinator() -> H265Player {
H265Player()
}
func makeUIView(context: Context) -> UIView {
let uiView = UIView(frame: .zero)
// Base layer for attaching sublayers
uiView.backgroundColor = .black // Screen background color (for iOS)
// Create the display layer and add it to uiView.layer
let displayLayer = context.coordinator.displayLayer
displayLayer.frame = uiView.bounds
displayLayer.backgroundColor = UIColor.clear.cgColor
uiView.layer.addSublayer(displayLayer)
// Start playback
context.coordinator.startPlayback()
return uiView
}
func updateUIView(_ uiView: UIView, context: Context) {
// Reset the frame of the AVSampleBufferDisplayLayer when the view's size changes.
let displayLayer = context.coordinator.displayLayer
displayLayer.frame = uiView.layer.bounds
// Optionally update the layer's background color, etc.
uiView.backgroundColor = .black
displayLayer.backgroundColor = UIColor.clear.cgColor
// Flush transactions if necessary
CATransaction.flush()
}
}
//
// H265Player.swift
// H265Decoder
//
// Created by Kohshin Tokunaga on 2025/02/15.
//
import Foundation
import AVFoundation
import CoreMedia
class H265Player: NSObject, VideoDecoderDelegate {
let displayLayer = AVSampleBufferDisplayLayer()
private var decoder: H265Decoder?
override init() {
super.init()
// Initial configuration for the display layer
displayLayer.videoGravity = .resizeAspect
// Initialize the decoder (delegate = self)
decoder = H265Decoder(delegate: self)
// For simple playback, set isBaseline to true
decoder?.isBaseline = true
}
func startPlayback() {
// Load the file "cars_320x240.h265"
guard let url = Bundle.main.url(forResource: "temp2", withExtension: "h265") else {
print("File not found")
return
}
do {
let data = try Data(contentsOf: url)
// Set FPS and video size as needed
let packet = VideoPacket(data: data,
type: .h265,
fps: 30,
videoSize: CGSize(width: 1080, height: 1920))
// Decode as a single packet
decoder?.decodeOnePacket(packet)
} catch {
print("Failed to load file: \(error)")
}
}
// MARK: - VideoDecoderDelegate
func decodeOutput(video: CMSampleBuffer) {
// When decoding is complete, send the output to AVSampleBufferDisplayLayer
displayLayer.enqueue(video)
}
func decodeOutput(error: DecodeError) {
print("Decoding error: \(error)")
}
}
operation couldn't be completed. (CoreMediaErrorDomain error -42709.)
this error shows after update
He has been coming here for the last 10 days. Please!
please correct it as soon as possible
I have been using SDAVAssetExportSession to compress videos in an app I am building, everything goes very smoothly until I have my new Iphone16, on the device, the spatial audio in camera setting is turned on by default, then the SDAVAssetExportSession starts to fail. I know it has something to do with audioSetting. the current setting is something like this:
exportSession.audioSettings = [
AVFormatIDKey: kAudioFormatMPEG4AAC,
AVNumberOfChannelsKey: 2,
AVSampleRateKey: 44100,
AVEncoderBitRateKey: 128000
]
And also, this is passed to the underlying object AVAssetReader or AVAssetWriter. I am not experienced in this area, and I really had a hard time trying to figure out.
Does anyone know how to set up AVAssetReader or AVAssetWriter to process video with spatial audio tracks ? thanks in advance.