Hello, I'm building a camera app around ARKit. I've created a Lockscreen Capture Extension and added a control to initiate my camera app, but when I launch the extension I see just a black screen with no hints at any errors. Also attaching the debugger to the running process shows no logs. Im wondering: Is LockedCameraCapture supported with ARView and ARSession? ARKit was featured in a WWDC video with a camera app use-case, also the introduction of captureHighResolutionFrame(completion:) made me pick it up as an interesting camera app backbone - but if lockscreen capture is not possible with it I have to refactor my codebase.

Hi everyone, I'm currently facing an issue with AVAudioPlayer in my SwiftUI project. Despite ensuring that the sound file "buttonsound.mp3" is properly added to the project's resources (I dragged and dropped it into Xcode), the application is still unable to locate the file when attempting to play it. Here's the simplified version of the code I'm using: import SwiftUI import AVFoundation struct ContentView: View { var body: some View { VStack { Button("Play sound") { playSound(named: "buttonsound", ofType: "mp3") } } } } func playSound(named name: String, ofType type: String) { guard let soundURL = Bundle.main.url(forResource: name, withExtension: type) else { print("Sound file not found") return } do { let audioPlayer = try AVAudioPlayer(contentsOf: soundURL) audioPlayer.prepareToPlay() audioPlayer.play() } catch let error { print("Error playing sound: \(error.localizedDescription)") } }

Hey - I am developing an app that uses the camera for recording video. I put the ability to choose a framerate and resolution and all combinations work perfectly fine, except for 4k 120fps for the new iPhone 16 pro. This just shows black on the preview. I tried to record even though the preview was black, but the recording is also just a black screen. Is there anything special that needs to be done in the camera setup for 4k 120fps to work? I have my camera setup code attached. Is it possible this is a bug in Apple's code, since this works with every other combination (1080p up to 240fps and 4k up to 60fps)? Thanks so much for the help. class CameraManager: NSObject { enum Errors: Error { case noCaptureDevice case couldNotAddInput case unsupportedConfiguration } enum Resolution { case hd1080p case uhd4K var preset: AVCaptureSession.Preset { switch self { case .hd1080p: return .hd1920x1080 case .uhd4K: return .hd4K3840x2160 } } var dimensions: CMVideoDimensions { switch self { case .hd1080p: return CMVideoDimensions(width: 1920, height: 1080) case .uhd4K: return CMVideoDimensions(width: 3840, height: 2160) } } } enum CameraType { case wide case ultraWide var captureDeviceType: AVCaptureDevice.DeviceType { switch self { case .wide: return .builtInWideAngleCamera case .ultraWide: return .builtInUltraWideCamera } } } enum FrameRate: Int { case fps60 = 60 case fps120 = 120 case fps240 = 240 } let orientationManager = OrientationManager() let captureSession: AVCaptureSession let previewLayer: AVCaptureVideoPreviewLayer let movieFileOutput = AVCaptureMovieFileOutput() let videoDataOutput = AVCaptureVideoDataOutput() private var videoCaptureDevice: AVCaptureDevice? override init() { self.captureSession = AVCaptureSession() self.previewLayer = AVCaptureVideoPreviewLayer(session: self.captureSession) super.init() self.previewLayer.videoGravity = .resizeAspect } func configureSession(resolution: Resolution, frameRate: FrameRate, stabilizationEnabled: Bool, cameraType: CameraType, sampleBufferDelegate: AVCaptureVideoDataOutputSampleBufferDelegate?) throws { assert(Thread.isMainThread) captureSession.beginConfiguration() defer { captureSession.commitConfiguration() } captureSession.sessionPreset = resolution.preset if captureSession.canAddOutput(movieFileOutput) { captureSession.addOutput(movieFileOutput) } else { throw Errors.couldNotAddInput } videoDataOutput.setSampleBufferDelegate(sampleBufferDelegate, queue: DispatchQueue(label: "VideoDataOutputQueue")) if captureSession.canAddOutput(videoDataOutput) { captureSession.addOutput(videoDataOutput) // Set the video orientation if needed if let connection = videoDataOutput.connection(with: .video) { //connection.videoOrientation = .portrait } } else { throw Errors.couldNotAddInput } guard let videoCaptureDevice = AVCaptureDevice.default(cameraType.captureDeviceType, for: .video, position: .back) else { throw Errors.noCaptureDevice } let useDimensions = resolution.dimensions guard let format = videoCaptureDevice.formats.first(where: { format in let dimensions = CMVideoFormatDescriptionGetDimensions(format.formatDescription) let isRes = dimensions.width == useDimensions.width && dimensions.height == useDimensions.height let frameRates = format.videoSupportedFrameRateRanges return isRes && frameRates.contains(where: { $0.maxFrameRate >= Float64(frameRate.rawValue) }) }) else { throw Errors.unsupportedConfiguration } self.videoCaptureDevice = videoCaptureDevice do { let videoInput = try AVCaptureDeviceInput(device: videoCaptureDevice) if captureSession.canAddInput(videoInput) { captureSession.addInput(videoInput) } else { throw Errors.couldNotAddInput } try videoCaptureDevice.lockForConfiguration() videoCaptureDevice.activeFormat = format videoCaptureDevice.activeVideoMinFrameDuration = CMTime(value: 1, timescale: CMTimeScale(frameRate.rawValue)) videoCaptureDevice.activeVideoMaxFrameDuration = CMTime(value: 1, timescale: CMTimeScale(frameRate.rawValue)) videoCaptureDevice.activeMaxExposureDuration = CMTime(seconds: 1.0 / 960, preferredTimescale: 1000000) videoCaptureDevice.exposureMode = .locked videoCaptureDevice.unlockForConfiguration() } catch { throw error } configureStabilization(enabled: stabilizationEnabled) }`

Hello there, I need to move through video loaded in an AVPlayer one frame at a time back or forth. For that I tried to use AVPlayerItem's method step(byCount:) and it works just fine. However I need to know when stepping happened and as far as I observed it is not immediate using the method. If I check the currentTime() just after calling the method it's the same and if I do it slightly later (depending of the video itself) it shows the correct "jumped" time. To achieve my goal I tried subclassing AVPlayerItem and implement my own async method utilizing NotificationCenter and the timeJumpedNotification assuming it would deliver it as the time actually jumps but it's not the case. Here is my "stripped" and simplified version of the custom Player Item: import AVFoundation final class PlayerItem: AVPlayerItem { private var jumpCompletion: ( (CMTime) -> () )? override init(asset: AVAsset, automaticallyLoadedAssetKeys: [String]?) { super .init(asset: asset, automaticallyLoadedAssetKeys: automaticallyLoadedAssetKeys) NotificationCenter.default.addObserver(self, selector: #selector(timeDidChange(_:)), name: AVPlayerItem.timeJumpedNotification, object: self) } deinit { NotificationCenter.default.removeObserver(self, name: AVPlayerItem.timeJumpedNotification, object: self) jumpCompletion = nil } @discardableResult func step(by count: Int) async -> CMTime { await withCheckedContinuation { continuation in step(by: count) { time in continuation.resume(returning: time) } } } func step(by count: Int, completion: @escaping ( (CMTime) -> () )) { guard jumpCompletion == nil else { completion(currentTime()) return } jumpCompletion = completion step(byCount: count) } @objc private func timeDidChange(_ notification: Notification) { switch notification.name { case AVPlayerItem.timeJumpedNotification where notification.object as? AVPlayerItem [==](https://www.example.com/) self: jumpCompletion?(currentTime()) jumpCompletion = nil default: return } } } In short the notification never gets called thus the above is not working. I guess the key there is that in the docs about the timeJumpedNotification: is said: "A notification the system posts when a player item’s time changes discontinuously." so the step(byCount:) is not considered as discontinuous operation and doesn't trigger it. I'd be really helpful if somebody can help as I don't want to use seek(to:toleranceBefore:toleranceAfter:) mainly cause it's not accurate in terms of the exact next/previous frame as the video might have VFR and that causes repeating frames sometimes or even skipping one or another. Thanks a lot

I am developing an iOS app with video call functionality and implementing Picture in Picture (PiP) mode for video calls. The issue I am facing is that the camera stops capturing video when the app goes to the background, even though the PiP view is still visible. I have noticed that some apps, like Telegram, manage to keep the camera working in PiP mode while the app is in the background. How can I achieve this in my app?

In my project, i want to add emoji to my video but emoji image becomes dark when add in hdr video, im trying to convert my emoji image to hdr format or create with hdr format but seems not work. I hope someone has experiences in this case could help me.

I'm trying to implement Ambisonic B-Format audio playback on Vision Pro with head tracking. So far audio plays, head tracking works, and the sound appears to be stereo. The problem is that it is not a proper binaural playback when compared to playing back the audiofile with a DAW. Has anyone successfully implemented B-Format playback on Vision Pro? Any suggestions on my current implementation: func playAmbiAudioForum() async { do { try AVAudioSession.sharedInstance().setCategory(.playback) try AVAudioSession.sharedInstance().setActive(true) // AudioFile laoding/preperation guard let testFileURL = Bundle.main.url(forResource: "audiofile", withExtension: "wav") else { print("Test file not found") return } let audioFile = try AVAudioFile(forReading: testFileURL) let audioFileFormat = audioFile.fileFormat // create AVAudioFormat with Ambisonics B Format guard let layout = AVAudioChannelLayout(layoutTag: kAudioChannelLayoutTag_Ambisonic_B_Format) else { print("layout failed") return } let format = AVAudioFormat( commonFormat: audioFile.processingFormat.commonFormat, sampleRate: audioFile.fileFormat.sampleRate, interleaved: false, channelLayout: layout ) // write audiofile to buffer guard let buffer = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: UInt32(audioFile.length)) else { print("buffer failed") return } try audioFile.read(into: buffer) playerNode.renderingAlgorithm = .HRTF // connecting nodes audioEngine.attach(playerNode) audioEngine.connect(playerNode, to: audioEngine.outputNode, format: format) audioEngine.prepare() playerNode.scheduleBuffer(buffer, at: nil) { print("File finished playing") } try audioEngine.start() playerNode.play() } catch { print("Setup error:", error) } }

I'm having the following issue: Type 'AVPlayer.Type' cannot conform to 'ObservableObject' struct MusicEditorView: View { @ObservedObject var audioPlayer = AVPlayer and this is the class: class MusicPlayer: ObservableObject { private var audioPlayer: AVPlayer? private var timer: Timer? func playSound(named sFileName: String){ if let url = Bundle.main.url(forResource: sFileName, withExtension: "mp3"){ audioPlayer = try? AVPlayer(url: url) audioPlayer?.play() } } func pause(){ audioPlayer?.pause() } func getcurrentProgress() -> Double{ guard let currentTime = audioPlayer?.currentItem?.currentTime().seconds else { return 0 } guard let duration = audioPlayer?.currentItem?.duration.seconds else { return 0 } return duration > 0 ? (currentTime / duration) * 100 : 0 } func startProgressTimer(updateProgress: @escaping (Double, Double) -> Void){ timer?.invalidate() timer = Timer.scheduledTimer(withTimeInterval: 1.0, repeats: true) { _ in guard let currentTime = self.audioPlayer?.currentItem?.currentTime().seconds else { return } guard let duration = self.audioPlayer?.currentItem?.duration.seconds else { return } updateProgress(currentTime, duration) } } func stopProgressTimer(){ timer?.invalidate() } struct Sound: Identifiable, Codable { var id = UUID() var name: String var fileName: String } } }

Hi, Apple's engineer. Hoping that you can reply to this one. We're developing a Text-to-Speak app. Everything went well until the IOS got upgraded to 18. AVSpeechSynthesisVoice(language: "zh-CN") is running well under IOS 16 AND IOS 17. It speaks Mandarin correctly. In IOS 18, we noticed that Siri's Language setting interrupted the performance of AVSpeechSynthesisVoice. It plays Cantonese instead of Mandarin. Buggy language setting in Siri that affects the AVSpeechSynthesisVoice : Chinese (Cantonese - China mainland) Chinese (Cantonese -Hong Kong)

I'm experiencing audio issues while developing for visionOS when playing PCM data through AVAudioPlayerNode. Issue Description: Occasionally, the speaker produces loud popping sounds or distorted noise This occurs during PCM audio playback using AVAudioPlayerNode The issue is intermittent and doesn't happen every time Technical Details: Platform: visionOS Device: vision pro / simulator Audio Framework: AVFoundation Audio Node: AVAudioPlayerNode Audio Format: PCM I would appreciate any insights on: Common causes of audio distortion with AVAudioPlayerNode Recommended best practices for handling PCM playback in visionOS Potential configuration issues that might cause this behavior Has anyone encountered similar issues or found solutions? Any guidance would be greatly helpful. Thank you in advance!

The AVCam sample code by Apple fails to build in Swift 6 language settings due to failed concurrency checks ((the only modification to make in that code is to append @preconcurrency to import AVFoundation). Here is a minimally reproducible sample code for one of the errors: import Foundation final class Recorder { var writer = Writer() var isRecording = false func startRecording() { Task { [writer] in await writer.startRecording() print("started recording") } } func stopRecording() { Task { [writer] in await writer.stopRecording() print("stopped recording") } } func observeValues() { Task { for await value in await writer.$isRecording.values { isRecording = value } } } } actor Writer { @Published private(set) public var isRecording = false func startRecording() { isRecording = true } func stopRecording() { isRecording = false } } The function observeValues gives an error: Non-sendable type 'Published<Bool>.Publisher' in implicitly asynchronous access to actor-isolated property '$isRecording' cannot cross actor boundary I tried everything to fix it but all in vain. Can someone please point out if the architecture of AVCam sample code is flawed or there is an easy fix?

I have an iOS application view that contains an AVCaptureSession, AVCaptureVideoPreviewLayer (created with the AVCaptureSession), and a UIImageView (in the backend the app takes the output of the AVCaptureSession, runs it through a Semantic Segmentation model, and displays the output in the UIImageView). When I pause the app and run the “Debug View Hierarchy”, it shows the UIImageView, the relevant buttons and labels. However, it does not seem to show AVCaptureVideoPreviewLayer that I have set up in my application. Is there some special set up that needs to be done to be able to view Camera Related features? The following is part of the view code, a component that is used to render the AVCaptureVideoPreviewLayer (not sure if this is enough, please let me know if its not): class CameraViewController: UIViewController { var session: AVCaptureSession? var frameRect: CGRect = CGRect() var rootLayer: CALayer! = nil private var previewLayer: AVCaptureVideoPreviewLayer! = nil init(session: AVCaptureSession) { self.session = session super.init(nibName: nil, bundle: nil) } required init?(coder: NSCoder) { super.init(coder: coder) } override func viewDidLoad() { super.viewDidLoad() setUp(session: session!) } private func setUp(session: AVCaptureSession) { previewLayer = AVCaptureVideoPreviewLayer(session: session) previewLayer.videoGravity = AVLayerVideoGravity.resizeAspectFill previewLayer.frame = self.frameRect DispatchQueue.main.async { [weak self] in self!.view.layer.addSublayer(self!.previewLayer) //self!.view.layer.addSublayer(self!.detectionLayer) } } } struct HostedCameraViewController: UIViewControllerRepresentable{ var session: AVCaptureSession! var frameRect: CGRect func makeUIViewController(context: Context) -> CameraViewController { let viewController = CameraViewController(session: session) viewController.frameRect = frameRect return viewController } func updateUIViewController(_ uiView: CameraViewController, context: Context) { } }

Support external cameras in your iPadOS app and use Swift to read multiple camera feeds? thanks

Hi everyone, I am wondering under which settings the camera(s) were set by the time they were calibrated. For instance, one aspect that is easy to find is the reference resolution of the images taken when calibrating the intrinsics, this is by retrieving intrinsicMatrixReferenceDimensions. Making sure that the principal point is referenced to the by the time resolution used when the calibration was ongoing. However, recently I saw that there are focusing modes that potentially displace the lens' physical position. Settings like: AutoFocusRangeRestriction: none, near, far setFocusModeLocked: Locks the lens position at the specified value, and sets the focus mode to a locked state. My concern lies the impact this focusing lens displacements have on the intrinsic matrix parameters, if the lens is displaced, these parameters no longer describe the camera since the lens position has changed w.r.t. the lensPosition set when they were calibrated [0-1]. If my understanding is correct the AutoFocusRangeRestriction is just a range freedom the system is allowed to auto-focus and not a specific lens position. Conversely, the setFocusModeLocked does indeed fix the lensPosition to a certain value [0 - 1]. In simple words, what is the focus lensPosition the cameras were set when calibrating them for intrnisics?

Hello dear community, I have the sample code from Apple “CapturingDepthUsingLiDAR” to access the LiDAR on my iPhone 12 Pro. My goal is to use the “photo output” function to generate a point cloud from a single image and then save it as a ply file. So far I have tested different approaches to create a .ply file from the depthmap, the intrinsic camera data and the rgba values. Unfortunately, I have had no success so far and the result has always been an incorrect point cloud. My question now is whether there are already approaches to this and whether anyone has any experience with it. Thank you very much in advance!!!

Hi everyone, I am wondering under which settings the camera(s) were set by the time they were calibrated. For instance, one aspect that is easy to find is the reference resolution of the images taken when calibrating the intrinsics, this is by retrieving intrinsicMatrixReferenceDimensions. Making sure that the principal point is referenced to the by the time resolution used when the calibration was ongoing. However, recently I saw that there are focusing modes that potentially displace the lens' physical position. Settings like: AutoFocusRangeRestriction: none, near, far setFocusModeLocked: Locks the lens position at the specified value, and sets the focus mode to a locked state. My concern lies the impact this focusing lens displacements can have on the intrinsic matrix parameters, like these parameters no longer describe the camera since the lens position has changed. In simple words, what is the focus 'mode'/'range' the cameras were set when calibrating them for intrnisics?

Can some one please answer me How Can I update the cookies of the previously set m3u8 video in AVPlayer without creating the new AVURLAsset and replacing the AVPlayer current Item with it

I see in most of the old sample codes from Apple that when using AVAssetWriter to append audio, video, and metadata samples in a real time camera recording setup, calls to .append(sampleBuffer) are either synchronised using an NSLock or all the samples are sent to the asset writer on the same dispatch queue thereby preventing concurrent writes. However I can't find any documentation that calls to assetWriterInput.append(sampleBuffer) for different media samples such as Audio and Video should not be done concurrently. Is it not valid for these methods to be executed in parallel for instance? `videoSamplesAssetWriterInput.append(videoSampleBuffer)` from DispatchQueue 1 `audioSamplesAssetWriterInput.append(audioSampleBuffer)` from DispatchQueue 2

Hello! I am building a video camera app and trying to implement Apple log for iPhone 15 Pro and 16 Pro. I am not seeing a lot of documentation on it and notice the amount of apps that use it on the app is rather limited. Less an 5 to be exact. Is Apple Log recording a feature that is accessible to developers? Here is a link to documentation: https://developer.apple.com/documentation/avfoundation/avcapturecolorspace/applelog

Hello, I'm developing a Command Line Tool in XCode, in order to capture system audio and save it to a file, which will then be used by a separate process. Everything works perfectly when running it from either XCode or the native terminal application (see image below), but as soon as I try to run it from any 3rd party application, it doesn't ask for permissions to record sound, and the resultant file ends up soundless. When archiving it and then running it from other 3rd party applications, e.g Warp (terminal) or spawning it as a child process from a bundled Electron application, it doesn't ask for permissions. Things of note: I've codesigned the application with "Developer ID Application" I've added NSAudioCaptureUsageDescriptionto Info.plist I've included Info.plist in the binary (see image below) I've added the com.apple.security.device.audio-input entitlement I've used the following resources as inspiration: https://github.com/insidegui/AudioCap https://developer.apple.com/documentation/coreaudio/capturing-system-audio-with-core-audio-taps As my use-case involves spawning the executable from Electron as a child process, I've tried to include the appropriate permissions to the parent application too, without success. I'm really at a loss here, it feels like I've tried everything. Any pointers are much appreciated! Thanks