Since iOS 18, the system setting “Allow Audio Playback” (enabled by default) allows third-party app audio to continue playing while the user is recording video with the Camera app. This has created a problem for the app I’m developing. ➡️ The problem: My app plays continuous audio in both foreground and background states. If the user starts recording video using the iOS Camera app, the app’s audio — still playing in the background — gets captured in the video — obviously an unintended behavior. Yes, the user could stop the app manually before starting the video recording, but that can’t be guaranteed. As a developer, I need a way to stop the app’s audio before the video recording begins. So far, I haven’t found a reliable way to detect when video recording starts if ‘Allow Audio Playback’ is ON. ➡️ What I’ve tried: — AVAudioSession.interruptionNotification → doesn’t fire — devicesChangedEventStream → not triggered I don’t want to request mic permission (app doesn’t use mic). also, disabling the app from playing audio in the background isn’t an option as it is a crucial part of the user experience ➡️ What I need: A reliable, supported way to detect when the Camera app begins video recording, without requiring mic access — so I can stop audio and avoid unintentional overlap with the user’s recordings. Any official guidance, workarounds, or AVFoundation techniques would be greatly appreciated. Thanks.

AVAudioFormat has no Swift concurrency annotations but the documentation states "Instances of this class are immutable." This made me always assume it was safe to pass AVAudioFormat instances around. Is this the case? If so can it be marked as Sendable? Am I missing something?

When using AVSampleBufferDisplayLayer to play uncompressed H.264 and H.265 video with B-frames more than 7, frame drops occur. The more B-frames there are, the more noticeable the frame drops become, for example 15 bframes. Use FFmpeg to transcode a video file with visible timestamps and frame numbers (x264 or x265 ): ffmpeg -i test.mp4 -vf "drawtext=fontsize=45:text=%{pts} %{n}:y=400" -c:v libx264 -x264-params "bframes=15:b-adapt=0" -crf 30 -y x264_bf15.mp4 ffmpeg -i test.mp4 -vf "drawtext=fontsize=45:text=%{pts} %{n}:y=400" -c:v libx265 -x265-params "bframes=15:b-adapt=0" -crf 30 -y x265_bf15.mp4 Use the demo player from this repository to reproduce the issue: https://github.com/msfrms/CustomPlayer frame drops can be observed. And following log can be found in devices console. mediaserverd <<<< IQ-CA >>>> piqca_gmstats_dump: FIQCA(0x1266f4000) recent frames: enqueued: 184, displayed: 138, dropped: 42, flushed: 0, evicted: 3, >16ms late: 2 PS. I was using iphone11 iOS14.6, to replay this issue. May I ask why frame drops occur in this case? Is there any configuration or API usage change that could help fix the frame drop issue? Many thanks!

I am creating a barcode reader using the AVfoundation framework for iOS and IPadOS. The read result goes into payloadstringvalue, but I want to check the control characters contained in the symbol, so I am using the raw data of the description, which is a property of NSObjectProtocol inherited by VNBarcodeObservation. However, I noticed that if the length set in the raw data exceeds 26, some of the raw data in the description is omitted. So my question is, is it possible to set it so that all the raw data in the description is written out without omitting any raw data? If so, could you please tell me how to set this up? Also, if you know of any other way to extract the raw barcode data, I would appreciate it if you could let me know. Thank you.

Recurring crash on install of any app with the new sourceVideoTrackProvider.next() dyld[41966]: Symbol not found: _$sSo19AVAssetReaderOutputC12AVFoundationE8ProviderC4nextxSgyYaKFTjTu Referenced from: <79AA2BE0-A6B4-32F5-A804-E84BBE5D1AEA> /Users/<username>/Library/Developer/Xcode/DerivedData/TrackProviderCrash-bbbhjptcxnmfdcackxtpucnunxyc/Build/Products/Debug-maccatalyst/TrackProviderCrash.app/Contents/MacOS/TrackProviderCrash.debug.dylib Expected in: <1B847AF9-7973-3B28-95C2-09E73F6DD50B> /usr/lib/swift/libswiftAVFoundation.dylib Can be reproduced with the current Xcode Beta 4 by running on to MacCatalyst and macOS https://developer.apple.com/documentation/AVFoundation/converting-projected-video-to-apple-projected-media-profile Crash goes away of you comment out lines 154-158 and 164-170 which are while let sampleBuffer = try await sourceVideoTrackProvider.next(){/*other code*/} Can also be reproduced if you add the code below to a MacCatalyst project import AVKit let asset: AVURLAsset = .init(url: Bundle.main.url(forResource: "SomeVideo.mp4", withExtension: nil)!) let videoReader = try! AVAssetReader(asset: asset) let videoTracks = try! await asset.loadTracks(withMediaCharacteristic: .visual) // Get the side-by-side video track. let videoTrack = videoTracks.first! let videoInputTrack = AVAssetReaderTrackOutput(track: videoTrack, outputSettings: nil) let sourceVideoTrackProvider: AVAssetReaderOutput.Provider<CMReadySampleBuffer<CMSampleBuffer.DynamicContent>> = videoReader.outputProvider(for: videoInputTrack) //Comment out this while let sb = try! await sourceVideoTrackProvider.next() { }

i'm trying to work on a simple screen recording app on macOS that always records the last 'x' seconds of your screen and saves it whenever you want, as a way to get comfortable with swift programming and apple APIs. i was able to get it running for the past '30 seconds' and record and store it. however i realised that there was a core issue with my solution: i was defining the SCStreamConfiguration.queueDepth = 900 (to account for 30fps for 30 seconds) which goes completely against apple's instructions: https://developer.apple.com/documentation/screencapturekit/scstreamconfiguration/queuedepth?language=objc now when i changed queueDepth back to 8, i am only able to record 8 frames and it saves only those first 8 frames. i am unsure what the flow of the apis should be while dealing with screenCaptureKit. for context, here's my recording manager code that handles this logic (queueDepth = 900) import Foundation import ScreenCaptureKit import AVFoundation class RecordingManager: NSObject, ObservableObject, SCStreamDelegate { static let shared = RecordingManager() @Published var isRecording = false private var isStreamActive = false // Custom state flag private var stream: SCStream? private var streamOutputQueue = DispatchQueue(label: "com.clipback.StreamOutput", qos: .userInteractive) private var screenStreamOutput: ScreenStreamOutput? // Strong reference to output private var lastDisplayID: CGDirectDisplayID? private let displayCheckQueue = DispatchQueue(label: "com.clipback.DisplayCheck", qos: .background) // In-memory rolling buffer for last 30 seconds private var rollingFrameBuffer: [(CMSampleBuffer, CMTime)] = [] private let rollingFrameBufferQueue = DispatchQueue(label: "com.clipback.RollingBuffer", qos: .userInteractive) private let rollingBufferDuration: TimeInterval = 30.0 // seconds // Track frame statistics private var frameCount: Int = 0 private var lastReportTime: Date = Date() // Monitor for display availability private var displayCheckTimer: Timer? private var isWaitingForDisplay = false func startRecording() { print("[DEBUG] startRecording called.") guard !isRecording && !isWaitingForDisplay else { print("[DEBUG] Already recording or waiting, ignoring startRecording call") return } isWaitingForDisplay = true isStreamActive = true // Set active state checkForDisplay() } private func setupAndStartRecording(for display: SCDisplay, excluding appToExclude: SCRunningApplication?) { print("[DEBUG] setupAndStartRecording called for display: \(display.displayID)") let excludedApps = [appToExclude].compactMap { $0 } let filter = SCContentFilter(display: display, excludingApplications: excludedApps, exceptingWindows: []) let config = SCStreamConfiguration() config.width = display.width config.height = display.height config.minimumFrameInterval = CMTime(value: 1, timescale: 30) // 30 FPS config.queueDepth = 900 config.showsCursor = true print("[DEBUG] SCStreamConfiguration created: width=\(config.width), height=\(config.height), FPS=\(config.minimumFrameInterval.timescale)") stream = SCStream(filter: filter, configuration: config, delegate: self) print("[DEBUG] SCStream initialized.") self.screenStreamOutput = ScreenStreamOutput { [weak self] sampleBuffer, outputType in guard let self = self else { return } guard outputType == .screen else { return } guard sampleBuffer.isValid else { return } guard let attachments = CMSampleBufferGetSampleAttachmentsArray(sampleBuffer, createIfNecessary: false) as? [[SCStreamFrameInfo: Any]], let statusRawValue = attachments.first?[.status] as? Int, let status = SCFrameStatus(rawValue: statusRawValue), status == .complete else { return } self.trackFrameRate() self.handleFrame(sampleBuffer) } do { try stream?.addStreamOutput(screenStreamOutput!, type: .screen, sampleHandlerQueue: streamOutputQueue) stream?.startCapture { [weak self] error in print("[DEBUG] SCStream.startCapture completion handler.") guard error == nil else { print("[DEBUG] Failed to start capture: \(error!.localizedDescription)") self?.handleStreamError(error!) return } DispatchQueue.main.async { self?.isRecording = true self?.isStreamActive = true // Update state on successful start print("[DEBUG] Recording started. isRecording = true.") } } } catch { print("[DEBUG] Error adding stream output: \(error.localizedDescription)") handleStreamError(error) } } private func handleFrame(_ sampleBuffer: CMSampleBuffer) { rollingFrameBufferQueue.async { [weak self] in guard let self = self else { return } let pts = CMSampleBufferGetPresentationTimeStamp(sampleBuffer) var retainedBuffer: CMSampleBuffer? CMSampleBufferCreateCopy(allocator: kCFAllocatorDefault, sampleBuffer: sampleBuffer, sampleBufferOut: &retainedBuffer) guard let buffer = retainedBuffer else { print("[DEBUG] Failed to copy sample buffer") return } self.rollingFrameBuffer.append((buffer, pts)) if let lastPTS = self.rollingFrameBuffer.last?.1 { while let firstPTS = self.rollingFrameBuffer.first?.1, CMTimeGetSeconds(CMTimeSubtract(lastPTS, firstPTS)) > self.rollingBufferDuration { self.rollingFrameBuffer.removeFirst() } } } } func stream(_ stream: SCStream, didStopWithError error: Error) { print("[DEBUG] Stream stopped with error: \(error.localizedDescription)") displayCheckQueue.async { [weak self] in // Move to displayCheckQueue for synchronization self?.handleStreamError(error) } } what could be the reason for this and what would be the possible fix logically? i dont understand why it's dependant on queueDepth, and if it is, how can I empty and append new recorded frames to it so that it continues working? any help or resource is greatly appreciated!

I developed an educational app that implements audio-video communication through RTC, while using WebView to display course materials during classes. However, some users are experiencing an issue where the audio playback from WebView is very quiet. I've checked that the AVAudioSessionCategory is set by RTC to AVAudioSessionCategoryPlayAndRecord, and the AVAudioSessionCategoryOption also includes AVAudioSessionCategoryOptionMixWithOthers. What could be causing the WebView audio to be suppressed, and how can this be resolved?

Hi all, I'm working on an audio visualizer app that plays files from the user's music library utilizing MediaPlayer and AVAudioEngine. I'm working on getting the music library functionality working before the visualizer aspect. After setting up the engine for file playback, my app inexplicably crashes with an EXC_BREAKPOINT with code = 1. Usually this means I'm unwrapping a nil value, but I think I'm handling the optionals correctly with guard statements. I'm not able to pinpoint where it's crashing. I think it's either in the play function or the setupAudioEngine function. I removed the processAudioBuffer function and my code still crashes the same way, so it's not that. The device that I'm testing this on is running iOS 26 beta 3, although my app is designed for iOS 18 and above. After commenting out code, it seems that the app crashes at the scheduleFile call in the play function, but I'm not fully sure. Here is the setupAudioEngine function: private func setupAudioEngine() { do { try AVAudioSession.sharedInstance().setCategory(.playback, mode: .default) try AVAudioSession.sharedInstance().setActive(true) } catch { print("Audio session error: \(error)") } engine.attach(playerNode) engine.attach(analyzer) engine.connect(playerNode, to: analyzer, format: nil) engine.connect(analyzer, to: engine.mainMixerNode, format: nil) analyzer.installTap(onBus: 0, bufferSize: 1024, format: nil) { [weak self] buffer, _ in self?.processAudioBuffer(buffer) } } Here is the play function: func play(_ mediaItem: MPMediaItem) { guard let assetURL = mediaItem.assetURL else { print("No asset URL for media item") return } stop() do { audioFile = try AVAudioFile(forReading: assetURL) guard let audioFile else { print("Failed to create audio file") return } duration = Double(audioFile.length) / audioFile.fileFormat.sampleRate if !engine.isRunning { try engine.start() } playerNode.scheduleFile(audioFile, at: nil) playerNode.play() DispatchQueue.main.async { [weak self] in self?.isPlaying = true self?.startDisplayLink() } } catch { print("Error playing audio: \(error)") DispatchQueue.main.async { [weak self] in self?.isPlaying = false self?.stopDisplayLink() } } } Here is a link to my test project if you want to try it out for yourself: https://github.com/aabagdi/VisualMan-example Thanks!

We found that on some iPad devices (such as iPad8,9/iPad OS/18.5), when the media is playing, the recorded human voice is very small and is basically suppressed by the sound of the playing media. When the media is not playing, the human voice can be recorded. Likely when use screen recording with microphone, the ambient sound is also basically suppressed by the media sound.

Hi all, I'm capturing a photo using AVCapturePhotoOutput, and I've set: let photoSettings = AVCapturePhotoSettings() photoSettings.isDepthDataDeliveryEnabled = true Then I create the handler like this: let data = photo.fileDataRepresentation() let handler = try ImageRequestHandler(data: data, orientation: .right) Now I’m wondering: If depth data delivery is enabled, is it actually included and used when I pass the Data to ImageRequestHandler? Or do I need to explicitly pass the depth data using the other initializer? let handler = try ImageRequestHandler( cvPixelBuffer: photo.pixelBuffer!, depthData: photo.depthData, orientation: .right ) In short: Does ImageRequestHandler(data:) make use of embedded depth info from AVCapturePhoto.fileDataRepresentation() — or is the pixel buffer + explicit depth data required? Thanks for any clarification!

I bought two "Apple USB-C to Headphone Jack Adapters". Upon closer inspection, they seems to be of different generations: The one with product ID 0x110a on top is working fine. The one with product ID 0x110b has two issues: There is a short but loud click noise on the headphone when I connect it to the iPad. When I play audio using AVAudioPlayer the first half of a second or so is cut off. Here's how I'm playing the audio: audioPlayer = try AVAudioPlayer(contentsOf: url) audioPlayer?.delegate = self audioPlayer?.prepareToPlay() audioPlayer?.play() Is this a known issue? Am I doing something wrong?

I'm streaming mp3 audio data using URLSession/AudioFileStream/AVAudioConverter and getting occasional silent buffers and glitches (little bleeps and whoops as opposed to clicks). The issues are present in an offline test, so this isn't an issue of underruns. Doing some buffering on the input coming from the URLSession (URLSessionDataTask) reduces the glitches/silent buffers to rather infrequent, but they do still happen occasionally. var bufferedData = Data() func parseBytes(data: Data) { bufferedData.append(data) // XXX: this buffering reduces glitching // to rather infrequent. But why? if bufferedData.count > 32768 { bufferedData.withUnsafeBytes { (bytes: UnsafeRawBufferPointer) in guard let baseAddress = bytes.baseAddress else { return } let result = AudioFileStreamParseBytes(audioStream!, UInt32(bufferedData.count), baseAddress, []) if result != noErr { print("❌ error parsing stream: \(result)") } } bufferedData = Data() } } No errors are returned by AudioFileStream or AVAudioConverter. func handlePackets(data: Data, packetDescriptions: [AudioStreamPacketDescription]) { guard let audioConverter else { return } var maxPacketSize: UInt32 = 0 for packetDescription in packetDescriptions { maxPacketSize = max(maxPacketSize, packetDescription.mDataByteSize) if packetDescription.mDataByteSize == 0 { print("EMPTY PACKET") } if Int(packetDescription.mStartOffset) + Int(packetDescription.mDataByteSize) > data.count { print("❌ Invalid packet: offset \(packetDescription.mStartOffset) + size \(packetDescription.mDataByteSize) > data.count \(data.count)") } } let bufferIn = AVAudioCompressedBuffer(format: inFormat!, packetCapacity: AVAudioPacketCount(packetDescriptions.count), maximumPacketSize: Int(maxPacketSize)) bufferIn.byteLength = UInt32(data.count) for i in 0 ..< Int(packetDescriptions.count) { bufferIn.packetDescriptions![i] = packetDescriptions[i] } bufferIn.packetCount = AVAudioPacketCount(packetDescriptions.count) _ = data.withUnsafeBytes { ptr in memcpy(bufferIn.data, ptr.baseAddress, data.count) } if verbose { print("handlePackets: \(data.count) bytes") } // Setup input provider closure var inputProvided = false let inputBlock: AVAudioConverterInputBlock = { packetCount, statusPtr in if !inputProvided { inputProvided = true statusPtr.pointee = .haveData return bufferIn } else { statusPtr.pointee = .noDataNow return nil } } // Loop until converter runs dry or is done while true { let bufferOut = AVAudioPCMBuffer(pcmFormat: outFormat, frameCapacity: 4096)! bufferOut.frameLength = 0 var error: NSError? let status = audioConverter.convert(to: bufferOut, error: &error, withInputFrom: inputBlock) switch status { case .haveData: if verbose { print("✅ convert returned haveData: \(bufferOut.frameLength) frames") } if bufferOut.frameLength > 0 { if bufferOut.isSilent { print("(haveData) SILENT BUFFER at frame \(totalFrames), pending: \(pendingFrames), inputPackets=\(bufferIn.packetCount), outputFrames=\(bufferOut.frameLength)") } outBuffers.append(bufferOut) totalFrames += Int(bufferOut.frameLength) } case .inputRanDry: if verbose { print("🔁 convert returned inputRanDry: \(bufferOut.frameLength) frames") } if bufferOut.frameLength > 0 { if bufferOut.isSilent { print("(inputRanDry) SILENT BUFFER at frame \(totalFrames), pending: \(pendingFrames), inputPackets=\(bufferIn.packetCount), outputFrames=\(bufferOut.frameLength)") } outBuffers.append(bufferOut) totalFrames += Int(bufferOut.frameLength) } return // wait for next handlePackets case .endOfStream: if verbose { print("✅ convert returned endOfStream") } return case .error: if verbose { print("❌ convert returned error") } if let error = error { print("error converting: \(error.localizedDescription)") } return @unknown default: fatalError() } } }

it will use about 300MB memory, it cause a memory peak

AVCaptureSession's startRunning method is thread blocking and seems to be slow. What is this method doing behind the scenes? For context: I'm working on Simulator Camera support and I have a 'fake' AVCaptureDevice that might be causing this. My hypothesis is that AVCaptureSession tries to connect to the device and waits for a notification to be posted back. I'd love to find a way to let my fake device message AVCaptureSession that it's connected.

AVCaptureVideoDataOutput.preparesCellularRadioForNetworkConnection requires com.apple.developer.avfoundation.video-data-output-prepares-cellular-radio-for-machine-readable-code-scanning. But I cannot acquire its entitlement. I can't find its entitlement on 'Certificates, Identifiers &amp; Profiles'. Any solutions? Provisioning profile "iOS Team Provisioning Profile: ......" doesn't include the com.apple.developer.avfoundation.video-data-output-prepares-cellular-radio-for-machine-readable-code-scanning entitlement.

Hi Apple Developer Team, I'm encountering a regression in iOS 18 related to the Picture-in-Picture (PiP) feature when using custom views. In previous versions of iOS (up to iOS 17), it's possible to show a custom UIView inside the PiP window — for example, a UILabel, UITableView, or other standard UI elements. This works well even when switching between apps. However, in iOS 18 (tested on the developer beta), there's an issue: If App A starts PiP mode and displays a custom view, and then the user switches to App B and starts a video call (e.g., using FaceTime or another VoIP app), all the custom views in the PiP window suddenly disappear. The PiP window itself remains, but its contents are empty. This behavior did not occur in earlier iOS versions. Steps to reproduce: In App A, start Picture-in-Picture with a custom UIView added to the PiP window. Switch to App B and initiate a video call (e.g., FaceTime). Observe the PiP window — the custom view is no longer visible. This issue breaks UI functionality that previously worked and may impact apps that rely on interactive or dynamic content in PiP. Is this a known issue in iOS 18, or is this behavior change intentional? Any suggested workarounds or updates? Thanks in advance for your support.

During the Camera lab session at WWDC25, support for DICOM was mentioned. I'm trying to work with the new AVFileType dcm to help with handling DICOM scans. Can anyone provide some assistance?

So I've spent the last five years optimizing my video AI system so that it runs with less than 5% CPU while processing a 30fps video feed on a Macbook Pro M2, and everything is great, until Sonoma comes out, and I find myself consuming 40% CPU for the exact same workload. So I fire up Instruments, and the "heaviest stack trace" (see screenshot) turns out to be Espresso doing some completely unasked-for and absolutely useless processing on my video frames. I turn off Reactions, but nothing helps - the CPU consumptions stays at 40%. "Reactions" is nothing but a useless toy to please some WWDC keynote fanboys, I don't want it anywhere near my app or my users, and I especially do not want to take the blame for it pissing away the user's CPU cycles and battery. Now, how do I make it go away, for ever? Best regards Jacob

I'm developing iPad app that will be mostly dedicated for certain external camera for visually impaired people. The linux UVC api (e.g. using guvcview) allows to enable automatic exposure for the camera. IOs api "isExposureModeSupported" unfortunately returns false for any of the exposure modes. Is it a bug? Or perhaps AVFoundation doesn't support UVC exposure yet?

When I play an HDR video in the iPhone Photos app, I can see the HDR effect obviously. But if this HDR video is played continuously for more than 30-40 minutes, the HDR effect will disappear and the brightness will be compressed to the SDR range. This issue will appear on any iPhone. Depending on the phone, it may be 20-30 minutes, or 30-40 minutes, or even a few minutes, such as iPhone 12 mini. Similarly, if I use AVPlayer to play and preview an HDR video, if it plays more than 30-40 minutes, the HDR effect will disappear and the screen brightness will dim. Also the currentEDRHeadroom will gradually decrease to 1 Note, test it with an HDR video longer than 1 hour, and if the video is short, please loop it. My question is how to avoid losing the HDR effect after 30-40 minutes when I use CAMetalLayer to render any HDR video.