AVFoundation

RSS for tag

Work with audiovisual assets, control device cameras, process audio, and configure system audio interactions using AVFoundation.

Posts under AVFoundation tag

200 Posts

Post

Replies

Boosts

Views

Activity

CarPlay: Is vehicle microphone capture without entering communications audio mode supported for third-party apps?
Hello, I am developing a native CarPlay application and would appreciate some clarification regarding AVAudioSession behaviour when using the vehicle microphone. The application is intentionally simple and designed to minimise driver distraction. The user presses a single button on the CarPlay screen, the application performs a brief (approximately five second) audio capture, performs application-specific processing on the captured audio, displays a simple confirmation to the user, and immediately releases the audio session. The goal is to allow the driver to continue their journey with as little interaction as possible. Because the audio being captured is often originating from the vehicle’s native DAB/FM radio rather than from the iPhone itself, preserving the existing listening experience during the brief capture is fundamental to the intended design. To better understand what is supported by CarPlay, I have carried out a number of controlled experiments. Test Environment Native CarPlay application Swift AVFoundation AVAudioSession AVAudioRecorder Vehicle connected via CarPlay Vehicle playing its native DAB radio Experiment 1 Configuration Category: AVAudioSession.Category.playAndRecord Mode: AVAudioSession.Mode.default Observed Route Input: CarPlay — CarAudio Output: CarPlay — CarAudio Result Recording succeeds using the vehicle microphone. Native DAB audio is muted during recording. The vehicle changes from “Audio Volume” to “Voice Volume”. When recording finishes and the AVAudioSession is deactivated, DAB resumes normally. Experiment 2 Changed only the session mode to: AVAudioSession.Mode.measurement Result Behaviour was identical to Experiment 1. Experiment 3 Changed only the session mode to: AVAudioSession.Mode.videoRecording Observed Route Input: iPhone microphone Output: CarPlay — CarAudio Result Input automatically switched from the vehicle microphone to the iPhone microphone. The vehicle remained in normal “Audio Volume”. However, the head unit switched away from its native DAB source to the CarPlay media source. Observation These experiments appear to suggest that the behaviour is specifically related to using the CarPlay “CarAudio” microphone route, rather than audio recording itself. Question Is this the expected behaviour for third-party CarPlay applications? More specifically: Is there any supported AVAudioSession configuration or CarPlay API that allows a third-party CarPlay application to perform a brief, user-initiated audio capture using the vehicle microphone without causing the head unit to enter its communications/voice audio mode or interrupt the vehicle’s native radio playback? If this behaviour is by design and no supported mechanism exists to achieve this, confirmation of that would be extremely valuable, as it would allow us to design the application accordingly. If additional information, sample code, AVAudioSession logs or detailed reproduction steps would be helpful, I would be more than happy to provide them. Thank you very much for your time. Kind regards, Neil Jenner Developer, HearSave
1
0
64
17h
ProResRAW shooting issue with AVCaptureMovieFileOutput with the first video
On iOS 26, the first ProRes RAW recording after launching the app consistently stalls for the entire take: audio records normally, but video contains only a few frames (for example, an 11.8-second clip at approximately 0.25 fps instead of 30 fps). Every subsequent recording works correctly. Before startRecording, the Bayer format is active, ProRes RAW is available, white balance is locked, and frame duration is pinned to 1/30. Reordering setOutputSettings and reasserting configuration have not resolved it; only discarding the first recording acts as a reliable warm-up. Is this a known one-time ProRes RAW encoder initialisation issue, and is there a supported way to prepare the encoder before the first user takes? I have a full working sample code if anyone needs but here is the setup function: import CoreMedia import CoreVideo @available(iOS 26.0, *) final class ProResRAWRecorder: NSObject, AVCaptureFileOutputRecordingDelegate { let session = AVCaptureSession() private let movieOutput = AVCaptureMovieFileOutput() private var camera: AVCaptureDevice! // Call on a serial capture queue. func configure() throws { session.beginConfiguration() defer { session.commitConfiguration() } session.sessionPreset = .inputPriority session.automaticallyConfiguresCaptureDeviceForWideColor = false guard let device = AVCaptureDevice.default( .builtInWideAngleCamera, for: .video, position: .back ) else { throw SampleError.noCamera } let cameraInput = try AVCaptureDeviceInput(device: device) guard session.canAddInput(cameraInput) else { throw SampleError.cannotAddInput } session.addInput(cameraInput) camera = device // Optional, but reproduces the real topology where audio remains healthy. if let microphone = AVCaptureDevice.default(for: .audio), let microphoneInput = try? AVCaptureDeviceInput(device: microphone), session.canAddInput(microphoneInput) { session.addInput(microphoneInput) } // Find a 12-bit packed Bayer format supporting 30 fps. guard let rawFormat = device.formats .filter({ CMFormatDescriptionGetMediaSubType($0.formatDescription) == kCVPixelFormatType_96VersatileBayerPacked12 && $0.videoSupportedFrameRateRanges.contains { $0.minFrameRate <= 30 && $0.maxFrameRate >= 30 } }) .max(by: { let a = CMVideoFormatDescriptionGetDimensions($0.formatDescription) let b = CMVideoFormatDescriptionGetDimensions($1.formatDescription) return Int(a.width) * Int(a.height) < Int(b.width) * Int(b.height) }) else { throw SampleError.noRAWFormat } try device.lockForConfiguration() device.activeFormat = rawFormat if rawFormat.supportedColorSpaces.contains(.appleLog) { device.activeColorSpace = .appleLog } let frameDuration = CMTime(value: 1, timescale: 30) device.activeVideoMinFrameDuration = frameDuration device.activeVideoMaxFrameDuration = frameDuration if device.isWhiteBalanceModeSupported(.locked) { device.whiteBalanceMode = .locked } device.unlockForConfiguration() guard session.canAddOutput(movieOutput) else { throw SampleError.cannotAddOutput } session.addOutput(movieOutput) session.startRunning() } // Call on the same serial capture queue after startRunning() returns. func record(to url: URL) throws { guard let connection = movieOutput.connection(with: .video) else { throw SampleError.noVideoConnection } // Reassert the required device state at the take boundary. try camera.lockForConfiguration() if camera.isWhiteBalanceModeSupported(.locked) { camera.whiteBalanceMode = .locked } let frameDuration = CMTime(value: 1, timescale: 30) camera.activeVideoMinFrameDuration = frameDuration camera.activeVideoMaxFrameDuration = frameDuration camera.unlockForConfiguration() guard movieOutput.availableVideoCodecTypes.contains(.proResRAW) else { throw SampleError.rawCodecUnavailable } movieOutput.setOutputSettings( [AVVideoCodecKey: AVVideoCodecType.proResRAW], for: connection ) movieOutput.startRecording(to: url, recordingDelegate: self) } func stop() { if movieOutput.isRecording { movieOutput.stopRecording() } } func fileOutput( _ output: AVCaptureFileOutput, didFinishRecordingTo outputFileURL: URL, from connections: [AVCaptureConnection], error: Error? ) { print("Finished:", outputFileURL, "error:", error as Any) } enum SampleError: Error { case noCamera case cannotAddInput case noRAWFormat case cannotAddOutput case noVideoConnection case rawCodecUnavailable } }
1
0
311
19h
Manual legible (subtitle) selection not honored on live LL-HLS — select(_:in:) reverts to “off” within ~2s; automatic selection never displays non-forced subtitles
Environment: iOS 18 / iOS 26, AVPlayer + AVPlayerItem, live low-latency HLS (LL-HLS). Subtitle renditions are regular (non-forced) WebVTT: AUTOSELECT=YES, FORCED=NO. System captioning (Closed Captions + SDH) is OFF (default). Direct CDN, no P2P. Master playlist (subtitle part): #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Korean (AI)",AUTOSELECT=YES,FORCED=NO,LANGUAGE="kor",URI="..." #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English (AI)",AUTOSELECT=YES,FORCED=NO,LANGUAGE="eng",URI="..." #EXT-X-STREAM-INF:...,SUBTITLES="subs" Problem When the user picks a subtitle language we call playerItem.select(option, in: legibleGroup) with an option that is a genuine member of the item's current legible group. Right after the call, item.currentMediaSelection.selectedMediaOption(in:) returns the requested option. But within ~2 seconds, on a subset of real-user sessions, the selection spontaneously reverts with no further app interaction: most often to Off (no legible output is delivered afterwards), or it stays stuck on the previously selected language (a subsequent select to another language, or to Off, is silently ignored). It is intermittent and only appears under real conditions (frequent live playlist reloads / reconnects); it does not reproduce in a short, clean session. What we verified The option passed to select(_:in:) is a real member of the current group (re-resolved from item.asset); the call returns without error and reads back correctly immediately after. appliesMediaSelectionCriteriaAutomatically is the default true. Per the AV Foundation Release Notes ("Advice about subtitles"), automatic selection excludes options that are not AVMediaCharacteristicContainsOnlyForcedSubtitles. So on each live reload the automatic re-selection resolves non-forced subtitles to Off, appearing to override the manual select(_:in:). Reading the selected option against a freshly obtained group instance returns the same value as against the original instance — so this is a real state change, not a mismatched-group read. Questions On live HLS reloads with appliesMediaSelectionCriteriaAutomatically == true, is manual select(_:in:) for .legible expected to be overridden by automatic media selection? If so, is setMediaSelectionCriteria(_:forMediaCharacteristic:) the intended way to persist a user's choice? setMediaSelectionCriteria for subtitles is itself reported as unreliable (sometimes no subtitles) — see thread 108403. What is the recommended, deterministic way to keep a user-selected non-forced subtitle displayed across live playlist reloads, including turning subtitles Off? Is this the same underlying behavior as FB13344652 ("Auto (Recommended) doesn't display subtitles despite language match / DEFAULT=YES")? Related: https://developer.apple.com/forums/thread/722752 (FB13344652) , https://developer.apple.com/forums/thread/108403
1
0
98
3d
iOS 27 beta: Opening Notification Center pauses AVPlayer playback
Since iOS 27 beta we are seeing a behavior change in our video streaming app and I would like to know whether others can reproduce it. Behavior on iOS 27 beta: Fully opening the Notification Center pauses playback. Audio continues for about 5 more seconds, then the app is suspended. Closing the Notification Center leaves the player paused. On iOS 26 the same build keeps playing in this situation. Setup: AVQueuePlayer playing video with audio, not muted audiovisualBackgroundPlaybackPolicy = .automatic (default) AVAudioSession category .playback, UIBackgroundModes: audio entitlement AVPictureInPictureController attached with canStartPictureInPictureAutomaticallyFromInline = true Filed as FB23893965 Questions: Can anyone reproduce this on iOS 27 beta? Is this intentional or a regression?
0
2
84
4d
Add a value to the Photos Caption field
In the iOS Photos app there is a caption field the user can write to. How can you write to this value from Swift when creating a photo? I see apps that do this, but there doesn't seem to be any official way to do this using the Photo library through PHAssetCreationRequest or PHAssetResourceCreationOptions or setting EXIF values, I tried settings a bunch of values there including IPTC values but nothing appears in the caption field in the iOS photos app. There must be some way to do it since I see other apps setting that value somehow after capturing a photo.
2
1
319
1w
AVRoutePickerView's presented route list is automatically dismissed when CXProvider reports the call as connected (`reportOutgoingCall(with:connectedAt:)`)
Hi, I want my user to be able to change their audio output device while calling someone, even before the call is established. But I've run into an issue combining CallKit and AVRoutePickerView. On iOS 26 (Xcode 26.5, tested on iPhone 13 Mini): when my AVRoutePickerView's route list is presented while my call is still connecting (not yet established), the presented list is automatically dismissed the moment the app reports the call as connected via CXProvider.reportOutgoingCall(with:connectedAt:). That doesn't seem natural to me — it should stay open, since the user is actively interacting with unrelated system UI at that moment. This happens even when nothing about the AVAudioSession itself changes: no category change, no route change, no didActivate/didDeactivate callback fires around that time. The dismissal is caused by the CallKit "connected" state transition itself, but I don't know why, or whether it's intentional. Steps to reproduce (sample project below): Tap "Start Fake Call". Within 5 seconds, tap the small route picker button and leave its list open. Wait — at t+5s the app reports the call connected via reportOutgoingCall(connectedAt:), and the list closes itself right after. Is this normal? And is there an official, CallKit-sanctioned way to report a call's connected state without this side effect — some way to keep the route picker stable across that transition? The two workarounds I've found both have real downsides: Calling reportOutgoingCall(connectedAt:) at the very start of the call, before the callee has actually answered , but then CallKit no longer reflects the real call state (Recents/duration would include ringing time, and calls that are never answered would still show as "connected"). Disabling audio device selection entirely until the call is established... works, but hurts the user experience, since users may want to switch output before the call connects. Here's the minimal reproduction: import CallKit import SwiftUI struct ContentView: View { @StateObject private var demo = CallKitDemo() var body: some View { VStack(spacing: 24) { Text(demo.statusText) .font(.system(.body, design: .monospaced)) .multilineTextAlignment(.center) Button("Start Fake Call") { demo.startFakeCall() } .disabled(demo.isCallInProgress) RoutePickerView() .frame(width: 60, height: 60) } .padding() } } private struct RoutePickerView: UIViewRepresentable { func makeUIView(context: Context) -> AVRoutePickerView { let picker = AVRoutePickerView() picker.delegate = context.coordinator return picker } func updateUIView(_ uiView: AVRoutePickerView, context: Context) {} func makeCoordinator() -> Coordinator { Coordinator() } final class Coordinator: NSObject, AVRoutePickerViewDelegate { func routePickerViewWillBeginPresentingRoutes(_ routePickerView: AVRoutePickerView) { print("[REPRO]", Date(), "picker WILL present routes") } func routePickerViewDidEndPresentingRoutes(_ routePickerView: AVRoutePickerView) { print("[REPRO]", Date(), "picker DID end presenting routes <- dismissed here") } } } @MainActor final class CallKitDemo: NSObject, ObservableObject { @Published var statusText = "Idle" @Published var isCallInProgress = false private let provider: CXProvider = { let configuration = CXProviderConfiguration() configuration.supportsVideo = false configuration.maximumCallGroups = 1 configuration.maximumCallsPerCallGroup = 1 configuration.supportedHandleTypes = [.generic] return CXProvider(configuration: configuration) }() private let callController = CXCallController() private var currentCallUUID: UUID? override init() { super.init() provider.setDelegate(self, queue: nil) } func startFakeCall() { let uuid = UUID() currentCallUUID = uuid isCallInProgress = true statusText = "Requesting CXStartCallAction..." let handle = CXHandle(type: .generic, value: "repro-call") let startAction = CXStartCallAction(call: uuid, handle: handle) callController.request(CXTransaction(action: startAction)) { error in if let error { print("[REPRO]", Date(), "CXStartCallAction request failed:", error) } } DispatchQueue.main.asyncAfter(deadline: .now() + 5) { [weak self] in guard let self, currentCallUUID == uuid else { return } print("[REPRO]", Date(), "reportOutgoingCall(connectedAt:)") statusText = "Reported connected at t+5s" provider.reportOutgoingCall(with: uuid, connectedAt: Date()) } DispatchQueue.main.asyncAfter(deadline: .now() + 8) { [weak self] in guard let self, currentCallUUID == uuid else { return } callController.request(CXTransaction(action: CXEndCallAction(call: uuid))) { _ in } currentCallUUID = nil isCallInProgress = false statusText = "Call ended" } } } extension CallKitDemo: CXProviderDelegate { func providerDidReset(_ provider: CXProvider) {} func provider(_ provider: CXProvider, perform action: CXStartCallAction) { action.fulfill() } func provider(_ provider: CXProvider, perform action: CXEndCallAction) { action.fulfill() } func provider(_ provider: CXProvider, didActivate audioSession: AVAudioSession) { print("[REPRO]", Date(), "didActivate") } func provider(_ provider: CXProvider, didDeactivate audioSession: AVAudioSession) { print("[REPRO]", Date(), "didDeactivate") } } Thanks for your help !
1
0
171
1w
How should live latency be measured and maintained with AVPlayer (HLS / LL-HLS)?
We keep live playback at a consistent distance from the live edge using small playback rate adjustments, with a target range based on recommendedTimeOffsetFromLive. Since the live edge is not exposed by AVPlayer, we currently fall back to seekableTimeRanges.end as our best approximation. What should be treated as the live edge, and how should the current live latency be measured? Is rate adjustment the appropriate way to hold a target latency? While playing above 1.0x, the playhead can reach the seekable end, at which point AVPlayerItemDidPlayToEndTime fires and halts the live stream. How can we guard against ? Does any of this differ between regular HLS and LL-HLS? A clear statement of the intended contract here would resolve a lot of uncertainty. Thanks in advance.
2
0
289
1w
`LockedCameraCaptureManager` practically unusable since iOS 26
Somewhere since iOS 26, the LockedCameraCapture framework gets in an unpredictable state after opening the main app from the LockedCamera extension using LockedCameraCaptureSession.openApplication(for userActivity:). (Feedback with sample code to reproduce: FB21966835) Opening the extension from the lock screen again doesn’t open the extension but puts the lock screen in a state as if it has. Content updated from LockedCameraCaptureManager.shared.sessionContentUpdates comes in inconsistently, usually needs the app to be opened again or the extension to be opened. This makes using this extension impossible for me as I use it to record video files that manually need to be imported when the app is launched (so not through PhotoKit). Does anybody have a suggestion to circumvent this issue or how to get this fixed?
1
2
612
1w
iOS 26.4 regression: The `.pauses` audiovisual background playback policy does not pause video playback anymore when backgrounding the app
Starting with iOS 26.4 and the iOS 26.4 SDK, the .pauses audiovisual background playback policy is not correctly applied anymore to an AVPlayer having an attached video layer displayed on screen. This means that, when backgrounding a video-playing app (without Picture in Picture support) or locking the device, playback is not paused automatically by the system anymore. This issue affects the Apple TV application as well. We have filed FB22488151 with more information.
2
0
713
2w
AVQueuePlayer unexpectedly performs network requests during offline HLS playback after several queued episode transitions
Hello, We are investigating an issue with offline HLS playback using AVQueuePlayer and would like to know whether anyone else has experienced similar behavior. Issue We download HLS content using AVAssetDownloadURLSession and play it offline using AVQueuePlayer. For some titles (but not all), after several consecutive episode transitions, the player unexpectedly attempts a network request while the next episode is already queued and the current episode has approximately 60 seconds remaining. If the device is offline, playback fails with: NSURLErrorDomain Code = -1009 and the next episode never starts. Characteristics The issue only affects certain titles. It is fully reproducible for affected titles. For example, if it occurs between Episodes 5 and 6 after starting playback from Episode 1, it always occurs at the same point when replaying from Episode 1. If playback starts directly from Episode 5, the issue does not occur. The issue only occurs when using AVQueuePlayer. Replacing the current item (removeAllItems() + replaceCurrentItem(with:)) avoids the issue, although this is unfortunately not a viable workaround because it breaks our Picture in Picture episode transition behavior. We compared the downloaded packages (boot.xml, Master Playlist, and Stream configuration) between affected and unaffected titles, but so far have not identified any meaningful structural differences that explain the behavior. Questions Has anyone experienced similar behavior with: offline HLS (.movpkg) AVQueuePlayer unexpected network requests during queued playback NSURLErrorDomain Code=-1009 even though the content is downloaded for offline playback If anyone has seen a similar issue or has any information, observations, or suggestions for further investigation, I would greatly appreciate hearing from you. For reference, I have already submitted this issue through Feedback Assistant. Feedback ID: FB23487817 Thank you in advance for any information.
2
0
285
2w
AVPlayer Reverse Audio Scrubbing?
Hey all, here seeking some perspective. I have an audio player app on macOS built on top of AVPlayer, I want to add the ability to scrub the audio, and hear the audio frames based on the playhead's position whether going forwards or backwards. When going backwards, the audio frame should be played in reverse as well. The audio tracks live online and are streamed. I tried playing with AVPlayer.rate, but the time pitch algos built in (.spectral, .varispeed, .timeDomain) all only guarantee up to 32x rate decoding accuracy. So technically, if the user scrubs fast enough, the audio rendered would not necessarily match the playhead's position. My current solution that works is to cache the raw audio bytes and play the appropriate frame when the user starts scrubbing. I decode the audio data manually using AudioToolbox's AudioFileOpenWithCallbacks into an AVAudioPCMBuffer, then pass it into AVAudioEngine+AVAudioPlayerNode combo. The problem with that is that means I need to cache this audio data myself (remember this is a stream), and since I don't have access to AVPlayer's own cache I need to also download it myself... which means two downloads for the same track which is less than ideal. This lead me to take it a step further and hijack AVPlayer's download process by implementing AVAssetResourceLoaderDelegate, that way AVPlayer and my audio scrubbing cache are both fed from the same source. Now... I feel like I went down a bit of a rabbit hole here. At the end of the day I simply want accurate audio scrubbing in both directions, while keeping in mind I want the audio snippets to play in reverse when the user goes backwards. Is there really no way to do this that's more "vanilla"? Am I missing something obvious? Genuinely open to any and all suggestions. Thanks.
0
0
228
2w
Custom UI over the Picture in Picture window for AVPlayerInterstitialEvent items — is there an official way?
I attach an ad in front of the main video using AVPlayerInterstitialEvent, and PiP works across the interstitial→content transition. I’ve noticed that some apps, while in Picture in Picture, show additional UI that: displays the remaining time of the interstitial item, and lets the user skip the interstitial item and switch to the main video. As far as I know, the AVPictureInPictureController window is system-drawn and can’t be customized by the developer. So my questions are: Is there an official/supported path to present this kind of UI (a remaining-time indicator and a skip control for the interstitial item) over the PiP window? If so, what is the mechanism — is it a system control that gets forwarded to a delegate, framework-provided interstitial UI, or When the interstitial is skipped this way, does the framework return to the primary item automatically, or does the app drive it? I’d really appreciate any insight intodone. Thanks!
0
0
118
3w
AVCaptureDevice.uniqueID for UVC video devices is unstable - bug or overstated documentation?
The documentation for AVCaptureDevice.uniqueID states the following: Capture devices have a unique identifier that persists on one system across device connections and disconnections, application restarts, and reboots of the system itself. You can store the value returned by this property to recall or track the status of a specific device in the future. For UVC capture devices this documentation does not hold. The video uniqueID is a hex string of the form 0x, and the identifying half is the locationID (bus number plus port path). Which identifies a port, not a device. I ran a suite of tests with three identical Elgato 4K X capture cards connected to a Mac Studio w/ M3 Ultra running macOS 26.5.2, and reproduced my findings on a MacBook w/ M3 Pro (same macOS version). See the script at the bottom of the post for how uniqueId & USB serial number are being retrieved. 1. The uniqueID follows the port. Swapping two cards between two built-in ports swaps their uniqueIDs: # Before swap. 4K X uid=0x2000000fd9009b serial=A7SNB50424UBQI 4K X uid=0x12000000fd9009b serial=A7SNB504219J0R # After swapping the cards between the same two ports. 4K X uid=0x2000000fd9009b serial=A7SNB504219J0R 4K X uid=0x12000000fd9009b serial=A7SNB50424UBQI An app that stored 0x2000000fd9009b to recall a specific capture card now silently opens another. 2. A reboot alone can swap uniqueIDs. External USB controllers (here, PCIe USB cards in two Thunderbolt enclosures) can race for bus numbers at boot, so with every cable left in place, a reboot swapped two of the cards: # Before reboot. 4K X uid=0x262000000fd9009b serial=A7SNB504219J0R 4K X uid=0x252000000fd9009b serial=A7SNB50423R73R # After reboot, no cables touched. 4K X uid=0x262000000fd9009b serial=A7SNB50423R73R 4K X uid=0x252000000fd9009b serial=A7SNB504219J0R This behavior is intermittent, a second reboot changed nothing, but a third caused another swap. Cards left alone in built-in ports retain their uniqueIDs across reboots in my testing; the failure requires dynamically enumerated external USB controllers. 3. Even the product ID tail can drift. One unit intermittently enumerates with idProduct 0x009c instead of 0x009b, same port (USB PCIe card in a Thunderbolt enclosure), cables untouched: # Before reboot. 4K X uid=0x222000000fd9009b serial=A7SNB50424UBQI # After reboot. 4K X uid=0x222000000fd9009c serial=A7SNB50424UBQI IOKit and AVFoundation agree each boot... So the change is upstream of both? I'm uncertain where to place blame for this specific issue (UVC device or macOS). Audio on the same physical units is unaffected. The audio uniqueID (AppleUSBAudioEngine:...:<serial>:...) embeds the USB serial and stayed stable through every test. So AVCaptureDevice can provide a stable per-device identifier, just not for UVC video devices. Questions: Is this a bug, or is the documentation overstating the persistence guarantee for USB video devices? What is the supported way to identify a specific physical UVC video device across reboots and port changes? The USB serial number is stable and is what I've fallen back on via IOKit, but there is no documented AVFoundation API to retrieve USB serial number from a UVC video AVCaptureDevice. Related: thread 803759, where the locationID-derived format is described. Script used for all output above (swift ./list-uvc.swift): import AVFoundation import IOKit func usbSerial(forLocation location: UInt32) -> String? { var iterator: io_iterator_t = 0 guard IOServiceGetMatchingServices(kIOMainPortDefault, IOServiceMatching("IOUSBHostDevice"), &iterator) == KERN_SUCCESS else { return nil } defer { IOObjectRelease(iterator) } var result: String? var service = IOIteratorNext(iterator) while service != 0 { var loc: UInt32 = 0 if let ref = IORegistryEntryCreateCFProperty(service, "locationID" as CFString, kCFAllocatorDefault, 0)?.takeRetainedValue(), let num = ref as? NSNumber { loc = num.uint32Value } if loc == location, let ref = IORegistryEntryCreateCFProperty(service, "USB Serial Number" as CFString, kCFAllocatorDefault, 0)?.takeRetainedValue(), let serial = ref as? String { result = serial } IOObjectRelease(service) if result != nil { break } service = IOIteratorNext(iterator) } return result } let session = AVCaptureDevice.DiscoverySession(deviceTypes: [.external], mediaType: .video, position: .unspecified) for device in session.devices { let uid = device.uniqueID let location = UInt32(truncatingIfNeeded: strtoull(uid, nil, 16) >> 32) let serial = usbSerial(forLocation: location) ?? "N/A" print("\(device.localizedName) uid=\(uid) serial=\(serial)") }
1
0
213
3w
AVCaptureDevice.uniqueID for UVC devices is unstable - bug or overstated documentation?
The documentation for AVCaptureDevice.uniqueID states the following: Capture devices have a unique identifier that persists on one system across device connections and disconnections, application restarts, and reboots of the system itself. You can store the value returned by this property to recall or track the status of a specific device in the future. For UVC capture devices this documentation does not hold. The video uniqueID is a hex string of the form 0x<locationID><vendorID><productID>, and the identifying half is the locationID (bus number plus port path). Which identifies a port, not a device. I ran a suite of tests with three identical Elgato 4K X capture cards connected to a Mac Studio w/ M3 Ultra running macOS 26.5.2, and reproduced my findings on a MacBook w/ M3 Pro (same macOS version). See the script at the bottom of the post for how uniqueId & USB serial number are being retrieved. 1. The uniqueID follows the port. Swapping two cards between two built-in ports swaps their uniqueIDs: # Before swap. 4K X uid=0x2000000fd9009b serial=A7SNB50424UBQI 4K X uid=0x12000000fd9009b serial=A7SNB504219J0R # After swapping the cards between the same two ports. 4K X uid=0x2000000fd9009b serial=A7SNB504219J0R 4K X uid=0x12000000fd9009b serial=A7SNB50424UBQI An app that stored 0x2000000fd9009b to recall a specific capture card now silently opens another. 2. A reboot alone can swap uniqueIDs. External USB controllers (here, PCIe USB cards in two Thunderbolt enclosures) can race for bus numbers at boot, so with every cable left in place, a reboot swapped two of the cards: # Before reboot. 4K X uid=0x262000000fd9009b serial=A7SNB504219J0R 4K X uid=0x252000000fd9009b serial=A7SNB50423R73R # After reboot, no cables touched. 4K X uid=0x262000000fd9009b serial=A7SNB50423R73R 4K X uid=0x252000000fd9009b serial=A7SNB504219J0R This behavior is intermittent, a second reboot changed nothing, but a third caused another swap. Cards left alone in built-in ports retain their uniqueIDs across reboots in my testing; the failure requires dynamically enumerated external USB controllers. 3. Even the product ID tail can drift. One unit intermittently enumerates with idProduct 0x009c instead of 0x009b, same port (USB PCIe card in a Thunderbolt enclosure), cables untouched: # Before reboot. 4K X uid=0x222000000fd9009b serial=A7SNB50424UBQI # After reboot. 4K X uid=0x222000000fd9009c serial=A7SNB50424UBQI IOKit and AVFoundation agree each boot... So the change is upstream of both? I'm uncertain where to place blame for this specific issue (UVC device or macOS). Audio on the same physical units is unaffected. The audio uniqueID (AppleUSBAudioEngine:...:<serial>:...) embeds the USB serial and stayed stable through every test. So AVCaptureDevice can provide a stable per-device identifier, just not for UVC video devices. Questions: Is this a bug, or is the documentation overstating the persistence guarantee for USB video devices? What is the supported way to identify a specific physical UVC video device across reboots and port changes? The USB serial number is stable and is what I've fallen back on via IOKit, but there is no documented AVFoundation API to retrieve USB serial number from a UVC video AVCaptureDevice. Related: thread 803759, where the locationID-derived format is described. Script used for all output above (swift ./list-uvc.swift): import AVFoundation import IOKit func usbSerial(forLocation location: UInt32) -> String? { var iterator: io_iterator_t = 0 guard IOServiceGetMatchingServices(kIOMainPortDefault, IOServiceMatching("IOUSBHostDevice"), &iterator) == KERN_SUCCESS else { return nil } defer { IOObjectRelease(iterator) } var result: String? var service = IOIteratorNext(iterator) while service != 0 { var loc: UInt32 = 0 if let ref = IORegistryEntryCreateCFProperty(service, "locationID" as CFString, kCFAllocatorDefault, 0)?.takeRetainedValue(), let num = ref as? NSNumber { loc = num.uint32Value } if loc == location, let ref = IORegistryEntryCreateCFProperty(service, "USB Serial Number" as CFString, kCFAllocatorDefault, 0)?.takeRetainedValue(), let serial = ref as? String { result = serial } IOObjectRelease(service) if result != nil { break } service = IOIteratorNext(iterator) } return result } let session = AVCaptureDevice.DiscoverySession(deviceTypes: [.external], mediaType: .video, position: .unspecified) for device in session.devices { let uid = device.uniqueID let location = UInt32(truncatingIfNeeded: strtoull(uid, nil, 16) >> 32) let serial = usbSerial(forLocation: location) ?? "N/A" print("\(device.localizedName) uid=\(uid) serial=\(serial)") }
1
0
262
3w
AVAudioEngine input tap intermittently delivers all-zero buffers — valid format, no error thrown
We have a long-form audio recording app built on AVAudioEngine. We install a tap on inputNode, accumulate the PCM buffers, and encode them to AAC in ~60-second chunks. Setup is essentially: let session = AVAudioSession.sharedInstance() try session.setCategory(.playAndRecord, mode: .default, options: [.defaultToSpeaker, .allowBluetooth, .allowBluetoothA2DP]) try session.setActive(true) let engine = AVAudioEngine() let input = engine.inputNode let format = input.inputFormat(forBus: 0) // valid, e.g. 48 kHz, 1 ch input.installTap(onBus: 0, bufferSize: 1024, format: format) { buffer, _ in // In the failure case, buffer.floatChannelData is entirely 0.0 // (accumulate + encode to AAC) } engine.prepare() try engine.start() Intermittently — and so far only reported from the field, never reproduced in normal testing — a recording comes out completely silent. When we decode the resulting AAC and inspect the raw PCM, every sample is exactly 0.0. The signature is very specific: The engine is running and the tap keeps firing for the full duration (normal number of buffers / full-length chunks). inputFormat is valid (sampleRate ≠ 0, e.g. 48 kHz). No error is thrown anywhere — setCategory, setActive, start(), and the tap callback all succeed. The PCM is literally all zeros (not low-level noise / room tone — exact 0.0). Two separate silent recordings decode to byte-identical AAC, confirming pure digital silence rather than corruption. So as far as our error handling, format checks, and tap-liveness are concerned, everything looks healthy — yet the microphone is delivering pure silence. One way we can reproduce it: recording while the iPhone is being driven via macOS iPhone Mirroring (the iPhone stays locked, the mic is effectively unavailable from the device, but our session still activates with a valid format and the tap fires zero-filled buffers for the whole recording — with no error at any point). What we've ruled out: microphone permission is granted; it's not truncation or short capture (full-length, full frame count); it's not our encoding step (the input buffers themselves are zero); it's not a quiet/obstructed mic (that would be low noise, not exact 0.0). Questions: What other device states or scenarios can cause a running AVAudioEngine input tap to deliver all-zero buffers with a valid format and no error? (e.g. another process/system feature holding the mic, Continuity Camera/Mic, CallKit/PushToTalk session ownership, etc.) Since this surfaces with no error and a valid format, what is the recommended way to detect it at runtime? Is monitoring the input level / PCM energy the only signal, or is there a supported API to know the input isn't actually live? What's the recommended recovery once detected — is a full session deactivate/reactivate re-handshake sufficient, or is recreating the engine required?
1
0
226
3w
AVAudioSession microphone recording changes Bluetooth audio route in vehicle
Title AVAudioSession microphone recording changes Bluetooth audio route in vehicle Hi everyone, I’m developing an iOS app called HearSave to help drivers safely remember and save songs they hear while driving. The app performs a very simple task: the user presses a button, the app records a short audio sample for music recognition, then immediately stops recording. I’m trying to understand whether the behaviour I’m seeing is an expected limitation of iOS audio routing or whether there is a better AVAudioSession configuration. Test setup iPhone connected to a vehicle via Bluetooth. Vehicle playing DAB radio. App requests microphone access and starts recording. Recording lasts approximately 7 seconds. Behaviour observed As soon as recording begins: The vehicle changes into a voice-call style audio route. DAB audio becomes muted. The vehicle’s volume control changes to voice volume. When recording finishes, the radio does not automatically recover until the user manually changes the audio source away from DAB and back again. I’ve experimented with several Expo Audio / AVAudioSession configurations, including releasing the audio session immediately after recording and adjusting audio session options, but the behaviour remains the same. My questions Is this expected behaviour when an iOS app starts microphone recording while connected to a vehicle over Bluetooth? Is there an AVAudioSession configuration that allows brief microphone capture without causing the vehicle to switch audio routes? Is it possible to explicitly use the iPhone’s built-in microphone while remaining connected to the vehicle via Bluetooth? If this behaviour is unavoidable over Bluetooth, is this also expected behaviour for native CarPlay apps, or are different audio routing capabilities available to CarPlay applications? Any guidance or documentation would be greatly appreciated. Thank you.
0
0
340
4w
Best practice for rapid sequential Live Photo captures with AVCapturePhotoOutput?
Hi everyone, I’m working on a camera app as a learning project and have reached a point where I’m trying to better understand the intended architecture for Live Photo capture using AVCapturePhotoOutput. The app currently supports: Live Photos Depth data Location metadata Multiple lens presets on a virtual multi-camera device Everything is working well, but I’m now thinking about capture throughput and rapid shutter presses. Right now, my implementation is fairly conservative. I wait for a Live Photo capture to finish processing and importing before allowing another capture. This is reliable, but it doesn’t feel particularly camera-like when compared to Apple’s Camera app. One observation from field testing caught my attention: I took a Live Photo, immediately switched lenses, then took another Live Photo. When I viewed the first Live Photo later, the movie portion included the lens-switching actions that occurred after I pressed the shutter. That made me realize that I may be thinking about the capture lifecycle incorrectly. My questions are: When using AVCapturePhotoOutput with Live Photos enabled, what is the earliest point at which a capture can be considered “safely secured”? Is it expected that apps wait for PhotoKit import to complete before accepting another Live Photo capture request? If supporting rapid sequential shutter presses, is the recommended approach to queue capture requests and process them one at a time? Are there any best practices around lens changes or camera reconfiguration while a Live Photo is still being captured or processed? I’m not looking for details about the implementation of Apple’s Camera app. I’m mainly trying to understand the recommended approach when working with the public AVFoundation APIs. I’d appreciate any guidance, documentation references, or examples from developers who have worked through similar problems. Thanks!
1
0
400
4w
Safari 26.5 Web Audio API Glitch: Automatic background audio playback triggering fast-forwarded "buffer catch-up" burst
https://streamable.com/y16s4q When I leave the Spotify Web Player open on its homepage and remain completely idle (without any physical user interaction or clicks), the page somehow self-triggers an active audio stream. Instantly, Safari's standard "This tab is playing audio" speaker icon appears normally on the right side of the URL/Address bar. Simultaneously, the browser emits a 1–2 second burst of extremely accelerated, fast-forwarded, and chaotic audio (resembling a time-stretch or "Alvin and the Chipmunks" effect) before going silent or stabilizing.
1
0
401
4w
CNAssetSpatialAudioInfo / Audio Mix rejects ProRes spatial captures (LPCM FOA) — only HEVC (APAC) is eligible. Intended? And how can Audio Mix coexist with ProRes recording?
On iOS/macOS 26, CNAssetSpatialAudioInfo(asset:) and CNAssetSpatialAudioInfo.assetContainsSpatialAudio(asset:) accept a spatial capture only when its spatial track is APAC-encoded which AVCaptureMovieFileOutput produces when the video codec is HEVC. An otherwise-identical ProRes capture, whose spatial track is LPCM (4-channel First-Order Ambisonics, kAudioChannelLayoutTag_HOA_ACN_SN3D | 4), is rejected with CNCinematicErrorDomain code 3 (CNCinematicErrorCodeIncomplete) "no eligible audio tracks in asset". This reproduces with Apple's own SpatialAudioCLI sample run on Apple's own stock iPhone captures, so it appears to be a property of the format/API rather than my code. I'd like to confirm whether this is intended, and find a supported way to obtain Audio-Mix-eligible spatial audio while still recording ProRes video (we use Apple Log/ProRes for color grading). Respectively, is wiring a manual AVAssetWriter setup the only way to manage spatial audio and ProRes video? Eligibility appears to require an APAC-encoded, exactly-4-channel FOA track. Because AVCaptureMovieFileOutput only writes APAC audio for HEVC (ProRes forces LPCM), ProRes spatial captures are never eligible — including Apple's own ProRes stock captures, which SpatialAudioCLI also rejects. Key finding: eligibility seems baked into the native APAC bitstream Starting from an eligible HEVC/APAC file, I used AVAssetReader/AVAssetWriter to re-encode only the FOA track (APAC → LPCM → APAC), leaving the AAC stereo track, the HEVC video, and the timed-metadata track untouched. The structurally-identical output is then rejected (code 3). Preserving the cinematic-audio metadata track is not sufficient. Re-encoding the APAC itself loses eligibility. This suggests the mix metadata that gates eligibility is carried inside the APAC bitstream and is produced only at capture time. Questions Is it intended that ProRes (LPCM FOA) spatial captures are not Audio-Mix-eligible via CNAssetSpatialAudioInfo, while HEVC (APAC FOA) captures are? Is this documented? Where exactly is the eligibility metadata stored — in the APAC bitstream, or in the cinematic-audio timed-metadata track (Re-encoding the APAC while preserving that metadata track still loses eligibility)? Is there any supported way to make an existing LPCM/ProRes FOA capture eligible after the fact (a transcode/encode path that produces the required APAC), or is native capture the only source? Any guidance, or a pointer to documentation, would be greatly appreciated. Thank you. Environment iPhone 16 Pro Max, iOS 26.x; macOS 26.2; Xcode 26.2.
1
0
418
Jun ’26
CarPlay: Is vehicle microphone capture without entering communications audio mode supported for third-party apps?
Hello, I am developing a native CarPlay application and would appreciate some clarification regarding AVAudioSession behaviour when using the vehicle microphone. The application is intentionally simple and designed to minimise driver distraction. The user presses a single button on the CarPlay screen, the application performs a brief (approximately five second) audio capture, performs application-specific processing on the captured audio, displays a simple confirmation to the user, and immediately releases the audio session. The goal is to allow the driver to continue their journey with as little interaction as possible. Because the audio being captured is often originating from the vehicle’s native DAB/FM radio rather than from the iPhone itself, preserving the existing listening experience during the brief capture is fundamental to the intended design. To better understand what is supported by CarPlay, I have carried out a number of controlled experiments. Test Environment Native CarPlay application Swift AVFoundation AVAudioSession AVAudioRecorder Vehicle connected via CarPlay Vehicle playing its native DAB radio Experiment 1 Configuration Category: AVAudioSession.Category.playAndRecord Mode: AVAudioSession.Mode.default Observed Route Input: CarPlay — CarAudio Output: CarPlay — CarAudio Result Recording succeeds using the vehicle microphone. Native DAB audio is muted during recording. The vehicle changes from “Audio Volume” to “Voice Volume”. When recording finishes and the AVAudioSession is deactivated, DAB resumes normally. Experiment 2 Changed only the session mode to: AVAudioSession.Mode.measurement Result Behaviour was identical to Experiment 1. Experiment 3 Changed only the session mode to: AVAudioSession.Mode.videoRecording Observed Route Input: iPhone microphone Output: CarPlay — CarAudio Result Input automatically switched from the vehicle microphone to the iPhone microphone. The vehicle remained in normal “Audio Volume”. However, the head unit switched away from its native DAB source to the CarPlay media source. Observation These experiments appear to suggest that the behaviour is specifically related to using the CarPlay “CarAudio” microphone route, rather than audio recording itself. Question Is this the expected behaviour for third-party CarPlay applications? More specifically: Is there any supported AVAudioSession configuration or CarPlay API that allows a third-party CarPlay application to perform a brief, user-initiated audio capture using the vehicle microphone without causing the head unit to enter its communications/voice audio mode or interrupt the vehicle’s native radio playback? If this behaviour is by design and no supported mechanism exists to achieve this, confirmation of that would be extremely valuable, as it would allow us to design the application accordingly. If additional information, sample code, AVAudioSession logs or detailed reproduction steps would be helpful, I would be more than happy to provide them. Thank you very much for your time. Kind regards, Neil Jenner Developer, HearSave
Replies
1
Boosts
0
Views
64
Activity
17h
ProResRAW shooting issue with AVCaptureMovieFileOutput with the first video
On iOS 26, the first ProRes RAW recording after launching the app consistently stalls for the entire take: audio records normally, but video contains only a few frames (for example, an 11.8-second clip at approximately 0.25 fps instead of 30 fps). Every subsequent recording works correctly. Before startRecording, the Bayer format is active, ProRes RAW is available, white balance is locked, and frame duration is pinned to 1/30. Reordering setOutputSettings and reasserting configuration have not resolved it; only discarding the first recording acts as a reliable warm-up. Is this a known one-time ProRes RAW encoder initialisation issue, and is there a supported way to prepare the encoder before the first user takes? I have a full working sample code if anyone needs but here is the setup function: import CoreMedia import CoreVideo @available(iOS 26.0, *) final class ProResRAWRecorder: NSObject, AVCaptureFileOutputRecordingDelegate { let session = AVCaptureSession() private let movieOutput = AVCaptureMovieFileOutput() private var camera: AVCaptureDevice! // Call on a serial capture queue. func configure() throws { session.beginConfiguration() defer { session.commitConfiguration() } session.sessionPreset = .inputPriority session.automaticallyConfiguresCaptureDeviceForWideColor = false guard let device = AVCaptureDevice.default( .builtInWideAngleCamera, for: .video, position: .back ) else { throw SampleError.noCamera } let cameraInput = try AVCaptureDeviceInput(device: device) guard session.canAddInput(cameraInput) else { throw SampleError.cannotAddInput } session.addInput(cameraInput) camera = device // Optional, but reproduces the real topology where audio remains healthy. if let microphone = AVCaptureDevice.default(for: .audio), let microphoneInput = try? AVCaptureDeviceInput(device: microphone), session.canAddInput(microphoneInput) { session.addInput(microphoneInput) } // Find a 12-bit packed Bayer format supporting 30 fps. guard let rawFormat = device.formats .filter({ CMFormatDescriptionGetMediaSubType($0.formatDescription) == kCVPixelFormatType_96VersatileBayerPacked12 && $0.videoSupportedFrameRateRanges.contains { $0.minFrameRate <= 30 && $0.maxFrameRate >= 30 } }) .max(by: { let a = CMVideoFormatDescriptionGetDimensions($0.formatDescription) let b = CMVideoFormatDescriptionGetDimensions($1.formatDescription) return Int(a.width) * Int(a.height) < Int(b.width) * Int(b.height) }) else { throw SampleError.noRAWFormat } try device.lockForConfiguration() device.activeFormat = rawFormat if rawFormat.supportedColorSpaces.contains(.appleLog) { device.activeColorSpace = .appleLog } let frameDuration = CMTime(value: 1, timescale: 30) device.activeVideoMinFrameDuration = frameDuration device.activeVideoMaxFrameDuration = frameDuration if device.isWhiteBalanceModeSupported(.locked) { device.whiteBalanceMode = .locked } device.unlockForConfiguration() guard session.canAddOutput(movieOutput) else { throw SampleError.cannotAddOutput } session.addOutput(movieOutput) session.startRunning() } // Call on the same serial capture queue after startRunning() returns. func record(to url: URL) throws { guard let connection = movieOutput.connection(with: .video) else { throw SampleError.noVideoConnection } // Reassert the required device state at the take boundary. try camera.lockForConfiguration() if camera.isWhiteBalanceModeSupported(.locked) { camera.whiteBalanceMode = .locked } let frameDuration = CMTime(value: 1, timescale: 30) camera.activeVideoMinFrameDuration = frameDuration camera.activeVideoMaxFrameDuration = frameDuration camera.unlockForConfiguration() guard movieOutput.availableVideoCodecTypes.contains(.proResRAW) else { throw SampleError.rawCodecUnavailable } movieOutput.setOutputSettings( [AVVideoCodecKey: AVVideoCodecType.proResRAW], for: connection ) movieOutput.startRecording(to: url, recordingDelegate: self) } func stop() { if movieOutput.isRecording { movieOutput.stopRecording() } } func fileOutput( _ output: AVCaptureFileOutput, didFinishRecordingTo outputFileURL: URL, from connections: [AVCaptureConnection], error: Error? ) { print("Finished:", outputFileURL, "error:", error as Any) } enum SampleError: Error { case noCamera case cannotAddInput case noRAWFormat case cannotAddOutput case noVideoConnection case rawCodecUnavailable } }
Replies
1
Boosts
0
Views
311
Activity
19h
Manual legible (subtitle) selection not honored on live LL-HLS — select(_:in:) reverts to “off” within ~2s; automatic selection never displays non-forced subtitles
Environment: iOS 18 / iOS 26, AVPlayer + AVPlayerItem, live low-latency HLS (LL-HLS). Subtitle renditions are regular (non-forced) WebVTT: AUTOSELECT=YES, FORCED=NO. System captioning (Closed Captions + SDH) is OFF (default). Direct CDN, no P2P. Master playlist (subtitle part): #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Korean (AI)",AUTOSELECT=YES,FORCED=NO,LANGUAGE="kor",URI="..." #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English (AI)",AUTOSELECT=YES,FORCED=NO,LANGUAGE="eng",URI="..." #EXT-X-STREAM-INF:...,SUBTITLES="subs" Problem When the user picks a subtitle language we call playerItem.select(option, in: legibleGroup) with an option that is a genuine member of the item's current legible group. Right after the call, item.currentMediaSelection.selectedMediaOption(in:) returns the requested option. But within ~2 seconds, on a subset of real-user sessions, the selection spontaneously reverts with no further app interaction: most often to Off (no legible output is delivered afterwards), or it stays stuck on the previously selected language (a subsequent select to another language, or to Off, is silently ignored). It is intermittent and only appears under real conditions (frequent live playlist reloads / reconnects); it does not reproduce in a short, clean session. What we verified The option passed to select(_:in:) is a real member of the current group (re-resolved from item.asset); the call returns without error and reads back correctly immediately after. appliesMediaSelectionCriteriaAutomatically is the default true. Per the AV Foundation Release Notes ("Advice about subtitles"), automatic selection excludes options that are not AVMediaCharacteristicContainsOnlyForcedSubtitles. So on each live reload the automatic re-selection resolves non-forced subtitles to Off, appearing to override the manual select(_:in:). Reading the selected option against a freshly obtained group instance returns the same value as against the original instance — so this is a real state change, not a mismatched-group read. Questions On live HLS reloads with appliesMediaSelectionCriteriaAutomatically == true, is manual select(_:in:) for .legible expected to be overridden by automatic media selection? If so, is setMediaSelectionCriteria(_:forMediaCharacteristic:) the intended way to persist a user's choice? setMediaSelectionCriteria for subtitles is itself reported as unreliable (sometimes no subtitles) — see thread 108403. What is the recommended, deterministic way to keep a user-selected non-forced subtitle displayed across live playlist reloads, including turning subtitles Off? Is this the same underlying behavior as FB13344652 ("Auto (Recommended) doesn't display subtitles despite language match / DEFAULT=YES")? Related: https://developer.apple.com/forums/thread/722752 (FB13344652) , https://developer.apple.com/forums/thread/108403
Replies
1
Boosts
0
Views
98
Activity
3d
iOS 27 beta: Opening Notification Center pauses AVPlayer playback
Since iOS 27 beta we are seeing a behavior change in our video streaming app and I would like to know whether others can reproduce it. Behavior on iOS 27 beta: Fully opening the Notification Center pauses playback. Audio continues for about 5 more seconds, then the app is suspended. Closing the Notification Center leaves the player paused. On iOS 26 the same build keeps playing in this situation. Setup: AVQueuePlayer playing video with audio, not muted audiovisualBackgroundPlaybackPolicy = .automatic (default) AVAudioSession category .playback, UIBackgroundModes: audio entitlement AVPictureInPictureController attached with canStartPictureInPictureAutomaticallyFromInline = true Filed as FB23893965 Questions: Can anyone reproduce this on iOS 27 beta? Is this intentional or a regression?
Replies
0
Boosts
2
Views
84
Activity
4d
Add a value to the Photos Caption field
In the iOS Photos app there is a caption field the user can write to. How can you write to this value from Swift when creating a photo? I see apps that do this, but there doesn't seem to be any official way to do this using the Photo library through PHAssetCreationRequest or PHAssetResourceCreationOptions or setting EXIF values, I tried settings a bunch of values there including IPTC values but nothing appears in the caption field in the iOS photos app. There must be some way to do it since I see other apps setting that value somehow after capturing a photo.
Replies
2
Boosts
1
Views
319
Activity
1w
AVRoutePickerView's presented route list is automatically dismissed when CXProvider reports the call as connected (`reportOutgoingCall(with:connectedAt:)`)
Hi, I want my user to be able to change their audio output device while calling someone, even before the call is established. But I've run into an issue combining CallKit and AVRoutePickerView. On iOS 26 (Xcode 26.5, tested on iPhone 13 Mini): when my AVRoutePickerView's route list is presented while my call is still connecting (not yet established), the presented list is automatically dismissed the moment the app reports the call as connected via CXProvider.reportOutgoingCall(with:connectedAt:). That doesn't seem natural to me — it should stay open, since the user is actively interacting with unrelated system UI at that moment. This happens even when nothing about the AVAudioSession itself changes: no category change, no route change, no didActivate/didDeactivate callback fires around that time. The dismissal is caused by the CallKit "connected" state transition itself, but I don't know why, or whether it's intentional. Steps to reproduce (sample project below): Tap "Start Fake Call". Within 5 seconds, tap the small route picker button and leave its list open. Wait — at t+5s the app reports the call connected via reportOutgoingCall(connectedAt:), and the list closes itself right after. Is this normal? And is there an official, CallKit-sanctioned way to report a call's connected state without this side effect — some way to keep the route picker stable across that transition? The two workarounds I've found both have real downsides: Calling reportOutgoingCall(connectedAt:) at the very start of the call, before the callee has actually answered , but then CallKit no longer reflects the real call state (Recents/duration would include ringing time, and calls that are never answered would still show as "connected"). Disabling audio device selection entirely until the call is established... works, but hurts the user experience, since users may want to switch output before the call connects. Here's the minimal reproduction: import CallKit import SwiftUI struct ContentView: View { @StateObject private var demo = CallKitDemo() var body: some View { VStack(spacing: 24) { Text(demo.statusText) .font(.system(.body, design: .monospaced)) .multilineTextAlignment(.center) Button("Start Fake Call") { demo.startFakeCall() } .disabled(demo.isCallInProgress) RoutePickerView() .frame(width: 60, height: 60) } .padding() } } private struct RoutePickerView: UIViewRepresentable { func makeUIView(context: Context) -> AVRoutePickerView { let picker = AVRoutePickerView() picker.delegate = context.coordinator return picker } func updateUIView(_ uiView: AVRoutePickerView, context: Context) {} func makeCoordinator() -> Coordinator { Coordinator() } final class Coordinator: NSObject, AVRoutePickerViewDelegate { func routePickerViewWillBeginPresentingRoutes(_ routePickerView: AVRoutePickerView) { print("[REPRO]", Date(), "picker WILL present routes") } func routePickerViewDidEndPresentingRoutes(_ routePickerView: AVRoutePickerView) { print("[REPRO]", Date(), "picker DID end presenting routes <- dismissed here") } } } @MainActor final class CallKitDemo: NSObject, ObservableObject { @Published var statusText = "Idle" @Published var isCallInProgress = false private let provider: CXProvider = { let configuration = CXProviderConfiguration() configuration.supportsVideo = false configuration.maximumCallGroups = 1 configuration.maximumCallsPerCallGroup = 1 configuration.supportedHandleTypes = [.generic] return CXProvider(configuration: configuration) }() private let callController = CXCallController() private var currentCallUUID: UUID? override init() { super.init() provider.setDelegate(self, queue: nil) } func startFakeCall() { let uuid = UUID() currentCallUUID = uuid isCallInProgress = true statusText = "Requesting CXStartCallAction..." let handle = CXHandle(type: .generic, value: "repro-call") let startAction = CXStartCallAction(call: uuid, handle: handle) callController.request(CXTransaction(action: startAction)) { error in if let error { print("[REPRO]", Date(), "CXStartCallAction request failed:", error) } } DispatchQueue.main.asyncAfter(deadline: .now() + 5) { [weak self] in guard let self, currentCallUUID == uuid else { return } print("[REPRO]", Date(), "reportOutgoingCall(connectedAt:)") statusText = "Reported connected at t+5s" provider.reportOutgoingCall(with: uuid, connectedAt: Date()) } DispatchQueue.main.asyncAfter(deadline: .now() + 8) { [weak self] in guard let self, currentCallUUID == uuid else { return } callController.request(CXTransaction(action: CXEndCallAction(call: uuid))) { _ in } currentCallUUID = nil isCallInProgress = false statusText = "Call ended" } } } extension CallKitDemo: CXProviderDelegate { func providerDidReset(_ provider: CXProvider) {} func provider(_ provider: CXProvider, perform action: CXStartCallAction) { action.fulfill() } func provider(_ provider: CXProvider, perform action: CXEndCallAction) { action.fulfill() } func provider(_ provider: CXProvider, didActivate audioSession: AVAudioSession) { print("[REPRO]", Date(), "didActivate") } func provider(_ provider: CXProvider, didDeactivate audioSession: AVAudioSession) { print("[REPRO]", Date(), "didDeactivate") } } Thanks for your help !
Replies
1
Boosts
0
Views
171
Activity
1w
How should live latency be measured and maintained with AVPlayer (HLS / LL-HLS)?
We keep live playback at a consistent distance from the live edge using small playback rate adjustments, with a target range based on recommendedTimeOffsetFromLive. Since the live edge is not exposed by AVPlayer, we currently fall back to seekableTimeRanges.end as our best approximation. What should be treated as the live edge, and how should the current live latency be measured? Is rate adjustment the appropriate way to hold a target latency? While playing above 1.0x, the playhead can reach the seekable end, at which point AVPlayerItemDidPlayToEndTime fires and halts the live stream. How can we guard against ? Does any of this differ between regular HLS and LL-HLS? A clear statement of the intended contract here would resolve a lot of uncertainty. Thanks in advance.
Replies
2
Boosts
0
Views
289
Activity
1w
`LockedCameraCaptureManager` practically unusable since iOS 26
Somewhere since iOS 26, the LockedCameraCapture framework gets in an unpredictable state after opening the main app from the LockedCamera extension using LockedCameraCaptureSession.openApplication(for userActivity:). (Feedback with sample code to reproduce: FB21966835) Opening the extension from the lock screen again doesn’t open the extension but puts the lock screen in a state as if it has. Content updated from LockedCameraCaptureManager.shared.sessionContentUpdates comes in inconsistently, usually needs the app to be opened again or the extension to be opened. This makes using this extension impossible for me as I use it to record video files that manually need to be imported when the app is launched (so not through PhotoKit). Does anybody have a suggestion to circumvent this issue or how to get this fixed?
Replies
1
Boosts
2
Views
612
Activity
1w
iOS 26.4 regression: The `.pauses` audiovisual background playback policy does not pause video playback anymore when backgrounding the app
Starting with iOS 26.4 and the iOS 26.4 SDK, the .pauses audiovisual background playback policy is not correctly applied anymore to an AVPlayer having an attached video layer displayed on screen. This means that, when backgrounding a video-playing app (without Picture in Picture support) or locking the device, playback is not paused automatically by the system anymore. This issue affects the Apple TV application as well. We have filed FB22488151 with more information.
Replies
2
Boosts
0
Views
713
Activity
2w
AVQueuePlayer unexpectedly performs network requests during offline HLS playback after several queued episode transitions
Hello, We are investigating an issue with offline HLS playback using AVQueuePlayer and would like to know whether anyone else has experienced similar behavior. Issue We download HLS content using AVAssetDownloadURLSession and play it offline using AVQueuePlayer. For some titles (but not all), after several consecutive episode transitions, the player unexpectedly attempts a network request while the next episode is already queued and the current episode has approximately 60 seconds remaining. If the device is offline, playback fails with: NSURLErrorDomain Code = -1009 and the next episode never starts. Characteristics The issue only affects certain titles. It is fully reproducible for affected titles. For example, if it occurs between Episodes 5 and 6 after starting playback from Episode 1, it always occurs at the same point when replaying from Episode 1. If playback starts directly from Episode 5, the issue does not occur. The issue only occurs when using AVQueuePlayer. Replacing the current item (removeAllItems() + replaceCurrentItem(with:)) avoids the issue, although this is unfortunately not a viable workaround because it breaks our Picture in Picture episode transition behavior. We compared the downloaded packages (boot.xml, Master Playlist, and Stream configuration) between affected and unaffected titles, but so far have not identified any meaningful structural differences that explain the behavior. Questions Has anyone experienced similar behavior with: offline HLS (.movpkg) AVQueuePlayer unexpected network requests during queued playback NSURLErrorDomain Code=-1009 even though the content is downloaded for offline playback If anyone has seen a similar issue or has any information, observations, or suggestions for further investigation, I would greatly appreciate hearing from you. For reference, I have already submitted this issue through Feedback Assistant. Feedback ID: FB23487817 Thank you in advance for any information.
Replies
2
Boosts
0
Views
285
Activity
2w
Custom AVVideoCompositing on a composition-backed AVPlayerItem fails with AVErrorUnknown Xcode 27 beta 2 / beta 3
Trivial pass-through compositor fails on Xcode 27 (beta 2, beta 3); error code -11800 underlying error -12784. Repro included https://github.com/BugorBN/avplayer-custom-compositor-repro It works well on Xcode26 and lower
Replies
0
Boosts
1
Views
148
Activity
2w
AVPlayer Reverse Audio Scrubbing?
Hey all, here seeking some perspective. I have an audio player app on macOS built on top of AVPlayer, I want to add the ability to scrub the audio, and hear the audio frames based on the playhead's position whether going forwards or backwards. When going backwards, the audio frame should be played in reverse as well. The audio tracks live online and are streamed. I tried playing with AVPlayer.rate, but the time pitch algos built in (.spectral, .varispeed, .timeDomain) all only guarantee up to 32x rate decoding accuracy. So technically, if the user scrubs fast enough, the audio rendered would not necessarily match the playhead's position. My current solution that works is to cache the raw audio bytes and play the appropriate frame when the user starts scrubbing. I decode the audio data manually using AudioToolbox's AudioFileOpenWithCallbacks into an AVAudioPCMBuffer, then pass it into AVAudioEngine+AVAudioPlayerNode combo. The problem with that is that means I need to cache this audio data myself (remember this is a stream), and since I don't have access to AVPlayer's own cache I need to also download it myself... which means two downloads for the same track which is less than ideal. This lead me to take it a step further and hijack AVPlayer's download process by implementing AVAssetResourceLoaderDelegate, that way AVPlayer and my audio scrubbing cache are both fed from the same source. Now... I feel like I went down a bit of a rabbit hole here. At the end of the day I simply want accurate audio scrubbing in both directions, while keeping in mind I want the audio snippets to play in reverse when the user goes backwards. Is there really no way to do this that's more "vanilla"? Am I missing something obvious? Genuinely open to any and all suggestions. Thanks.
Replies
0
Boosts
0
Views
228
Activity
2w
Custom UI over the Picture in Picture window for AVPlayerInterstitialEvent items — is there an official way?
I attach an ad in front of the main video using AVPlayerInterstitialEvent, and PiP works across the interstitial→content transition. I’ve noticed that some apps, while in Picture in Picture, show additional UI that: displays the remaining time of the interstitial item, and lets the user skip the interstitial item and switch to the main video. As far as I know, the AVPictureInPictureController window is system-drawn and can’t be customized by the developer. So my questions are: Is there an official/supported path to present this kind of UI (a remaining-time indicator and a skip control for the interstitial item) over the PiP window? If so, what is the mechanism — is it a system control that gets forwarded to a delegate, framework-provided interstitial UI, or When the interstitial is skipped this way, does the framework return to the primary item automatically, or does the app drive it? I’d really appreciate any insight intodone. Thanks!
Replies
0
Boosts
0
Views
118
Activity
3w
AVCaptureDevice.uniqueID for UVC video devices is unstable - bug or overstated documentation?
The documentation for AVCaptureDevice.uniqueID states the following: Capture devices have a unique identifier that persists on one system across device connections and disconnections, application restarts, and reboots of the system itself. You can store the value returned by this property to recall or track the status of a specific device in the future. For UVC capture devices this documentation does not hold. The video uniqueID is a hex string of the form 0x, and the identifying half is the locationID (bus number plus port path). Which identifies a port, not a device. I ran a suite of tests with three identical Elgato 4K X capture cards connected to a Mac Studio w/ M3 Ultra running macOS 26.5.2, and reproduced my findings on a MacBook w/ M3 Pro (same macOS version). See the script at the bottom of the post for how uniqueId & USB serial number are being retrieved. 1. The uniqueID follows the port. Swapping two cards between two built-in ports swaps their uniqueIDs: # Before swap. 4K X uid=0x2000000fd9009b serial=A7SNB50424UBQI 4K X uid=0x12000000fd9009b serial=A7SNB504219J0R # After swapping the cards between the same two ports. 4K X uid=0x2000000fd9009b serial=A7SNB504219J0R 4K X uid=0x12000000fd9009b serial=A7SNB50424UBQI An app that stored 0x2000000fd9009b to recall a specific capture card now silently opens another. 2. A reboot alone can swap uniqueIDs. External USB controllers (here, PCIe USB cards in two Thunderbolt enclosures) can race for bus numbers at boot, so with every cable left in place, a reboot swapped two of the cards: # Before reboot. 4K X uid=0x262000000fd9009b serial=A7SNB504219J0R 4K X uid=0x252000000fd9009b serial=A7SNB50423R73R # After reboot, no cables touched. 4K X uid=0x262000000fd9009b serial=A7SNB50423R73R 4K X uid=0x252000000fd9009b serial=A7SNB504219J0R This behavior is intermittent, a second reboot changed nothing, but a third caused another swap. Cards left alone in built-in ports retain their uniqueIDs across reboots in my testing; the failure requires dynamically enumerated external USB controllers. 3. Even the product ID tail can drift. One unit intermittently enumerates with idProduct 0x009c instead of 0x009b, same port (USB PCIe card in a Thunderbolt enclosure), cables untouched: # Before reboot. 4K X uid=0x222000000fd9009b serial=A7SNB50424UBQI # After reboot. 4K X uid=0x222000000fd9009c serial=A7SNB50424UBQI IOKit and AVFoundation agree each boot... So the change is upstream of both? I'm uncertain where to place blame for this specific issue (UVC device or macOS). Audio on the same physical units is unaffected. The audio uniqueID (AppleUSBAudioEngine:...:<serial>:...) embeds the USB serial and stayed stable through every test. So AVCaptureDevice can provide a stable per-device identifier, just not for UVC video devices. Questions: Is this a bug, or is the documentation overstating the persistence guarantee for USB video devices? What is the supported way to identify a specific physical UVC video device across reboots and port changes? The USB serial number is stable and is what I've fallen back on via IOKit, but there is no documented AVFoundation API to retrieve USB serial number from a UVC video AVCaptureDevice. Related: thread 803759, where the locationID-derived format is described. Script used for all output above (swift ./list-uvc.swift): import AVFoundation import IOKit func usbSerial(forLocation location: UInt32) -> String? { var iterator: io_iterator_t = 0 guard IOServiceGetMatchingServices(kIOMainPortDefault, IOServiceMatching("IOUSBHostDevice"), &iterator) == KERN_SUCCESS else { return nil } defer { IOObjectRelease(iterator) } var result: String? var service = IOIteratorNext(iterator) while service != 0 { var loc: UInt32 = 0 if let ref = IORegistryEntryCreateCFProperty(service, "locationID" as CFString, kCFAllocatorDefault, 0)?.takeRetainedValue(), let num = ref as? NSNumber { loc = num.uint32Value } if loc == location, let ref = IORegistryEntryCreateCFProperty(service, "USB Serial Number" as CFString, kCFAllocatorDefault, 0)?.takeRetainedValue(), let serial = ref as? String { result = serial } IOObjectRelease(service) if result != nil { break } service = IOIteratorNext(iterator) } return result } let session = AVCaptureDevice.DiscoverySession(deviceTypes: [.external], mediaType: .video, position: .unspecified) for device in session.devices { let uid = device.uniqueID let location = UInt32(truncatingIfNeeded: strtoull(uid, nil, 16) >> 32) let serial = usbSerial(forLocation: location) ?? "N/A" print("\(device.localizedName) uid=\(uid) serial=\(serial)") }
Replies
1
Boosts
0
Views
213
Activity
3w
AVCaptureDevice.uniqueID for UVC devices is unstable - bug or overstated documentation?
The documentation for AVCaptureDevice.uniqueID states the following: Capture devices have a unique identifier that persists on one system across device connections and disconnections, application restarts, and reboots of the system itself. You can store the value returned by this property to recall or track the status of a specific device in the future. For UVC capture devices this documentation does not hold. The video uniqueID is a hex string of the form 0x<locationID><vendorID><productID>, and the identifying half is the locationID (bus number plus port path). Which identifies a port, not a device. I ran a suite of tests with three identical Elgato 4K X capture cards connected to a Mac Studio w/ M3 Ultra running macOS 26.5.2, and reproduced my findings on a MacBook w/ M3 Pro (same macOS version). See the script at the bottom of the post for how uniqueId & USB serial number are being retrieved. 1. The uniqueID follows the port. Swapping two cards between two built-in ports swaps their uniqueIDs: # Before swap. 4K X uid=0x2000000fd9009b serial=A7SNB50424UBQI 4K X uid=0x12000000fd9009b serial=A7SNB504219J0R # After swapping the cards between the same two ports. 4K X uid=0x2000000fd9009b serial=A7SNB504219J0R 4K X uid=0x12000000fd9009b serial=A7SNB50424UBQI An app that stored 0x2000000fd9009b to recall a specific capture card now silently opens another. 2. A reboot alone can swap uniqueIDs. External USB controllers (here, PCIe USB cards in two Thunderbolt enclosures) can race for bus numbers at boot, so with every cable left in place, a reboot swapped two of the cards: # Before reboot. 4K X uid=0x262000000fd9009b serial=A7SNB504219J0R 4K X uid=0x252000000fd9009b serial=A7SNB50423R73R # After reboot, no cables touched. 4K X uid=0x262000000fd9009b serial=A7SNB50423R73R 4K X uid=0x252000000fd9009b serial=A7SNB504219J0R This behavior is intermittent, a second reboot changed nothing, but a third caused another swap. Cards left alone in built-in ports retain their uniqueIDs across reboots in my testing; the failure requires dynamically enumerated external USB controllers. 3. Even the product ID tail can drift. One unit intermittently enumerates with idProduct 0x009c instead of 0x009b, same port (USB PCIe card in a Thunderbolt enclosure), cables untouched: # Before reboot. 4K X uid=0x222000000fd9009b serial=A7SNB50424UBQI # After reboot. 4K X uid=0x222000000fd9009c serial=A7SNB50424UBQI IOKit and AVFoundation agree each boot... So the change is upstream of both? I'm uncertain where to place blame for this specific issue (UVC device or macOS). Audio on the same physical units is unaffected. The audio uniqueID (AppleUSBAudioEngine:...:<serial>:...) embeds the USB serial and stayed stable through every test. So AVCaptureDevice can provide a stable per-device identifier, just not for UVC video devices. Questions: Is this a bug, or is the documentation overstating the persistence guarantee for USB video devices? What is the supported way to identify a specific physical UVC video device across reboots and port changes? The USB serial number is stable and is what I've fallen back on via IOKit, but there is no documented AVFoundation API to retrieve USB serial number from a UVC video AVCaptureDevice. Related: thread 803759, where the locationID-derived format is described. Script used for all output above (swift ./list-uvc.swift): import AVFoundation import IOKit func usbSerial(forLocation location: UInt32) -> String? { var iterator: io_iterator_t = 0 guard IOServiceGetMatchingServices(kIOMainPortDefault, IOServiceMatching("IOUSBHostDevice"), &iterator) == KERN_SUCCESS else { return nil } defer { IOObjectRelease(iterator) } var result: String? var service = IOIteratorNext(iterator) while service != 0 { var loc: UInt32 = 0 if let ref = IORegistryEntryCreateCFProperty(service, "locationID" as CFString, kCFAllocatorDefault, 0)?.takeRetainedValue(), let num = ref as? NSNumber { loc = num.uint32Value } if loc == location, let ref = IORegistryEntryCreateCFProperty(service, "USB Serial Number" as CFString, kCFAllocatorDefault, 0)?.takeRetainedValue(), let serial = ref as? String { result = serial } IOObjectRelease(service) if result != nil { break } service = IOIteratorNext(iterator) } return result } let session = AVCaptureDevice.DiscoverySession(deviceTypes: [.external], mediaType: .video, position: .unspecified) for device in session.devices { let uid = device.uniqueID let location = UInt32(truncatingIfNeeded: strtoull(uid, nil, 16) >> 32) let serial = usbSerial(forLocation: location) ?? "N/A" print("\(device.localizedName) uid=\(uid) serial=\(serial)") }
Replies
1
Boosts
0
Views
262
Activity
3w
AVAudioEngine input tap intermittently delivers all-zero buffers — valid format, no error thrown
We have a long-form audio recording app built on AVAudioEngine. We install a tap on inputNode, accumulate the PCM buffers, and encode them to AAC in ~60-second chunks. Setup is essentially: let session = AVAudioSession.sharedInstance() try session.setCategory(.playAndRecord, mode: .default, options: [.defaultToSpeaker, .allowBluetooth, .allowBluetoothA2DP]) try session.setActive(true) let engine = AVAudioEngine() let input = engine.inputNode let format = input.inputFormat(forBus: 0) // valid, e.g. 48 kHz, 1 ch input.installTap(onBus: 0, bufferSize: 1024, format: format) { buffer, _ in // In the failure case, buffer.floatChannelData is entirely 0.0 // (accumulate + encode to AAC) } engine.prepare() try engine.start() Intermittently — and so far only reported from the field, never reproduced in normal testing — a recording comes out completely silent. When we decode the resulting AAC and inspect the raw PCM, every sample is exactly 0.0. The signature is very specific: The engine is running and the tap keeps firing for the full duration (normal number of buffers / full-length chunks). inputFormat is valid (sampleRate ≠ 0, e.g. 48 kHz). No error is thrown anywhere — setCategory, setActive, start(), and the tap callback all succeed. The PCM is literally all zeros (not low-level noise / room tone — exact 0.0). Two separate silent recordings decode to byte-identical AAC, confirming pure digital silence rather than corruption. So as far as our error handling, format checks, and tap-liveness are concerned, everything looks healthy — yet the microphone is delivering pure silence. One way we can reproduce it: recording while the iPhone is being driven via macOS iPhone Mirroring (the iPhone stays locked, the mic is effectively unavailable from the device, but our session still activates with a valid format and the tap fires zero-filled buffers for the whole recording — with no error at any point). What we've ruled out: microphone permission is granted; it's not truncation or short capture (full-length, full frame count); it's not our encoding step (the input buffers themselves are zero); it's not a quiet/obstructed mic (that would be low noise, not exact 0.0). Questions: What other device states or scenarios can cause a running AVAudioEngine input tap to deliver all-zero buffers with a valid format and no error? (e.g. another process/system feature holding the mic, Continuity Camera/Mic, CallKit/PushToTalk session ownership, etc.) Since this surfaces with no error and a valid format, what is the recommended way to detect it at runtime? Is monitoring the input level / PCM energy the only signal, or is there a supported API to know the input isn't actually live? What's the recommended recovery once detected — is a full session deactivate/reactivate re-handshake sufficient, or is recreating the engine required?
Replies
1
Boosts
0
Views
226
Activity
3w
AVAudioSession microphone recording changes Bluetooth audio route in vehicle
Title AVAudioSession microphone recording changes Bluetooth audio route in vehicle Hi everyone, I’m developing an iOS app called HearSave to help drivers safely remember and save songs they hear while driving. The app performs a very simple task: the user presses a button, the app records a short audio sample for music recognition, then immediately stops recording. I’m trying to understand whether the behaviour I’m seeing is an expected limitation of iOS audio routing or whether there is a better AVAudioSession configuration. Test setup iPhone connected to a vehicle via Bluetooth. Vehicle playing DAB radio. App requests microphone access and starts recording. Recording lasts approximately 7 seconds. Behaviour observed As soon as recording begins: The vehicle changes into a voice-call style audio route. DAB audio becomes muted. The vehicle’s volume control changes to voice volume. When recording finishes, the radio does not automatically recover until the user manually changes the audio source away from DAB and back again. I’ve experimented with several Expo Audio / AVAudioSession configurations, including releasing the audio session immediately after recording and adjusting audio session options, but the behaviour remains the same. My questions Is this expected behaviour when an iOS app starts microphone recording while connected to a vehicle over Bluetooth? Is there an AVAudioSession configuration that allows brief microphone capture without causing the vehicle to switch audio routes? Is it possible to explicitly use the iPhone’s built-in microphone while remaining connected to the vehicle via Bluetooth? If this behaviour is unavoidable over Bluetooth, is this also expected behaviour for native CarPlay apps, or are different audio routing capabilities available to CarPlay applications? Any guidance or documentation would be greatly appreciated. Thank you.
Replies
0
Boosts
0
Views
340
Activity
4w
Best practice for rapid sequential Live Photo captures with AVCapturePhotoOutput?
Hi everyone, I’m working on a camera app as a learning project and have reached a point where I’m trying to better understand the intended architecture for Live Photo capture using AVCapturePhotoOutput. The app currently supports: Live Photos Depth data Location metadata Multiple lens presets on a virtual multi-camera device Everything is working well, but I’m now thinking about capture throughput and rapid shutter presses. Right now, my implementation is fairly conservative. I wait for a Live Photo capture to finish processing and importing before allowing another capture. This is reliable, but it doesn’t feel particularly camera-like when compared to Apple’s Camera app. One observation from field testing caught my attention: I took a Live Photo, immediately switched lenses, then took another Live Photo. When I viewed the first Live Photo later, the movie portion included the lens-switching actions that occurred after I pressed the shutter. That made me realize that I may be thinking about the capture lifecycle incorrectly. My questions are: When using AVCapturePhotoOutput with Live Photos enabled, what is the earliest point at which a capture can be considered “safely secured”? Is it expected that apps wait for PhotoKit import to complete before accepting another Live Photo capture request? If supporting rapid sequential shutter presses, is the recommended approach to queue capture requests and process them one at a time? Are there any best practices around lens changes or camera reconfiguration while a Live Photo is still being captured or processed? I’m not looking for details about the implementation of Apple’s Camera app. I’m mainly trying to understand the recommended approach when working with the public AVFoundation APIs. I’d appreciate any guidance, documentation references, or examples from developers who have worked through similar problems. Thanks!
Replies
1
Boosts
0
Views
400
Activity
4w
Safari 26.5 Web Audio API Glitch: Automatic background audio playback triggering fast-forwarded "buffer catch-up" burst
https://streamable.com/y16s4q When I leave the Spotify Web Player open on its homepage and remain completely idle (without any physical user interaction or clicks), the page somehow self-triggers an active audio stream. Instantly, Safari's standard "This tab is playing audio" speaker icon appears normally on the right side of the URL/Address bar. Simultaneously, the browser emits a 1–2 second burst of extremely accelerated, fast-forwarded, and chaotic audio (resembling a time-stretch or "Alvin and the Chipmunks" effect) before going silent or stabilizing.
Replies
1
Boosts
0
Views
401
Activity
4w
CNAssetSpatialAudioInfo / Audio Mix rejects ProRes spatial captures (LPCM FOA) — only HEVC (APAC) is eligible. Intended? And how can Audio Mix coexist with ProRes recording?
On iOS/macOS 26, CNAssetSpatialAudioInfo(asset:) and CNAssetSpatialAudioInfo.assetContainsSpatialAudio(asset:) accept a spatial capture only when its spatial track is APAC-encoded which AVCaptureMovieFileOutput produces when the video codec is HEVC. An otherwise-identical ProRes capture, whose spatial track is LPCM (4-channel First-Order Ambisonics, kAudioChannelLayoutTag_HOA_ACN_SN3D | 4), is rejected with CNCinematicErrorDomain code 3 (CNCinematicErrorCodeIncomplete) "no eligible audio tracks in asset". This reproduces with Apple's own SpatialAudioCLI sample run on Apple's own stock iPhone captures, so it appears to be a property of the format/API rather than my code. I'd like to confirm whether this is intended, and find a supported way to obtain Audio-Mix-eligible spatial audio while still recording ProRes video (we use Apple Log/ProRes for color grading). Respectively, is wiring a manual AVAssetWriter setup the only way to manage spatial audio and ProRes video? Eligibility appears to require an APAC-encoded, exactly-4-channel FOA track. Because AVCaptureMovieFileOutput only writes APAC audio for HEVC (ProRes forces LPCM), ProRes spatial captures are never eligible — including Apple's own ProRes stock captures, which SpatialAudioCLI also rejects. Key finding: eligibility seems baked into the native APAC bitstream Starting from an eligible HEVC/APAC file, I used AVAssetReader/AVAssetWriter to re-encode only the FOA track (APAC → LPCM → APAC), leaving the AAC stereo track, the HEVC video, and the timed-metadata track untouched. The structurally-identical output is then rejected (code 3). Preserving the cinematic-audio metadata track is not sufficient. Re-encoding the APAC itself loses eligibility. This suggests the mix metadata that gates eligibility is carried inside the APAC bitstream and is produced only at capture time. Questions Is it intended that ProRes (LPCM FOA) spatial captures are not Audio-Mix-eligible via CNAssetSpatialAudioInfo, while HEVC (APAC FOA) captures are? Is this documented? Where exactly is the eligibility metadata stored — in the APAC bitstream, or in the cinematic-audio timed-metadata track (Re-encoding the APAC while preserving that metadata track still loses eligibility)? Is there any supported way to make an existing LPCM/ProRes FOA capture eligible after the fact (a transcode/encode path that produces the required APAC), or is native capture the only source? Any guidance, or a pointer to documentation, would be greatly appreciated. Thank you. Environment iPhone 16 Pro Max, iOS 26.x; macOS 26.2; Xcode 26.2.
Replies
1
Boosts
0
Views
418
Activity
Jun ’26