AVFoundation

RSS for tag

Work with audiovisual assets, control device cameras, process audio, and configure system audio interactions using AVFoundation.

Posts under AVFoundation tag

200 Posts

Post

Replies

Boosts

Views

Activity

Where is the API for variable aperture of the iPhone 18 Pro?
According to Apple's press release (https://www.apple.com/pt/newsroom/2026/09/apple-debuts-iphone-18-pro-and-iphone-18-pro-max/): Variable aperture also provides more control for creative pros by giving them the ability to manually adjust any of the four aperture settings in the Camera app, and an API is available to developers for even more control across the aperture range in their apps. Can a DTS engineer specify the quoted API please? I can't see it anywhere in the documentation yet. What does it mean "even more control across the aperture range"? Will it be possible to set the aperture in a 0.0-1.0 range?
3
3
1.4k
Sep ’26
Picture-In-Picture not starting with Custom Player
Team, I am trying to start PictureInPicture for my video from custom player as mentioned in the following link, but I am getting an error Steps: Created a player using AVPlayer. Initiated AVPictureInPictureController with AVPlayerLayer Added observer on PIP button enable status When PIP button is tapped executed pipController.startPictureInPicture() statement. PIP is not getting started & getting following error Project created in Xcode11 & currently using Xcode13.2 Error Domain=AVKitErrorDomain Code=-1001 "Failed to start picture in picture." UserInfo={NSLocalizedDescription=Failed to start picture in picture., NSLocalizedFailureReason=The UIScene for the content source has an activation state other than UISceneActivationStateForegroundActive, which is not allowed.} I didn't get any solutions online, plz help me on this issue. FYI: PIP button/mode is working if I use AVPlayerViewController(with default playerItem buttons) instead of AVPlayer & AVPictureInPictureController with custom controls Thanks in advance. Adopting Picture in Picture in a Custom Player
3
2
1.8k
Sep ’26
`LockedCameraCaptureManager` practically unusable since iOS 26
Somewhere since iOS 26, the LockedCameraCapture framework gets in an unpredictable state after opening the main app from the LockedCamera extension using LockedCameraCaptureSession.openApplication(for userActivity:). (Feedback with sample code to reproduce: FB21966835) Opening the extension from the lock screen again doesn’t open the extension but puts the lock screen in a state as if it has. Content updated from LockedCameraCaptureManager.shared.sessionContentUpdates comes in inconsistently, usually needs the app to be opened again or the extension to be opened. This makes using this extension impossible for me as I use it to record video files that manually need to be imported when the app is launched (so not through PhotoKit). Does anybody have a suggestion to circumvent this issue or how to get this fixed?
2
3
1.5k
Sep ’26
AVPlayerItemSampleBufferOutput
I am using AVPlayer, but I am unable to retrieve PCM data from .m3u8 streams via the AVPlayerItemSampleBufferOutput API in iOS 27 beta. I have tried both the Objective-C and Swift APIs: the Objective-C delegate methods are not being called, and the Swift methods nextAvailableSampleBuffer() and nextSampleBuffer() are returning no data.
2
1
833
Sep ’26
fail to get HLS realtime stream via AVPlayerItemSampleBufferOutputDelegate
Hi, I'm trying to do some realtime audio processing on audio served from an HLS stream (i.e. an AVPlayer created using an M3U HTTP URL). And I find that new APIs(AVPlayerItemSampleBufferOutput) are available in iOS 27.0 to achieve it. However, I failed to get the available data via the AVPlayerItemSampleBufferOutputDelegate. And I found some error in the system log: I create AVPlayerItemSampleBufferOutput and set the delegate after receiving the AVPlayerItemStatusReadyToPlay event. And here's my code: @interface OCAudioSamplebuffer () <AVPlayerItemSampleBufferOutputDelegate> @property (nonatomic, strong) AVPlayerItemSampleBufferOutput *bufferOutput; @property (nonatomic, strong) dispatch_queue_t bufferOutputQueue; @property (nonatomic, strong) AVPlayerItem *playerItem; @end - (void)playItem:(AVPlayerItem *)item { if (@available(iOS 27, *)) { AVAudioSession *audioSession = [AVAudioSession sharedInstance]; if([NSThread mainThread]){ [audioSession setCategory:AVAudioSessionCategoryPlayback withOptions:AVAudioSessionCategoryPlayback error:nil]; [audioSession setActive:YES error:nil]; }else{ dispatch_async(dispatch_get_main_queue(), ^{ [audioSession setCategory:AVAudioSessionCategoryPlayback withOptions:AVAudioSessionCategoryPlayback error:nil]; [audioSession setActive:YES error:nil]; }); } AVPlayerItemSampleBufferOutputAudioConfiguration *cfg = [[AVPlayerItemSampleBufferOutputAudioConfiguration alloc] init]; CMFormatDescriptionRef formatDescription = [self createPCMFormatDescriptionWithSampleRate:44100.0 channels:2 isFloat:YES]; cfg.requestedAudioFormat = formatDescription; if (formatDescription) { NSLog(@"create PCM Format Description success"); CFRelease(formatDescription); } else { NSLog(@"fail to create PCM Format Description"); } self.bufferOutput = [[AVPlayerItemSampleBufferOutput alloc] initWithConfiguration:cfg]; NSLog(@"create buffer success, PCM Format Description%@---,%@", cfg.requestedAudioFormat, formatDescription); self.bufferOutputQueue = dispatch_queue_create("audioSamplebufferQueue", DISPATCH_QUEUE_CONCURRENT); [self.bufferOutput setDelegate:self queue:self.bufferOutputQueue]; [item addOutput:self.bufferOutput]; self.playerItem = item; } else { // Fallback on earlier versions } } Any help. Thank you
2
0
1.1k
Sep ’26
How can an app detect a TrueDepth pipeline failure when no depth output or drop callback is delivered?
How can an app detect a TrueDepth pipeline failure when no depth output or drop callback is delivered? We are using AVFoundation to stream synchronized RGB and depth data from the front-facing TrueDepth camera. On a small number of physical devices, the capture session and RGB stream remain healthy, but the depth pipeline becomes silent or returns unusable data. We would like to understand the intended public-API behavior and the supported way to detect this state without relying on private system logs or an application-defined timeout. We need an AVFoundation-only solution rather than ARKit. Capture configuration Our setup follows Apple's TrueDepth streaming sample: Discover .builtInTrueDepthCamera. Select an activeFormat with a nonempty supportedDepthDataFormats list. Set a supported activeDepthDataFormat, typically DepthFloat16. Add AVCaptureVideoDataOutput and AVCaptureDepthDataOutput. Enable the .depthData connection. Test both direct depth delegate delivery and AVCaptureDataOutputSynchronizer. Observe capture-session runtime errors, interruptions, connection state, and system pressure. The same code and configuration continuously deliver valid RGB and depth frames on healthy control devices. Observed failure modes 1. No depth callback of any kind The video delegate continues to receive RGB frames at the expected rate. The depth connection reports enabled and active. The capture session remains running. No AVCaptureSessionRuntimeErrorNotification or interruption is reported. System pressure is nominal. depthDataOutput(_:didOutput:timestamp:connection:) is never called. depthDataOutput(_:didDrop:timestamp:connection:reason:) is also never called. With AVCaptureDataOutputSynchronizer, RGB synchronization points may continue, but no AVCaptureSynchronizedDepthData object is present for the depth output. 2. One depth frame, followed by permanent silence On other affected devices, one depth frame is delivered shortly after startup, while hundreds of later RGB frames continue with no further depth output or drop callback. 3. A depth object is delivered, but the map is unusable We have also observed an AVDepthData object whose map is empty or has no usable spatial information, for example: a zero-sized/empty depth representation; an unfiltered map containing only NaN; or a filtered map containing a single constant value across nearly all pixels. Metadata such as depth quality, accuracy, calibration availability, supported formats, and connection state does not reliably distinguish these frames from healthy ones. Relevant documented behavior The documentation for AVCaptureSynchronizedDepthData.depthDataWasDropped says that a dropped depth object is different from a synchronization timestamp for which no depth capture occurred. In the latter case, no AVCaptureSynchronizedDepthData object is present in the collection. The documentation for depthDataOutput(_:didDrop:timestamp:connection:reason:) says that the callback reports captured depth data that wasn't processed, with an empty-shell AVDepthData object. This appears to leave a diagnostic gap: if the TrueDepth pipeline fails before a depth object is produced, the application may receive neither an output callback nor a drop callback. System-log correlation We do not want to depend on these private and undocumented strings, but sysdiagnose/device logs from affected devices correlate the public-API silence with messages such as: Can't activate Pearl projector: no projector token found PROJECTOR ON DENIED Object too close (no depth) GMC status: -2 Projector GMC hasn't completed yet - dropping depth/dx buffers Pulses exceed limit. Projector Off In the first group, depth delivery never starts. In the second group, depth may never start or may stop after one frame. RGB capture can remain active in both cases. These logs suggest that depth or disparity buffers can be rejected before they reach AVCaptureDepthDataOutput, which would explain why the application sees neither didOutput nor didDrop. However, none of these projector, IR, calibration, protection, or token states appears to be exposed by a public API. Questions Is it expected that AVCaptureDepthDataOutput delivers neither didOutput nor didDrop when no depth object is produced upstream? Is the absence of AVCaptureSynchronizedDepthData from a synchronization collection the only public signal for this condition? Is there a supported API, notification, KVO property, or error that reports that the TrueDepth IR/projector/depth provider is unavailable, disabled, protected, stalled, or unable to produce data? Can an app distinguish a healthy but temporarily delayed depth stream from a pipeline that will never deliver depth, other than by implementing its own timeout and frame-count watchdog? Is there a documented validity rule for an empty, all-NaN, or near-constant AVDepthData.depthDataMap? Should applications treat these as unavailable depth even when the AVDepthData metadata reports a normal quality or accuracy? What is the recommended recovery procedure: rebuild AVCaptureDepthDataOutput, restart the capture session, switch cameras and switch back, wait for a system notification, or stop retrying until the device state changes? ARKit can report ARError.sensorFailed when a required sensor does not deliver input. Is there an AVFoundation equivalent that identifies which required input failed, without adopting ARKit? If no public signal currently exists, would Apple consider exposing a depth-provider status/error callback that distinguishes at least: captured but dropped; no depth capture for the timestamp; provider temporarily unavailable; and provider disabled or failed until recovery/service? We can provide a minimal sample project, affected-device sysdiagnose, and synchronized timestamp traces in Feedback Assistant if that would help. Please let us know which diagnostics and logging profiles would be most useful.
0
0
192
Sep ’26
AVCaptureVideoDataOutput stops zooming while AVCaptureVideoPreviewLayer continues — physical wide / ultra-wide / telephoto only
We use a single AVCaptureSession with AVCaptureVideoPreviewLayer and AVCaptureVideoDataOutput (preview-sized buffers, BGRA). When we increase videoZoomFactor, beyond a certain zoom level the image from AVCaptureVideoDataOutput no longer zooms further, while AVCaptureVideoPreviewLayer continues to zoom with the same zoom control. The preview and the video-data output therefore diverge. This behavior appears when the active camera is a physical lens device — wide, ultra-wide, or telephoto (e.g. builtInWideAngleCamera, builtInUltraWideCamera, builtInTelephotoCamera, or similar). It does not appear when the active input is a virtual / multi-camera (e.g. triple camera, dual-wide, or other system multi-camera). Are there known conditions under which this mismatch between preview and video-data output is expected? Thank you.
2
0
1.1k
Sep ’26
AVAssetDownloadConfiguration does not download supplemental tracks on iOS 16
We're downloading HLS content for offline playback using AVAssetDownloadConfiguration. It's critical for our app that all subtitle tracks are downloaded together with the video, so users can watch any subtitle and audio option while offline. On iOS 16, not all tracks are downloaded, and when the user goes offline, playback fails with NSURLErrorNotConnectedToInternet (error -1009) because AVFoundation tries to fetch the missing subtitle segments over the network. On iOS 17 and later, everything works as expected — all subtitle and audio tracks are downloaded. However, on iOS 16 we observe two issues: First issue: Supplemental tracks are not downloaded on iOS 16 (device) Only the default audio track is downloaded. All non-default supplemental tracks — both subtitles and additional audio tracks — exhibit the same problem: Some supplemental tracks listed in the manifest don't even get a directory created in the .movpkg package Some supplemental tracks do get a directory created, but it contains only StreamInfoBoot.xml and StreamInfoRoot.xml — no actual segment data (.frag files) is downloaded. The StreamInfoBoot.xml for these directories shows NO, 0, and 0, with Supplemental Additionally, a Manifest.m3u8 stream directory is created with Complete=NO and no segment data. On iOS 17+ devices, all supplemental track directories are created and all contain segment data. Second issue: Downloads fail entirely on iOS 16 Simulator We receive the following error: Error Domain=NSURLErrorDomain Code=-16090 UserInfo={ NSLocalizedDescription=The operation could not be completed, _NSURLErrorFailingURLSessionTaskErrorKey=BackgroundAVAssetDownloadTask <DEC0D555-6C05-416C-91B8-F490F5A29046>.<3>, _NSURLErrorRelatedURLSessionTaskErrorKey=( "BackgroundAVAssetDownloadTask <DEC0D555-6C05-416C-91B8-F490F5A29046>.<3>" ), NSLocalizedFailureReason=An unknown error occurred (-16090) } On iOS 17+ Simulators, downloads work without issues. Our setup: let urlAsset = AVURLAsset(url: manifestURL) let preferredMediaSelection = try await urlAsset.load(.allMediaSelections) let config = AVAssetDownloadConfiguration(asset: urlAsset, title: title) config.primaryContentConfiguration.mediaSelections = preferredMediaSelection let task = assetDownloadURLSession.makeAssetDownloadTask(downloadConfiguration: config) task.resume() Example HLS manifest: #EXTM3U #EXT-X-VERSION:7 #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="French",LANGUAGE="fr",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_25/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="French (forced)",LANGUAGE="fr-x-cc",DEFAULT=NO,AUTOSELECT=YES,FORCED=YES,URI="s_31/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",LANGUAGE="en",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_37/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Kazakh",LANGUAGE="kk",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_43/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Uzbek",LANGUAGE="uz",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_49/stream.m3u8" #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aud_ec3",NAME="French 5.1",LANGUAGE="fr",DEFAULT=YES,AUTOSELECT=YES,CHANNELS="6",URI="a_18/stream.m3u8" #EXT-X-STREAM-INF:AUDIO="aud_ec3",SUBTITLES="subs",BANDWIDTH=1931408,VIDEO-RANGE=SDR,CODECS="hvc1.1.6.L90.90,ec-3",RESOLUTION=1024x428,FRAME-RATE=25,HDCP-LEVEL=NONE v_6/stream.m3u8 #EXT-X-SESSION-KEY:METHOD=SAMPLE-AES,URI="skd://XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",KEYFORMAT="com.apple.streamingkeydelivery",KEYFORMATVERSIONS="1" Questions: What is the correct way to make sure all supplemental tracks (subtitles, non-default audio) are actually downloaded on an iOS 16 real device? We pass asset.load(.allMediaSelections) to primaryContentConfiguration.mediaSelections — is there a different configuration, or should we use auxiliaryContentConfigurations, or something else entirely? Is there any way to make asset downloads work on the iOS 16 Simulator at all? We always get NSURLErrorDomain error -16090, while iOS 17+ Simulators work fine. If the supplemental tracks download bug on iOS 16 cannot be fixed or worked around from the app side, do you have any recommended approach for handling incomplete .movpkg packages at offline playback time?
1
0
534
Sep ’26
Variable aperture on the iPhone 18 Pro
Does iOS 27 provide an API which allows developers to set the iPhone 18 Pro camera's aperture? If it does, is the aperture going to be limited to just 4 discrete settings (like in the Camera app and like mentioned in the Tech Specs for the 18 Pro) or does the hypothetical iOS API allow us to set the aperture to a continuous value in the 0.0–1.0 range (similar to how a camera's focus can be set)?
2
1
889
Sep ’26
Capturing 48mp photos with .builtInTripleCamera
I am able to capture 48mp photos using .builtInWideAngleCamera, but it seems like .builtInTripleCamera is capped at 12mp? Is there a way to capture 48mp photos using .builtInTripleCamera? Because .builtInTripleCamera provides smooth transition between cameras during zooming, and I'd like to keep this behavior. New iPhone 17 Pro have all their cameras at 48mp. Is there a chance that their .builtInTripleCamera is capable of capturing 48mp? Or is this an API limitation?
4
0
1.2k
Sep ’26
Working with kCVPixelFormatType_96VersatileBayerPacked12
Whilst AVCaptureSession is setup to capture ProRes RAW video, is it possible to get video pixel data which can read and processed, such as using CIImage(cvPixelBuffer: ) AVCaptureVideoDataOutput outputs ProRes RAW in kCVPixelFormatType_96VersatileBayerPacked12 pixel format. Is there a provided way to debayer this pixel format into something more usable?
1
1
1.2k
Sep ’26
visionOS Developer Strap: UVC video works, but how can I access the capture device’s USB Audio Class (UAC) audio?
I’m developing a native visionOS application that uses an external HDMI capture device connected to Apple Vision Pro through the Developer Strap. The goal is to use Apple Vision Pro as a low-latency display for external HDMI sources such as an Xbox Series X, including both the HDMI video and the original digital game audio. My current hardware path is: Xbox Series X → HDMI → Elgato 4K X → USB-C → Apple Vision Pro Developer Strap The Elgato 4K X receives both video and audio from the Xbox over HDMI. What is working UVC video capture is working successfully on visionOS. The external Elgato capture device can be used as the video source and the Xbox video is displayed correctly inside the Vision Pro application. I have also verified the Elgato 4K X independently on macOS, where the device is recognized correctly. The problem I cannot find a supported way on visionOS to access the USB Audio Class (UAC) audio interface associated with the capture device. I specifically need the digital game audio originating from: Xbox HDMI → Elgato 4K X → USB/UAC → visionOS I do not want to use the Apple Vision Pro’s built-in microphone. Selecting/capturing the Vision Pro microphone would only record environmental sound and is not a substitute for the digital HDMI audio stream. Apple’s visionOS documentation for external devices through the Developer Strap appears to focus on UVC video capture. I haven’t been able to determine whether the accompanying UAC interface of a UVC/UAC composite capture device is exposed to visionOS applications. What I’ve tried / verified Connected an Xbox Series X to the HDMI input of an Elgato 4K X. Connected the 4K X to Apple Vision Pro through the Developer Strap. Successfully discovered and received the external UVC video stream. Successfully displayed the Xbox video feed in visionOS. Confirmed that the 4K X itself is recognized and functional on macOS. Attempted to obtain audio corresponding to the external capture source. The Vision Pro microphone is available as an audio source, but this is not the audio source required. I have not found a documented visionOS API that exposes the 4K X’s USB/UAC audio endpoint through the Developer Strap. Questions Does visionOS currently support USB Audio Class input devices connected through the Apple Vision Pro Developer Strap? If a device exposes both UVC video and UAC audio interfaces, is there a supported way to associate/access both interfaces from a visionOS application? If UAC is supported, which framework/API should be used to discover the external audio endpoint — AVFoundation, Core Audio, or another API? Does accessing the UAC interface require an entitlement in addition to UVC device access? If generic UAC access through the Developer Strap is currently unavailable to third-party visionOS applications, is there a restricted/managed entitlement that developers can request? If this isn’t currently supported, what is Apple’s recommended architecture for receiving synchronized video and audio from an HDMI capture device on Apple Vision Pro? Ultimately, I’m trying to achieve: HDMI source → UVC video + UAC audio → visionOS application → video displayed in Vision Pro + audio played through Vision Pro speakers/AirPods Since UVC video from this exact hardware is already working, I’m primarily trying to determine whether access to the accompanying USB audio interface is possible with the current visionOS SDK, requires another entitlement/API, or is currently unsupported. Any guidance from Apple or developers who have worked with composite UVC/UAC devices on Vision Pro would be greatly appreciated.
0
1
827
Sep ’26
iOS 27 beta: setExposureModeCustom silently ignored on the front camera
On every iOS 27.0 beta, setExposureModeCustom has no effect on the front camera (.builtInTrueDepthCamera). exposureMode does change to .custom, but the sensor keeps auto-exposing: device.iso and device.exposureDuration report auto-exposure's values, not the requested ones, and frame brightness does not change. No error is thrown. The same code works on iOS 18.x and 26.x on the same devices. Device: iPhone 15 Pro, every iOS 27.0 beta. Also seen on iPhone 14 Pro Max, 16, 16 Pro Max and 17 Pro Max. Session: front camera, AVCaptureVideoDataOutput (BGRA) + AVCaptureDepthDataOutput. Format advertises ISO 20...1920, duration 0.00004...0.06667 s. isExposureModeSupported(.custom) is true. Repro: let auto-exposure settle, then try device.lockForConfiguration() device.setExposureModeCustom(duration: CMTime(value: 1, timescale: 15), iso: 1920, completionHandler: nil) device.unlockForConfiguration() then poll device.iso / device.exposureDuration (the completion handler never fires on this device) and measure frame luma. Readback, polled 0.8 s after the call: requested ISO 1920, 66.7 ms -> reported ISO 349, 60.0 ms requested ISO 20, 0.04 ms -> reported ISO 188, 66.5 ms Requesting the shortest exposure returns the longest, and the same ISO 1920 request returned 249, 349 and 450 on three consecutive runs, so these are auto-exposure's values. Frame luma stayed within 1% between the two phases. setExposureTargetBias, however, is honored on the same device: at +4 EV auto-exposure drives the sensor to ISO 1920 / 66.5 ms and at -4 EV to ISO 24 / 30.0 ms, moving mean luma from 49 to 232. So the sensor does reach the values setExposureModeCustom cannot set. Question: is this intentional or a regression, and how should an app detect that a custom exposure request was refused?
2
0
873
Sep ’26
Does the TN3135 audio-session networking exception have a defined lifetime? Seeing a ~38.5 s revoke/re-grant cycle
TN3135 describes the exception that lets a watchOS app use low-level networking while it holds an active audio session. I have that working, and the app functions — but the network path is withdrawn and restored on a strikingly regular cycle, and I would like to know whether that is expected behaviour rather than something I am doing wrong. Setup Apple Watch Series 10 (Watch7,9), watchOS 26.5. Reproduced on a Series 6 (Watch6,2). UIBackgroundModes: [audio]; AVAudioSession category .playAndRecord, mode .spokenAudio; activated with the async activate(options:completionHandler:). NWConnection with NWProtocolWebSocket to a WebSocket relay over TLS. The app streams 16 kHz mono PCM continuously while transmitting and holds the socket open otherwise. Symptom NWPathMonitor reports .unsatisfied, then .satisfied about two seconds later, over and over. Measured with the iPhone powered off, so the watch was on its own Wi-Fi: Uptime between drops Outage 36.4 s 2.1 s 36.7 s 1.9 s 36.9 s 2.1 s The regularity is what prompts the question — uptime varies by ±0.3 s and the outage is consistently 2.0 s. That reads as a timeout expiring rather than radio behaviour. What I have ruled out Not the network or the server. A browser client on the same relay, same TLS, same wire protocol, holds a WebSocket indefinitely. Not the interface. Identical cadence over the companion ipsec1 tunnel with the iPhone present, and over the watch's own en0 with the iPhone powered off. Pinning requiredInterfaceType = .wifi while the iPhone is reachable fails outright — the path offers only ipsec1. Not audio-session interruption. I observe interruptionNotification, routeChangeNotification, mediaServicesWereResetNotification and silenceSecondaryAudioHintNotification. None fire at a drop. At the moment the path goes .unsatisfied, the engine is running and the player node is actively playing. Not session idleness. Playing continuous silence for the whole session, rather than only while reconnecting, made no difference — still 36.4 s. The control that surprised me To test whether this affects any long-lived watch socket or only audio-unlocked ones, I built a second app with no AVAudioSession at all, no audio background mode, holding a URLSessionWebSocketTask and kept alive by a WKExtendedRuntimeSession so screen sleep was not a factor. It never connected. NWPathMonitor reported .unsatisfied once and never changed, across a 30 s run, and every request failed with "The Internet connection appears to be offline." I had expected URLSession to be permitted regardless. Questions Does the audio-session networking exception in TN3135 have a defined lifetime, and is a periodic revoke/re-grant cycle expected? If so, is there a supported way to hold it continuously — or is the correct design simply to expect the interruption and reconnect through it? Is it expected that an app with no audio session gets no network path at all on watchOS, including via URLSession, even in the foreground with an extended runtime session?
16
0
4.1k
Sep ’26
AirPods Sleep Detection pauses meditation playback with no API/callback to identify the reason
Good morning! I work on a meditation app, and since Sleep Detection was introduced for AirPods in iOS 26, we've been receiving complaints from users reporting that their meditation sessions unexpectedly stop after approximately 15–20 minutes when Sleep Detection is enabled. This seems to happen particularly during deep meditation sessions, where the user may remain still and exhibit behavior that could potentially be interpreted as falling asleep. I've also tested this on iOS 27 with the latest beta firmware available for AirPods Pro, and the same behavior still occurs. While investigating and debugging the issue, I found that there doesn't appear to be any public API that allows an app to determine that playback was paused specifically because of AirPods Sleep Detection. From the application's perspective, the AirPods appear to simply send a standard pause command to the AVPlayer, without providing any additional context or reason. There also doesn't seem to be an API that allows us to determine whether the user has Sleep Detection enabled. Even having access to that information would allow us to warn users that their meditation session could potentially be interrupted. AVAudioSession.InterruptionReason does not seem to help in this case either: https://developer.apple.com/documentation/avfaudio/avaudiosession/interruptionreason Is there currently any supported way for an app to: Detect that playback was paused specifically because of AirPods Sleep Detection? Determine whether Sleep Detection is enabled for the connected AirPods? Prevent Sleep Detection from pausing playback for specific apps or specific types of audio sessions? If none of these are currently possible, could Apple consider providing either a notification/callback indicating that Sleep Detection triggered the pause, or an option for users to exclude specific apps from Sleep Detection? This behavior is particularly problematic for meditation apps, since a user in a deep meditation session can easily be mistaken for someone who has fallen asleep. Other meditation apps appear to be affected by the same behavior as well. Best regards, Carlos Antunes
1
0
1.4k
Sep ’26
Can an iOS app programmatically enable Voice Isolation microphone mode?
Hi, I’m developing an iOS app that captures the user’s voice and sends the microphone input to an AI model for speech processing. For this use case, Voice Isolation provides a significantly better input signal in noisy environments. However, I’m currently unable to enable Voice Isolation programmatically when the app starts recording. As I understand it, microphone modes such as Automatic, Standard, Voice Isolation, and Wide Spectrum are controlled by the user through the system UI. My app can detect/observe the current microphone mode, but I have not found a public API that allows the app to select Voice Isolation itself. My questions are: Is there currently a supported public API to programmatically switch the microphone mode from Automatic/Standard → Voice Isolation? If direct selection is not supported, is there a recommended way for an app that primarily processes speech/voice input for an AI model to request or prefer Voice Isolation? Would using AVAudioSession with an appropriate voice-processing configuration, such as .voiceChat, or enabling voice processing through AVAudioEngine, provide equivalent Voice Isolation processing? Is AVCaptureDevice.showSystemUserInterface(.microphoneModes) the intended approach when an app needs the user to change the microphone mode? I understand that Apple may intentionally keep microphone-mode selection under user control. I’m mainly trying to determine whether there is a supported programmatic approach that I may have overlooked, rather than relying on a custom noise-suppression implementation. Any guidance on the recommended architecture for this use case would be greatly appreciated. Thank you!
0
0
795
Sep ’26
AVCaptureDevice.uniqueID for UVC devices is unstable - bug or overstated documentation?
The documentation for AVCaptureDevice.uniqueID states the following: Capture devices have a unique identifier that persists on one system across device connections and disconnections, application restarts, and reboots of the system itself. You can store the value returned by this property to recall or track the status of a specific device in the future. For UVC capture devices this documentation does not hold. The video uniqueID is a hex string of the form 0x<locationID><vendorID><productID>, and the identifying half is the locationID (bus number plus port path). Which identifies a port, not a device. I ran a suite of tests with three identical Elgato 4K X capture cards connected to a Mac Studio w/ M3 Ultra running macOS 26.5.2, and reproduced my findings on a MacBook w/ M3 Pro (same macOS version). See the script at the bottom of the post for how uniqueId & USB serial number are being retrieved. 1. The uniqueID follows the port. Swapping two cards between two built-in ports swaps their uniqueIDs: # Before swap. 4K X uid=0x2000000fd9009b serial=A7SNB50424UBQI 4K X uid=0x12000000fd9009b serial=A7SNB504219J0R # After swapping the cards between the same two ports. 4K X uid=0x2000000fd9009b serial=A7SNB504219J0R 4K X uid=0x12000000fd9009b serial=A7SNB50424UBQI An app that stored 0x2000000fd9009b to recall a specific capture card now silently opens another. 2. A reboot alone can swap uniqueIDs. External USB controllers (here, PCIe USB cards in two Thunderbolt enclosures) can race for bus numbers at boot, so with every cable left in place, a reboot swapped two of the cards: # Before reboot. 4K X uid=0x262000000fd9009b serial=A7SNB504219J0R 4K X uid=0x252000000fd9009b serial=A7SNB50423R73R # After reboot, no cables touched. 4K X uid=0x262000000fd9009b serial=A7SNB50423R73R 4K X uid=0x252000000fd9009b serial=A7SNB504219J0R This behavior is intermittent, a second reboot changed nothing, but a third caused another swap. Cards left alone in built-in ports retain their uniqueIDs across reboots in my testing; the failure requires dynamically enumerated external USB controllers. 3. Even the product ID tail can drift. One unit intermittently enumerates with idProduct 0x009c instead of 0x009b, same port (USB PCIe card in a Thunderbolt enclosure), cables untouched: # Before reboot. 4K X uid=0x222000000fd9009b serial=A7SNB50424UBQI # After reboot. 4K X uid=0x222000000fd9009c serial=A7SNB50424UBQI IOKit and AVFoundation agree each boot... So the change is upstream of both? I'm uncertain where to place blame for this specific issue (UVC device or macOS). Audio on the same physical units is unaffected. The audio uniqueID (AppleUSBAudioEngine:...:<serial>:...) embeds the USB serial and stayed stable through every test. So AVCaptureDevice can provide a stable per-device identifier, just not for UVC video devices. Questions: Is this a bug, or is the documentation overstating the persistence guarantee for USB video devices? What is the supported way to identify a specific physical UVC video device across reboots and port changes? The USB serial number is stable and is what I've fallen back on via IOKit, but there is no documented AVFoundation API to retrieve USB serial number from a UVC video AVCaptureDevice. Related: thread 803759, where the locationID-derived format is described. Script used for all output above (swift ./list-uvc.swift): import AVFoundation import IOKit func usbSerial(forLocation location: UInt32) -> String? { var iterator: io_iterator_t = 0 guard IOServiceGetMatchingServices(kIOMainPortDefault, IOServiceMatching("IOUSBHostDevice"), &iterator) == KERN_SUCCESS else { return nil } defer { IOObjectRelease(iterator) } var result: String? var service = IOIteratorNext(iterator) while service != 0 { var loc: UInt32 = 0 if let ref = IORegistryEntryCreateCFProperty(service, "locationID" as CFString, kCFAllocatorDefault, 0)?.takeRetainedValue(), let num = ref as? NSNumber { loc = num.uint32Value } if loc == location, let ref = IORegistryEntryCreateCFProperty(service, "USB Serial Number" as CFString, kCFAllocatorDefault, 0)?.takeRetainedValue(), let serial = ref as? String { result = serial } IOObjectRelease(service) if result != nil { break } service = IOIteratorNext(iterator) } return result } let session = AVCaptureDevice.DiscoverySession(deviceTypes: [.external], mediaType: .video, position: .unspecified) for device in session.devices { let uid = device.uniqueID let location = UInt32(truncatingIfNeeded: strtoull(uid, nil, 16) >> 32) let serial = usbSerial(forLocation: location) ?? "N/A" print("\(device.localizedName) uid=\(uid) serial=\(serial)") }
6
0
2.1k
Sep ’26
PhotoKit IMG_XXXX names for videos?
I save stills and videos with PHAssetCreationRequest. I do not set originalFilename. Photos saved with addResource(with: .photo, data:) show originalFilename values like IMG_XXXX.HEIC in PHAssetResource. Videos saved with addResource(with: .video, fileURL:) from AVCaptureMovieFileOutput keep the temp file name (a UUID .mov). The docs say that if originalFilename is omitted, Photos infers a name from the file URL when one is provided, otherwise it generates a name. Is there a supported way for a third-party app to get a system IMG_XXXX original filename for a newly saved video? Thank you.
1
0
1k
Aug ’26
iOS 27 beta 5: cameracaptured repeatedly crashes in BWVISNode with two stabilized front-camera video outputs
I’m seeing a repeatable first-party cameracaptured crash on an iPhone 14 Pro (iPhone15,2) running iOS 27 beta 5 (24A5424a). The single-camera AVCaptureSession configuration is: Front wide camera 3840×2160 at 30 fps, SDR/420f Two AVCaptureVideoDataOutput connections: Main recording output: 3840×2160 Auxiliary preview output: approximately 2096×1178 Both connections request and activate .standard video stabilization Within roughly two minutes, cameracaptured crashed three times—once during recording and twice during idle preview. The application receives AVErrorMediaServicesWereReset (-11819) after each crash. All three reports have the same faulting queue and underlying stack: com.apple.coremedia.capture.videodata-stabilization.front FigCFDictionaryGetCGRectIfPresent -[BWVISNode _transformRectanglesInMetadata:pts:] -[BWVISNode _stabilizeMetadataForSampleBuffer:] -[BWVISNode _tallyAndEmitSampleBuffer:] VideoStabilizationV2 The first two crashes are: EXC_BAD_ACCESS / EXC_ARM_PAC_FAIL address: 0x00000000dac11a30 termination: PAC_EXCEPTION, code 261 The third reaches the same stabilization stack but terminates with: EXC_BAD_ACCESS / SIGSEGV KERN_INVALID_ADDRESS at 0x30 App-side frame analytics were disabled, system pressure remained nominal, and neither audiomxd nor bluetoothd restarted. The first crash report also shows two front- stabilization worker threads, consistent with the two stabilized video outputs. I have filed Feedback Assistant report FB24553011 with the complete log archive and all three cameracaptured reports. With the iOS 27 public release approaching, waiting for an OS-side fix is not a viable shipping strategy; an app-side workaround is effectively the only option available for our release. Is this a known issue? In particular: Is it safe to leave .standard stabilization on the main recording connection while forcing the auxiliary preview connection to .off? Would disabling/removing the auxiliary AVCaptureVideoDataOutput reliably reduce this to one stabilization pipeline? If a single stabilized front output remains affected, is disabling front-camera stabilization entirely the only supported containment? Is there any public API for disabling the rectangle-metadata processing performed by BWVISNode? The current field workaround is to use the front camera with stabilization Off, or use a rear camera when stabilization is required.
0
0
710
Aug ’26
Where is the API for variable aperture of the iPhone 18 Pro?
According to Apple's press release (https://www.apple.com/pt/newsroom/2026/09/apple-debuts-iphone-18-pro-and-iphone-18-pro-max/): Variable aperture also provides more control for creative pros by giving them the ability to manually adjust any of the four aperture settings in the Camera app, and an API is available to developers for even more control across the aperture range in their apps. Can a DTS engineer specify the quoted API please? I can't see it anywhere in the documentation yet. What does it mean "even more control across the aperture range"? Will it be possible to set the aperture in a 0.0-1.0 range?
Replies
3
Boosts
3
Views
1.4k
Activity
Sep ’26
Picture-In-Picture not starting with Custom Player
Team, I am trying to start PictureInPicture for my video from custom player as mentioned in the following link, but I am getting an error Steps: Created a player using AVPlayer. Initiated AVPictureInPictureController with AVPlayerLayer Added observer on PIP button enable status When PIP button is tapped executed pipController.startPictureInPicture() statement. PIP is not getting started & getting following error Project created in Xcode11 & currently using Xcode13.2 Error Domain=AVKitErrorDomain Code=-1001 "Failed to start picture in picture." UserInfo={NSLocalizedDescription=Failed to start picture in picture., NSLocalizedFailureReason=The UIScene for the content source has an activation state other than UISceneActivationStateForegroundActive, which is not allowed.} I didn't get any solutions online, plz help me on this issue. FYI: PIP button/mode is working if I use AVPlayerViewController(with default playerItem buttons) instead of AVPlayer & AVPictureInPictureController with custom controls Thanks in advance. Adopting Picture in Picture in a Custom Player
Replies
3
Boosts
2
Views
1.8k
Activity
Sep ’26
`LockedCameraCaptureManager` practically unusable since iOS 26
Somewhere since iOS 26, the LockedCameraCapture framework gets in an unpredictable state after opening the main app from the LockedCamera extension using LockedCameraCaptureSession.openApplication(for userActivity:). (Feedback with sample code to reproduce: FB21966835) Opening the extension from the lock screen again doesn’t open the extension but puts the lock screen in a state as if it has. Content updated from LockedCameraCaptureManager.shared.sessionContentUpdates comes in inconsistently, usually needs the app to be opened again or the extension to be opened. This makes using this extension impossible for me as I use it to record video files that manually need to be imported when the app is launched (so not through PhotoKit). Does anybody have a suggestion to circumvent this issue or how to get this fixed?
Replies
2
Boosts
3
Views
1.5k
Activity
Sep ’26
AVPlayerItemSampleBufferOutput
I am using AVPlayer, but I am unable to retrieve PCM data from .m3u8 streams via the AVPlayerItemSampleBufferOutput API in iOS 27 beta. I have tried both the Objective-C and Swift APIs: the Objective-C delegate methods are not being called, and the Swift methods nextAvailableSampleBuffer() and nextSampleBuffer() are returning no data.
Replies
2
Boosts
1
Views
833
Activity
Sep ’26
fail to get HLS realtime stream via AVPlayerItemSampleBufferOutputDelegate
Hi, I'm trying to do some realtime audio processing on audio served from an HLS stream (i.e. an AVPlayer created using an M3U HTTP URL). And I find that new APIs(AVPlayerItemSampleBufferOutput) are available in iOS 27.0 to achieve it. However, I failed to get the available data via the AVPlayerItemSampleBufferOutputDelegate. And I found some error in the system log: I create AVPlayerItemSampleBufferOutput and set the delegate after receiving the AVPlayerItemStatusReadyToPlay event. And here's my code: @interface OCAudioSamplebuffer () <AVPlayerItemSampleBufferOutputDelegate> @property (nonatomic, strong) AVPlayerItemSampleBufferOutput *bufferOutput; @property (nonatomic, strong) dispatch_queue_t bufferOutputQueue; @property (nonatomic, strong) AVPlayerItem *playerItem; @end - (void)playItem:(AVPlayerItem *)item { if (@available(iOS 27, *)) { AVAudioSession *audioSession = [AVAudioSession sharedInstance]; if([NSThread mainThread]){ [audioSession setCategory:AVAudioSessionCategoryPlayback withOptions:AVAudioSessionCategoryPlayback error:nil]; [audioSession setActive:YES error:nil]; }else{ dispatch_async(dispatch_get_main_queue(), ^{ [audioSession setCategory:AVAudioSessionCategoryPlayback withOptions:AVAudioSessionCategoryPlayback error:nil]; [audioSession setActive:YES error:nil]; }); } AVPlayerItemSampleBufferOutputAudioConfiguration *cfg = [[AVPlayerItemSampleBufferOutputAudioConfiguration alloc] init]; CMFormatDescriptionRef formatDescription = [self createPCMFormatDescriptionWithSampleRate:44100.0 channels:2 isFloat:YES]; cfg.requestedAudioFormat = formatDescription; if (formatDescription) { NSLog(@"create PCM Format Description success"); CFRelease(formatDescription); } else { NSLog(@"fail to create PCM Format Description"); } self.bufferOutput = [[AVPlayerItemSampleBufferOutput alloc] initWithConfiguration:cfg]; NSLog(@"create buffer success, PCM Format Description%@---,%@", cfg.requestedAudioFormat, formatDescription); self.bufferOutputQueue = dispatch_queue_create("audioSamplebufferQueue", DISPATCH_QUEUE_CONCURRENT); [self.bufferOutput setDelegate:self queue:self.bufferOutputQueue]; [item addOutput:self.bufferOutput]; self.playerItem = item; } else { // Fallback on earlier versions } } Any help. Thank you
Replies
2
Boosts
0
Views
1.1k
Activity
Sep ’26
How can an app detect a TrueDepth pipeline failure when no depth output or drop callback is delivered?
How can an app detect a TrueDepth pipeline failure when no depth output or drop callback is delivered? We are using AVFoundation to stream synchronized RGB and depth data from the front-facing TrueDepth camera. On a small number of physical devices, the capture session and RGB stream remain healthy, but the depth pipeline becomes silent or returns unusable data. We would like to understand the intended public-API behavior and the supported way to detect this state without relying on private system logs or an application-defined timeout. We need an AVFoundation-only solution rather than ARKit. Capture configuration Our setup follows Apple's TrueDepth streaming sample: Discover .builtInTrueDepthCamera. Select an activeFormat with a nonempty supportedDepthDataFormats list. Set a supported activeDepthDataFormat, typically DepthFloat16. Add AVCaptureVideoDataOutput and AVCaptureDepthDataOutput. Enable the .depthData connection. Test both direct depth delegate delivery and AVCaptureDataOutputSynchronizer. Observe capture-session runtime errors, interruptions, connection state, and system pressure. The same code and configuration continuously deliver valid RGB and depth frames on healthy control devices. Observed failure modes 1. No depth callback of any kind The video delegate continues to receive RGB frames at the expected rate. The depth connection reports enabled and active. The capture session remains running. No AVCaptureSessionRuntimeErrorNotification or interruption is reported. System pressure is nominal. depthDataOutput(_:didOutput:timestamp:connection:) is never called. depthDataOutput(_:didDrop:timestamp:connection:reason:) is also never called. With AVCaptureDataOutputSynchronizer, RGB synchronization points may continue, but no AVCaptureSynchronizedDepthData object is present for the depth output. 2. One depth frame, followed by permanent silence On other affected devices, one depth frame is delivered shortly after startup, while hundreds of later RGB frames continue with no further depth output or drop callback. 3. A depth object is delivered, but the map is unusable We have also observed an AVDepthData object whose map is empty or has no usable spatial information, for example: a zero-sized/empty depth representation; an unfiltered map containing only NaN; or a filtered map containing a single constant value across nearly all pixels. Metadata such as depth quality, accuracy, calibration availability, supported formats, and connection state does not reliably distinguish these frames from healthy ones. Relevant documented behavior The documentation for AVCaptureSynchronizedDepthData.depthDataWasDropped says that a dropped depth object is different from a synchronization timestamp for which no depth capture occurred. In the latter case, no AVCaptureSynchronizedDepthData object is present in the collection. The documentation for depthDataOutput(_:didDrop:timestamp:connection:reason:) says that the callback reports captured depth data that wasn't processed, with an empty-shell AVDepthData object. This appears to leave a diagnostic gap: if the TrueDepth pipeline fails before a depth object is produced, the application may receive neither an output callback nor a drop callback. System-log correlation We do not want to depend on these private and undocumented strings, but sysdiagnose/device logs from affected devices correlate the public-API silence with messages such as: Can't activate Pearl projector: no projector token found PROJECTOR ON DENIED Object too close (no depth) GMC status: -2 Projector GMC hasn't completed yet - dropping depth/dx buffers Pulses exceed limit. Projector Off In the first group, depth delivery never starts. In the second group, depth may never start or may stop after one frame. RGB capture can remain active in both cases. These logs suggest that depth or disparity buffers can be rejected before they reach AVCaptureDepthDataOutput, which would explain why the application sees neither didOutput nor didDrop. However, none of these projector, IR, calibration, protection, or token states appears to be exposed by a public API. Questions Is it expected that AVCaptureDepthDataOutput delivers neither didOutput nor didDrop when no depth object is produced upstream? Is the absence of AVCaptureSynchronizedDepthData from a synchronization collection the only public signal for this condition? Is there a supported API, notification, KVO property, or error that reports that the TrueDepth IR/projector/depth provider is unavailable, disabled, protected, stalled, or unable to produce data? Can an app distinguish a healthy but temporarily delayed depth stream from a pipeline that will never deliver depth, other than by implementing its own timeout and frame-count watchdog? Is there a documented validity rule for an empty, all-NaN, or near-constant AVDepthData.depthDataMap? Should applications treat these as unavailable depth even when the AVDepthData metadata reports a normal quality or accuracy? What is the recommended recovery procedure: rebuild AVCaptureDepthDataOutput, restart the capture session, switch cameras and switch back, wait for a system notification, or stop retrying until the device state changes? ARKit can report ARError.sensorFailed when a required sensor does not deliver input. Is there an AVFoundation equivalent that identifies which required input failed, without adopting ARKit? If no public signal currently exists, would Apple consider exposing a depth-provider status/error callback that distinguishes at least: captured but dropped; no depth capture for the timestamp; provider temporarily unavailable; and provider disabled or failed until recovery/service? We can provide a minimal sample project, affected-device sysdiagnose, and synchronized timestamp traces in Feedback Assistant if that would help. Please let us know which diagnostics and logging profiles would be most useful.
Replies
0
Boosts
0
Views
192
Activity
Sep ’26
AVCaptureVideoDataOutput stops zooming while AVCaptureVideoPreviewLayer continues — physical wide / ultra-wide / telephoto only
We use a single AVCaptureSession with AVCaptureVideoPreviewLayer and AVCaptureVideoDataOutput (preview-sized buffers, BGRA). When we increase videoZoomFactor, beyond a certain zoom level the image from AVCaptureVideoDataOutput no longer zooms further, while AVCaptureVideoPreviewLayer continues to zoom with the same zoom control. The preview and the video-data output therefore diverge. This behavior appears when the active camera is a physical lens device — wide, ultra-wide, or telephoto (e.g. builtInWideAngleCamera, builtInUltraWideCamera, builtInTelephotoCamera, or similar). It does not appear when the active input is a virtual / multi-camera (e.g. triple camera, dual-wide, or other system multi-camera). Are there known conditions under which this mismatch between preview and video-data output is expected? Thank you.
Replies
2
Boosts
0
Views
1.1k
Activity
Sep ’26
AVAssetDownloadConfiguration does not download supplemental tracks on iOS 16
We're downloading HLS content for offline playback using AVAssetDownloadConfiguration. It's critical for our app that all subtitle tracks are downloaded together with the video, so users can watch any subtitle and audio option while offline. On iOS 16, not all tracks are downloaded, and when the user goes offline, playback fails with NSURLErrorNotConnectedToInternet (error -1009) because AVFoundation tries to fetch the missing subtitle segments over the network. On iOS 17 and later, everything works as expected — all subtitle and audio tracks are downloaded. However, on iOS 16 we observe two issues: First issue: Supplemental tracks are not downloaded on iOS 16 (device) Only the default audio track is downloaded. All non-default supplemental tracks — both subtitles and additional audio tracks — exhibit the same problem: Some supplemental tracks listed in the manifest don't even get a directory created in the .movpkg package Some supplemental tracks do get a directory created, but it contains only StreamInfoBoot.xml and StreamInfoRoot.xml — no actual segment data (.frag files) is downloaded. The StreamInfoBoot.xml for these directories shows NO, 0, and 0, with Supplemental Additionally, a Manifest.m3u8 stream directory is created with Complete=NO and no segment data. On iOS 17+ devices, all supplemental track directories are created and all contain segment data. Second issue: Downloads fail entirely on iOS 16 Simulator We receive the following error: Error Domain=NSURLErrorDomain Code=-16090 UserInfo={ NSLocalizedDescription=The operation could not be completed, _NSURLErrorFailingURLSessionTaskErrorKey=BackgroundAVAssetDownloadTask <DEC0D555-6C05-416C-91B8-F490F5A29046>.<3>, _NSURLErrorRelatedURLSessionTaskErrorKey=( "BackgroundAVAssetDownloadTask <DEC0D555-6C05-416C-91B8-F490F5A29046>.<3>" ), NSLocalizedFailureReason=An unknown error occurred (-16090) } On iOS 17+ Simulators, downloads work without issues. Our setup: let urlAsset = AVURLAsset(url: manifestURL) let preferredMediaSelection = try await urlAsset.load(.allMediaSelections) let config = AVAssetDownloadConfiguration(asset: urlAsset, title: title) config.primaryContentConfiguration.mediaSelections = preferredMediaSelection let task = assetDownloadURLSession.makeAssetDownloadTask(downloadConfiguration: config) task.resume() Example HLS manifest: #EXTM3U #EXT-X-VERSION:7 #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="French",LANGUAGE="fr",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_25/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="French (forced)",LANGUAGE="fr-x-cc",DEFAULT=NO,AUTOSELECT=YES,FORCED=YES,URI="s_31/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",LANGUAGE="en",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_37/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Kazakh",LANGUAGE="kk",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_43/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Uzbek",LANGUAGE="uz",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_49/stream.m3u8" #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aud_ec3",NAME="French 5.1",LANGUAGE="fr",DEFAULT=YES,AUTOSELECT=YES,CHANNELS="6",URI="a_18/stream.m3u8" #EXT-X-STREAM-INF:AUDIO="aud_ec3",SUBTITLES="subs",BANDWIDTH=1931408,VIDEO-RANGE=SDR,CODECS="hvc1.1.6.L90.90,ec-3",RESOLUTION=1024x428,FRAME-RATE=25,HDCP-LEVEL=NONE v_6/stream.m3u8 #EXT-X-SESSION-KEY:METHOD=SAMPLE-AES,URI="skd://XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",KEYFORMAT="com.apple.streamingkeydelivery",KEYFORMATVERSIONS="1" Questions: What is the correct way to make sure all supplemental tracks (subtitles, non-default audio) are actually downloaded on an iOS 16 real device? We pass asset.load(.allMediaSelections) to primaryContentConfiguration.mediaSelections — is there a different configuration, or should we use auxiliaryContentConfigurations, or something else entirely? Is there any way to make asset downloads work on the iOS 16 Simulator at all? We always get NSURLErrorDomain error -16090, while iOS 17+ Simulators work fine. If the supplemental tracks download bug on iOS 16 cannot be fixed or worked around from the app side, do you have any recommended approach for handling incomplete .movpkg packages at offline playback time?
Replies
1
Boosts
0
Views
534
Activity
Sep ’26
Variable aperture on the iPhone 18 Pro
Does iOS 27 provide an API which allows developers to set the iPhone 18 Pro camera's aperture? If it does, is the aperture going to be limited to just 4 discrete settings (like in the Camera app and like mentioned in the Tech Specs for the 18 Pro) or does the hypothetical iOS API allow us to set the aperture to a continuous value in the 0.0–1.0 range (similar to how a camera's focus can be set)?
Replies
2
Boosts
1
Views
889
Activity
Sep ’26
Capturing 48mp photos with .builtInTripleCamera
I am able to capture 48mp photos using .builtInWideAngleCamera, but it seems like .builtInTripleCamera is capped at 12mp? Is there a way to capture 48mp photos using .builtInTripleCamera? Because .builtInTripleCamera provides smooth transition between cameras during zooming, and I'd like to keep this behavior. New iPhone 17 Pro have all their cameras at 48mp. Is there a chance that their .builtInTripleCamera is capable of capturing 48mp? Or is this an API limitation?
Replies
4
Boosts
0
Views
1.2k
Activity
Sep ’26
Working with kCVPixelFormatType_96VersatileBayerPacked12
Whilst AVCaptureSession is setup to capture ProRes RAW video, is it possible to get video pixel data which can read and processed, such as using CIImage(cvPixelBuffer: ) AVCaptureVideoDataOutput outputs ProRes RAW in kCVPixelFormatType_96VersatileBayerPacked12 pixel format. Is there a provided way to debayer this pixel format into something more usable?
Replies
1
Boosts
1
Views
1.2k
Activity
Sep ’26
visionOS Developer Strap: UVC video works, but how can I access the capture device’s USB Audio Class (UAC) audio?
I’m developing a native visionOS application that uses an external HDMI capture device connected to Apple Vision Pro through the Developer Strap. The goal is to use Apple Vision Pro as a low-latency display for external HDMI sources such as an Xbox Series X, including both the HDMI video and the original digital game audio. My current hardware path is: Xbox Series X → HDMI → Elgato 4K X → USB-C → Apple Vision Pro Developer Strap The Elgato 4K X receives both video and audio from the Xbox over HDMI. What is working UVC video capture is working successfully on visionOS. The external Elgato capture device can be used as the video source and the Xbox video is displayed correctly inside the Vision Pro application. I have also verified the Elgato 4K X independently on macOS, where the device is recognized correctly. The problem I cannot find a supported way on visionOS to access the USB Audio Class (UAC) audio interface associated with the capture device. I specifically need the digital game audio originating from: Xbox HDMI → Elgato 4K X → USB/UAC → visionOS I do not want to use the Apple Vision Pro’s built-in microphone. Selecting/capturing the Vision Pro microphone would only record environmental sound and is not a substitute for the digital HDMI audio stream. Apple’s visionOS documentation for external devices through the Developer Strap appears to focus on UVC video capture. I haven’t been able to determine whether the accompanying UAC interface of a UVC/UAC composite capture device is exposed to visionOS applications. What I’ve tried / verified Connected an Xbox Series X to the HDMI input of an Elgato 4K X. Connected the 4K X to Apple Vision Pro through the Developer Strap. Successfully discovered and received the external UVC video stream. Successfully displayed the Xbox video feed in visionOS. Confirmed that the 4K X itself is recognized and functional on macOS. Attempted to obtain audio corresponding to the external capture source. The Vision Pro microphone is available as an audio source, but this is not the audio source required. I have not found a documented visionOS API that exposes the 4K X’s USB/UAC audio endpoint through the Developer Strap. Questions Does visionOS currently support USB Audio Class input devices connected through the Apple Vision Pro Developer Strap? If a device exposes both UVC video and UAC audio interfaces, is there a supported way to associate/access both interfaces from a visionOS application? If UAC is supported, which framework/API should be used to discover the external audio endpoint — AVFoundation, Core Audio, or another API? Does accessing the UAC interface require an entitlement in addition to UVC device access? If generic UAC access through the Developer Strap is currently unavailable to third-party visionOS applications, is there a restricted/managed entitlement that developers can request? If this isn’t currently supported, what is Apple’s recommended architecture for receiving synchronized video and audio from an HDMI capture device on Apple Vision Pro? Ultimately, I’m trying to achieve: HDMI source → UVC video + UAC audio → visionOS application → video displayed in Vision Pro + audio played through Vision Pro speakers/AirPods Since UVC video from this exact hardware is already working, I’m primarily trying to determine whether access to the accompanying USB audio interface is possible with the current visionOS SDK, requires another entitlement/API, or is currently unsupported. Any guidance from Apple or developers who have worked with composite UVC/UAC devices on Vision Pro would be greatly appreciated.
Replies
0
Boosts
1
Views
827
Activity
Sep ’26
iOS 27 beta: setExposureModeCustom silently ignored on the front camera
On every iOS 27.0 beta, setExposureModeCustom has no effect on the front camera (.builtInTrueDepthCamera). exposureMode does change to .custom, but the sensor keeps auto-exposing: device.iso and device.exposureDuration report auto-exposure's values, not the requested ones, and frame brightness does not change. No error is thrown. The same code works on iOS 18.x and 26.x on the same devices. Device: iPhone 15 Pro, every iOS 27.0 beta. Also seen on iPhone 14 Pro Max, 16, 16 Pro Max and 17 Pro Max. Session: front camera, AVCaptureVideoDataOutput (BGRA) + AVCaptureDepthDataOutput. Format advertises ISO 20...1920, duration 0.00004...0.06667 s. isExposureModeSupported(.custom) is true. Repro: let auto-exposure settle, then try device.lockForConfiguration() device.setExposureModeCustom(duration: CMTime(value: 1, timescale: 15), iso: 1920, completionHandler: nil) device.unlockForConfiguration() then poll device.iso / device.exposureDuration (the completion handler never fires on this device) and measure frame luma. Readback, polled 0.8 s after the call: requested ISO 1920, 66.7 ms -> reported ISO 349, 60.0 ms requested ISO 20, 0.04 ms -> reported ISO 188, 66.5 ms Requesting the shortest exposure returns the longest, and the same ISO 1920 request returned 249, 349 and 450 on three consecutive runs, so these are auto-exposure's values. Frame luma stayed within 1% between the two phases. setExposureTargetBias, however, is honored on the same device: at +4 EV auto-exposure drives the sensor to ISO 1920 / 66.5 ms and at -4 EV to ISO 24 / 30.0 ms, moving mean luma from 49 to 232. So the sensor does reach the values setExposureModeCustom cannot set. Question: is this intentional or a regression, and how should an app detect that a custom exposure request was refused?
Replies
2
Boosts
0
Views
873
Activity
Sep ’26
Does the TN3135 audio-session networking exception have a defined lifetime? Seeing a ~38.5 s revoke/re-grant cycle
TN3135 describes the exception that lets a watchOS app use low-level networking while it holds an active audio session. I have that working, and the app functions — but the network path is withdrawn and restored on a strikingly regular cycle, and I would like to know whether that is expected behaviour rather than something I am doing wrong. Setup Apple Watch Series 10 (Watch7,9), watchOS 26.5. Reproduced on a Series 6 (Watch6,2). UIBackgroundModes: [audio]; AVAudioSession category .playAndRecord, mode .spokenAudio; activated with the async activate(options:completionHandler:). NWConnection with NWProtocolWebSocket to a WebSocket relay over TLS. The app streams 16 kHz mono PCM continuously while transmitting and holds the socket open otherwise. Symptom NWPathMonitor reports .unsatisfied, then .satisfied about two seconds later, over and over. Measured with the iPhone powered off, so the watch was on its own Wi-Fi: Uptime between drops Outage 36.4 s 2.1 s 36.7 s 1.9 s 36.9 s 2.1 s The regularity is what prompts the question — uptime varies by ±0.3 s and the outage is consistently 2.0 s. That reads as a timeout expiring rather than radio behaviour. What I have ruled out Not the network or the server. A browser client on the same relay, same TLS, same wire protocol, holds a WebSocket indefinitely. Not the interface. Identical cadence over the companion ipsec1 tunnel with the iPhone present, and over the watch's own en0 with the iPhone powered off. Pinning requiredInterfaceType = .wifi while the iPhone is reachable fails outright — the path offers only ipsec1. Not audio-session interruption. I observe interruptionNotification, routeChangeNotification, mediaServicesWereResetNotification and silenceSecondaryAudioHintNotification. None fire at a drop. At the moment the path goes .unsatisfied, the engine is running and the player node is actively playing. Not session idleness. Playing continuous silence for the whole session, rather than only while reconnecting, made no difference — still 36.4 s. The control that surprised me To test whether this affects any long-lived watch socket or only audio-unlocked ones, I built a second app with no AVAudioSession at all, no audio background mode, holding a URLSessionWebSocketTask and kept alive by a WKExtendedRuntimeSession so screen sleep was not a factor. It never connected. NWPathMonitor reported .unsatisfied once and never changed, across a 30 s run, and every request failed with "The Internet connection appears to be offline." I had expected URLSession to be permitted regardless. Questions Does the audio-session networking exception in TN3135 have a defined lifetime, and is a periodic revoke/re-grant cycle expected? If so, is there a supported way to hold it continuously — or is the correct design simply to expect the interruption and reconnect through it? Is it expected that an app with no audio session gets no network path at all on watchOS, including via URLSession, even in the foreground with an extended runtime session?
Replies
16
Boosts
0
Views
4.1k
Activity
Sep ’26
AirPods Sleep Detection pauses meditation playback with no API/callback to identify the reason
Good morning! I work on a meditation app, and since Sleep Detection was introduced for AirPods in iOS 26, we've been receiving complaints from users reporting that their meditation sessions unexpectedly stop after approximately 15–20 minutes when Sleep Detection is enabled. This seems to happen particularly during deep meditation sessions, where the user may remain still and exhibit behavior that could potentially be interpreted as falling asleep. I've also tested this on iOS 27 with the latest beta firmware available for AirPods Pro, and the same behavior still occurs. While investigating and debugging the issue, I found that there doesn't appear to be any public API that allows an app to determine that playback was paused specifically because of AirPods Sleep Detection. From the application's perspective, the AirPods appear to simply send a standard pause command to the AVPlayer, without providing any additional context or reason. There also doesn't seem to be an API that allows us to determine whether the user has Sleep Detection enabled. Even having access to that information would allow us to warn users that their meditation session could potentially be interrupted. AVAudioSession.InterruptionReason does not seem to help in this case either: https://developer.apple.com/documentation/avfaudio/avaudiosession/interruptionreason Is there currently any supported way for an app to: Detect that playback was paused specifically because of AirPods Sleep Detection? Determine whether Sleep Detection is enabled for the connected AirPods? Prevent Sleep Detection from pausing playback for specific apps or specific types of audio sessions? If none of these are currently possible, could Apple consider providing either a notification/callback indicating that Sleep Detection triggered the pause, or an option for users to exclude specific apps from Sleep Detection? This behavior is particularly problematic for meditation apps, since a user in a deep meditation session can easily be mistaken for someone who has fallen asleep. Other meditation apps appear to be affected by the same behavior as well. Best regards, Carlos Antunes
Replies
1
Boosts
0
Views
1.4k
Activity
Sep ’26
Can an iOS app programmatically enable Voice Isolation microphone mode?
Hi, I’m developing an iOS app that captures the user’s voice and sends the microphone input to an AI model for speech processing. For this use case, Voice Isolation provides a significantly better input signal in noisy environments. However, I’m currently unable to enable Voice Isolation programmatically when the app starts recording. As I understand it, microphone modes such as Automatic, Standard, Voice Isolation, and Wide Spectrum are controlled by the user through the system UI. My app can detect/observe the current microphone mode, but I have not found a public API that allows the app to select Voice Isolation itself. My questions are: Is there currently a supported public API to programmatically switch the microphone mode from Automatic/Standard → Voice Isolation? If direct selection is not supported, is there a recommended way for an app that primarily processes speech/voice input for an AI model to request or prefer Voice Isolation? Would using AVAudioSession with an appropriate voice-processing configuration, such as .voiceChat, or enabling voice processing through AVAudioEngine, provide equivalent Voice Isolation processing? Is AVCaptureDevice.showSystemUserInterface(.microphoneModes) the intended approach when an app needs the user to change the microphone mode? I understand that Apple may intentionally keep microphone-mode selection under user control. I’m mainly trying to determine whether there is a supported programmatic approach that I may have overlooked, rather than relying on a custom noise-suppression implementation. Any guidance on the recommended architecture for this use case would be greatly appreciated. Thank you!
Replies
0
Boosts
0
Views
795
Activity
Sep ’26
AVCaptureDevice.uniqueID for UVC devices is unstable - bug or overstated documentation?
The documentation for AVCaptureDevice.uniqueID states the following: Capture devices have a unique identifier that persists on one system across device connections and disconnections, application restarts, and reboots of the system itself. You can store the value returned by this property to recall or track the status of a specific device in the future. For UVC capture devices this documentation does not hold. The video uniqueID is a hex string of the form 0x<locationID><vendorID><productID>, and the identifying half is the locationID (bus number plus port path). Which identifies a port, not a device. I ran a suite of tests with three identical Elgato 4K X capture cards connected to a Mac Studio w/ M3 Ultra running macOS 26.5.2, and reproduced my findings on a MacBook w/ M3 Pro (same macOS version). See the script at the bottom of the post for how uniqueId & USB serial number are being retrieved. 1. The uniqueID follows the port. Swapping two cards between two built-in ports swaps their uniqueIDs: # Before swap. 4K X uid=0x2000000fd9009b serial=A7SNB50424UBQI 4K X uid=0x12000000fd9009b serial=A7SNB504219J0R # After swapping the cards between the same two ports. 4K X uid=0x2000000fd9009b serial=A7SNB504219J0R 4K X uid=0x12000000fd9009b serial=A7SNB50424UBQI An app that stored 0x2000000fd9009b to recall a specific capture card now silently opens another. 2. A reboot alone can swap uniqueIDs. External USB controllers (here, PCIe USB cards in two Thunderbolt enclosures) can race for bus numbers at boot, so with every cable left in place, a reboot swapped two of the cards: # Before reboot. 4K X uid=0x262000000fd9009b serial=A7SNB504219J0R 4K X uid=0x252000000fd9009b serial=A7SNB50423R73R # After reboot, no cables touched. 4K X uid=0x262000000fd9009b serial=A7SNB50423R73R 4K X uid=0x252000000fd9009b serial=A7SNB504219J0R This behavior is intermittent, a second reboot changed nothing, but a third caused another swap. Cards left alone in built-in ports retain their uniqueIDs across reboots in my testing; the failure requires dynamically enumerated external USB controllers. 3. Even the product ID tail can drift. One unit intermittently enumerates with idProduct 0x009c instead of 0x009b, same port (USB PCIe card in a Thunderbolt enclosure), cables untouched: # Before reboot. 4K X uid=0x222000000fd9009b serial=A7SNB50424UBQI # After reboot. 4K X uid=0x222000000fd9009c serial=A7SNB50424UBQI IOKit and AVFoundation agree each boot... So the change is upstream of both? I'm uncertain where to place blame for this specific issue (UVC device or macOS). Audio on the same physical units is unaffected. The audio uniqueID (AppleUSBAudioEngine:...:<serial>:...) embeds the USB serial and stayed stable through every test. So AVCaptureDevice can provide a stable per-device identifier, just not for UVC video devices. Questions: Is this a bug, or is the documentation overstating the persistence guarantee for USB video devices? What is the supported way to identify a specific physical UVC video device across reboots and port changes? The USB serial number is stable and is what I've fallen back on via IOKit, but there is no documented AVFoundation API to retrieve USB serial number from a UVC video AVCaptureDevice. Related: thread 803759, where the locationID-derived format is described. Script used for all output above (swift ./list-uvc.swift): import AVFoundation import IOKit func usbSerial(forLocation location: UInt32) -> String? { var iterator: io_iterator_t = 0 guard IOServiceGetMatchingServices(kIOMainPortDefault, IOServiceMatching("IOUSBHostDevice"), &iterator) == KERN_SUCCESS else { return nil } defer { IOObjectRelease(iterator) } var result: String? var service = IOIteratorNext(iterator) while service != 0 { var loc: UInt32 = 0 if let ref = IORegistryEntryCreateCFProperty(service, "locationID" as CFString, kCFAllocatorDefault, 0)?.takeRetainedValue(), let num = ref as? NSNumber { loc = num.uint32Value } if loc == location, let ref = IORegistryEntryCreateCFProperty(service, "USB Serial Number" as CFString, kCFAllocatorDefault, 0)?.takeRetainedValue(), let serial = ref as? String { result = serial } IOObjectRelease(service) if result != nil { break } service = IOIteratorNext(iterator) } return result } let session = AVCaptureDevice.DiscoverySession(deviceTypes: [.external], mediaType: .video, position: .unspecified) for device in session.devices { let uid = device.uniqueID let location = UInt32(truncatingIfNeeded: strtoull(uid, nil, 16) >> 32) let serial = usbSerial(forLocation: location) ?? "N/A" print("\(device.localizedName) uid=\(uid) serial=\(serial)") }
Replies
6
Boosts
0
Views
2.1k
Activity
Sep ’26
PhotoKit IMG_XXXX names for videos?
I save stills and videos with PHAssetCreationRequest. I do not set originalFilename. Photos saved with addResource(with: .photo, data:) show originalFilename values like IMG_XXXX.HEIC in PHAssetResource. Videos saved with addResource(with: .video, fileURL:) from AVCaptureMovieFileOutput keep the temp file name (a UUID .mov). The docs say that if originalFilename is omitted, Photos infers a name from the file URL when one is provided, otherwise it generates a name. Is there a supported way for a third-party app to get a system IMG_XXXX original filename for a newly saved video? Thank you.
Replies
1
Boosts
0
Views
1k
Activity
Aug ’26
iOS 27 beta 5: cameracaptured repeatedly crashes in BWVISNode with two stabilized front-camera video outputs
I’m seeing a repeatable first-party cameracaptured crash on an iPhone 14 Pro (iPhone15,2) running iOS 27 beta 5 (24A5424a). The single-camera AVCaptureSession configuration is: Front wide camera 3840×2160 at 30 fps, SDR/420f Two AVCaptureVideoDataOutput connections: Main recording output: 3840×2160 Auxiliary preview output: approximately 2096×1178 Both connections request and activate .standard video stabilization Within roughly two minutes, cameracaptured crashed three times—once during recording and twice during idle preview. The application receives AVErrorMediaServicesWereReset (-11819) after each crash. All three reports have the same faulting queue and underlying stack: com.apple.coremedia.capture.videodata-stabilization.front FigCFDictionaryGetCGRectIfPresent -[BWVISNode _transformRectanglesInMetadata:pts:] -[BWVISNode _stabilizeMetadataForSampleBuffer:] -[BWVISNode _tallyAndEmitSampleBuffer:] VideoStabilizationV2 The first two crashes are: EXC_BAD_ACCESS / EXC_ARM_PAC_FAIL address: 0x00000000dac11a30 termination: PAC_EXCEPTION, code 261 The third reaches the same stabilization stack but terminates with: EXC_BAD_ACCESS / SIGSEGV KERN_INVALID_ADDRESS at 0x30 App-side frame analytics were disabled, system pressure remained nominal, and neither audiomxd nor bluetoothd restarted. The first crash report also shows two front- stabilization worker threads, consistent with the two stabilized video outputs. I have filed Feedback Assistant report FB24553011 with the complete log archive and all three cameracaptured reports. With the iOS 27 public release approaching, waiting for an OS-side fix is not a viable shipping strategy; an app-side workaround is effectively the only option available for our release. Is this a known issue? In particular: Is it safe to leave .standard stabilization on the main recording connection while forcing the auxiliary preview connection to .off? Would disabling/removing the auxiliary AVCaptureVideoDataOutput reliably reduce this to one stabilization pipeline? If a single stabilized front output remains affected, is disabling front-camera stabilization entirely the only supported containment? Is there any public API for disabling the rectangle-metadata processing performed by BWVISNode? The current field workaround is to use the front camera with stabilization Off, or use a rear camera when stabilization is required.
Replies
0
Boosts
0
Views
710
Activity
Aug ’26