AVFoundation

RSS for tag

Work with audiovisual assets, control device cameras, process audio, and configure system audio interactions using AVFoundation.

Posts under AVFoundation tag

200 Posts

Post

Replies

Boosts

Views

Activity

CMLogItem.timestamp: which clock is it on, and how to convert to AVCaptureSession.synchronizationClock / host time?
We receive live relative-altitude updates from CMAltimeter.startRelativeAltitudeUpdates(to:withHandler:) on iPhone. CMAltitudeData inherits CMLogItem.timestamp, which the documentation describes as the time when the item is valid and as seconds since device boot. We need to determine whether a pressure measurement's event time falls inside an application operation bounded by a supported monotonic clock. Could Apple clarify the supported contract for CMAltitudeData.timestamp on iOS? Does CMAltitudeData.timestamp use the same epoch and rate as mach_absolute_time(), DispatchTime.uptimeNanoseconds, CACurrentMediaTime(), or ProcessInfo.systemUptime? If only some are compatible, which ones? Does it advance or pause during device sleep, device lock, application suspension, and background execution? In particular, are its sleep/suspension semantics guaranteed to match any of the clocks above? Is there a supported API for converting CMLogItem.timestamp to a host CMClock/Mach time, or for sampling “now” in the exact same clock domain used by CMAltitudeData.timestamp? Are the answers contractual across supported iPhone hardware and iOS releases, or are they implementation details that applications should not rely on? If direct comparison is not supported, what Apple-supported clock or conversion mechanism should an application use to compare a CMAltitudeData event time with two application-side monotonic operation boundaries? The question concerns clock semantics only. A minimal reproducer can be supplied if requested, but no application identifier, production data, sensor values, or user data is required to answer it.
4
0
333
11s
iPhone Duo: Does CameraCaptureAccessory accept touch, what counts as an active camera session, and can both front cameras run at once?
After watching "Leverage multiple displays and scenes on iPhone Duo" and "Build a great camera experience for iPhone Duo", we have four questions about the outer display while the device is open: Does a view hosted in CameraCaptureAccessory receive touch input on the outer display, or is it display-only? The accessory is described as available when the app "has an active camera session." Does that require a running preview layer or an attached output (photo, movie or data), or is a running AVCaptureSession with an input enough? Is a session whose main purpose is to host the accessory a supported use? Can AVCaptureMultiCamSession run the inner (under-display) front camera and the outer front camera at the same time on iPhone Duo? If so, which device-type combinations are supported, and are there thermal or power limits? Is CameraCaptureAccessory the only way for an app to show content on the outer display while the inner display is live, or are scene accessories that don't depend on the camera planned? Our use case is not photography. We're exploring a companion view for a second person sitting across from the device owner, so we want to understand whether the camera accessory is the right tool before designing around it.
4
0
40
15h
Camera direction and rotation in the half-folded pose on iPhone Duo
"Build a great camera experience for iPhone Duo" covers the device open and the device closed, including reconfiguring the session from AVCaptureDeviceDirectionCoordinator's change handler when the forward-facing camera changes. It does not cover the partially open pose, which "Leverage multiple displays and scenes on iPhone Duo" treats as a first-class hinge state with a continuous angle. Consider a camera app that keeps a live preview on screen and adopts AVCaptureDevice.RotationCoordinator — the documented remedy for cases like the square front sensor on iPhone 17, where a hard-coded 90° leaves the preview tilted — applying videoRotationAngleForHorizonLevelPreview to both the preview and the capture connection. In the partially open pose the two halves are at different physical attitudes: one stands up, the other lies flat. Which camera does AVCaptureDeviceDirectionCoordinator report as forward-facing while the hinge status is partially open? Does the hinge angle affect that decision? RotationCoordinator derives its angle from device attitude. With the two halves at different attitudes, which one does videoRotationAngleForHorizonLevelPreview reflect — the half carrying the active camera, the half hosting the preview layer, or a single device-level reference? If an app deliberately keeps the rear camera on the scene while its preview is drawn on the half facing the user, is that a supported configuration, or is the coordinator expected to override that choice? Is any of this modelled in the iPhone Duo simulator, or does the partially open pose require real hardware to test? Question 4 matters for planning: building against a rotation value that only appears on device would be worth knowing about before the hardware ships.
4
0
60
15h
Simultaneous capture and fold transitions in a live-streaming app on iPhone Duo
I build Lumecast, a live-streaming app (custom AVCapture pipeline feeding WebRTC). Three questions:   1. Does iPhone Duo support AVCaptureMultiCamSession — specifically front + rear simultaneously? Our co-casting feature would let a solo host fill two panes with their face and what they're looking at. How should a continuous capture session (live broadcast, can't pause) respond to camera direction changes as the device opens and closes? Is there a recommended pattern for swapping to the Virtual Front Camera mid-session without a visible gap for viewers?  3. Which cameras are available to third-party apps in each posture (folded / half-open / fully open), and does availability change at a specific hinge state? Goal: a host folds or unfolds mid-broadcast and their audience never sees a dropped frame.
3
0
90
1d
Is AVCaptureMultiCamSession supported on iPhone Duo
is AVCaptureMultiCamSession supported on iPhone Duo — specifically streaming two cameras that face opposite directions simultaneously (e.g. .builtInInnerUltraWideCamera + .builtInOuterUltraWideCamera, or the Virtual Front Camera plus a backward-facing camera)? I noticed the article recommends single-input reconfiguration over multi-cam; is that a performance recommendation, or is opposite-direction multi-cam unavailable on this device? Our use case is a live co-cast where one host fills two video panes: their face and what they're looking at.
0
0
15
1d
Where is the API for variable aperture of the iPhone 18 Pro?
According to Apple's press release (https://www.apple.com/pt/newsroom/2026/09/apple-debuts-iphone-18-pro-and-iphone-18-pro-max/): Variable aperture also provides more control for creative pros by giving them the ability to manually adjust any of the four aperture settings in the Camera app, and an API is available to developers for even more control across the aperture range in their apps. Can a DTS engineer specify the quoted API please? I can't see it anywhere in the documentation yet. What does it mean "even more control across the aperture range"? Will it be possible to set the aperture in a 0.0-1.0 range?
3
3
923
1d
Picture-In-Picture not starting with Custom Player
Team, I am trying to start PictureInPicture for my video from custom player as mentioned in the following link, but I am getting an error Steps: Created a player using AVPlayer. Initiated AVPictureInPictureController with AVPlayerLayer Added observer on PIP button enable status When PIP button is tapped executed pipController.startPictureInPicture() statement. PIP is not getting started & getting following error Project created in Xcode11 & currently using Xcode13.2 Error Domain=AVKitErrorDomain Code=-1001 "Failed to start picture in picture." UserInfo={NSLocalizedDescription=Failed to start picture in picture., NSLocalizedFailureReason=The UIScene for the content source has an activation state other than UISceneActivationStateForegroundActive, which is not allowed.} I didn't get any solutions online, plz help me on this issue. FYI: PIP button/mode is working if I use AVPlayerViewController(with default playerItem buttons) instead of AVPlayer & AVPictureInPictureController with custom controls Thanks in advance. Adopting Picture in Picture in a Custom Player
3
2
1.6k
1d
`LockedCameraCaptureManager` practically unusable since iOS 26
Somewhere since iOS 26, the LockedCameraCapture framework gets in an unpredictable state after opening the main app from the LockedCamera extension using LockedCameraCaptureSession.openApplication(for userActivity:). (Feedback with sample code to reproduce: FB21966835) Opening the extension from the lock screen again doesn’t open the extension but puts the lock screen in a state as if it has. Content updated from LockedCameraCaptureManager.shared.sessionContentUpdates comes in inconsistently, usually needs the app to be opened again or the extension to be opened. This makes using this extension impossible for me as I use it to record video files that manually need to be imported when the app is launched (so not through PhotoKit). Does anybody have a suggestion to circumvent this issue or how to get this fixed?
2
3
1.2k
2d
AVPlayerItemSampleBufferOutput
I am using AVPlayer, but I am unable to retrieve PCM data from .m3u8 streams via the AVPlayerItemSampleBufferOutput API in iOS 27 beta. I have tried both the Objective-C and Swift APIs: the Objective-C delegate methods are not being called, and the Swift methods nextAvailableSampleBuffer() and nextSampleBuffer() are returning no data.
2
1
631
2d
fail to get HLS realtime stream via AVPlayerItemSampleBufferOutputDelegate
Hi, I'm trying to do some realtime audio processing on audio served from an HLS stream (i.e. an AVPlayer created using an M3U HTTP URL). And I find that new APIs(AVPlayerItemSampleBufferOutput) are available in iOS 27.0 to achieve it. However, I failed to get the available data via the AVPlayerItemSampleBufferOutputDelegate. And I found some error in the system log: I create AVPlayerItemSampleBufferOutput and set the delegate after receiving the AVPlayerItemStatusReadyToPlay event. And here's my code: @interface OCAudioSamplebuffer () <AVPlayerItemSampleBufferOutputDelegate> @property (nonatomic, strong) AVPlayerItemSampleBufferOutput *bufferOutput; @property (nonatomic, strong) dispatch_queue_t bufferOutputQueue; @property (nonatomic, strong) AVPlayerItem *playerItem; @end - (void)playItem:(AVPlayerItem *)item { if (@available(iOS 27, *)) { AVAudioSession *audioSession = [AVAudioSession sharedInstance]; if([NSThread mainThread]){ [audioSession setCategory:AVAudioSessionCategoryPlayback withOptions:AVAudioSessionCategoryPlayback error:nil]; [audioSession setActive:YES error:nil]; }else{ dispatch_async(dispatch_get_main_queue(), ^{ [audioSession setCategory:AVAudioSessionCategoryPlayback withOptions:AVAudioSessionCategoryPlayback error:nil]; [audioSession setActive:YES error:nil]; }); } AVPlayerItemSampleBufferOutputAudioConfiguration *cfg = [[AVPlayerItemSampleBufferOutputAudioConfiguration alloc] init]; CMFormatDescriptionRef formatDescription = [self createPCMFormatDescriptionWithSampleRate:44100.0 channels:2 isFloat:YES]; cfg.requestedAudioFormat = formatDescription; if (formatDescription) { NSLog(@"create PCM Format Description success"); CFRelease(formatDescription); } else { NSLog(@"fail to create PCM Format Description"); } self.bufferOutput = [[AVPlayerItemSampleBufferOutput alloc] initWithConfiguration:cfg]; NSLog(@"create buffer success, PCM Format Description%@---,%@", cfg.requestedAudioFormat, formatDescription); self.bufferOutputQueue = dispatch_queue_create("audioSamplebufferQueue", DISPATCH_QUEUE_CONCURRENT); [self.bufferOutput setDelegate:self queue:self.bufferOutputQueue]; [item addOutput:self.bufferOutput]; self.playerItem = item; } else { // Fallback on earlier versions } } Any help. Thank you
2
0
908
2d
iOS 27 beta: Opening Notification Center pauses AVPlayer playback
Since iOS 27 beta we are seeing a behavior change in our video streaming app and I would like to know whether others can reproduce it. Behavior on iOS 27 beta: Fully opening the Notification Center pauses playback. Audio continues for about 5 more seconds, then the app is suspended. Closing the Notification Center leaves the player paused. On iOS 26 the same build keeps playing in this situation. Setup: AVQueuePlayer playing video with audio, not muted audiovisualBackgroundPlaybackPolicy = .automatic (default) AVAudioSession category .playback, UIBackgroundModes: audio entitlement AVPictureInPictureController attached with canStartPictureInPictureAutomaticallyFromInline = true Filed as FB23893965 Questions: Can anyone reproduce this on iOS 27 beta? Is this intentional or a regression?
1
3
787
6d
How can an app detect a TrueDepth pipeline failure when no depth output or drop callback is delivered?
How can an app detect a TrueDepth pipeline failure when no depth output or drop callback is delivered? We are using AVFoundation to stream synchronized RGB and depth data from the front-facing TrueDepth camera. On a small number of physical devices, the capture session and RGB stream remain healthy, but the depth pipeline becomes silent or returns unusable data. We would like to understand the intended public-API behavior and the supported way to detect this state without relying on private system logs or an application-defined timeout. We need an AVFoundation-only solution rather than ARKit. Capture configuration Our setup follows Apple's TrueDepth streaming sample: Discover .builtInTrueDepthCamera. Select an activeFormat with a nonempty supportedDepthDataFormats list. Set a supported activeDepthDataFormat, typically DepthFloat16. Add AVCaptureVideoDataOutput and AVCaptureDepthDataOutput. Enable the .depthData connection. Test both direct depth delegate delivery and AVCaptureDataOutputSynchronizer. Observe capture-session runtime errors, interruptions, connection state, and system pressure. The same code and configuration continuously deliver valid RGB and depth frames on healthy control devices. Observed failure modes 1. No depth callback of any kind The video delegate continues to receive RGB frames at the expected rate. The depth connection reports enabled and active. The capture session remains running. No AVCaptureSessionRuntimeErrorNotification or interruption is reported. System pressure is nominal. depthDataOutput(_:didOutput:timestamp:connection:) is never called. depthDataOutput(_:didDrop:timestamp:connection:reason:) is also never called. With AVCaptureDataOutputSynchronizer, RGB synchronization points may continue, but no AVCaptureSynchronizedDepthData object is present for the depth output. 2. One depth frame, followed by permanent silence On other affected devices, one depth frame is delivered shortly after startup, while hundreds of later RGB frames continue with no further depth output or drop callback. 3. A depth object is delivered, but the map is unusable We have also observed an AVDepthData object whose map is empty or has no usable spatial information, for example: a zero-sized/empty depth representation; an unfiltered map containing only NaN; or a filtered map containing a single constant value across nearly all pixels. Metadata such as depth quality, accuracy, calibration availability, supported formats, and connection state does not reliably distinguish these frames from healthy ones. Relevant documented behavior The documentation for AVCaptureSynchronizedDepthData.depthDataWasDropped says that a dropped depth object is different from a synchronization timestamp for which no depth capture occurred. In the latter case, no AVCaptureSynchronizedDepthData object is present in the collection. The documentation for depthDataOutput(_:didDrop:timestamp:connection:reason:) says that the callback reports captured depth data that wasn't processed, with an empty-shell AVDepthData object. This appears to leave a diagnostic gap: if the TrueDepth pipeline fails before a depth object is produced, the application may receive neither an output callback nor a drop callback. System-log correlation We do not want to depend on these private and undocumented strings, but sysdiagnose/device logs from affected devices correlate the public-API silence with messages such as: Can't activate Pearl projector: no projector token found PROJECTOR ON DENIED Object too close (no depth) GMC status: -2 Projector GMC hasn't completed yet - dropping depth/dx buffers Pulses exceed limit. Projector Off In the first group, depth delivery never starts. In the second group, depth may never start or may stop after one frame. RGB capture can remain active in both cases. These logs suggest that depth or disparity buffers can be rejected before they reach AVCaptureDepthDataOutput, which would explain why the application sees neither didOutput nor didDrop. However, none of these projector, IR, calibration, protection, or token states appears to be exposed by a public API. Questions Is it expected that AVCaptureDepthDataOutput delivers neither didOutput nor didDrop when no depth object is produced upstream? Is the absence of AVCaptureSynchronizedDepthData from a synchronization collection the only public signal for this condition? Is there a supported API, notification, KVO property, or error that reports that the TrueDepth IR/projector/depth provider is unavailable, disabled, protected, stalled, or unable to produce data? Can an app distinguish a healthy but temporarily delayed depth stream from a pipeline that will never deliver depth, other than by implementing its own timeout and frame-count watchdog? Is there a documented validity rule for an empty, all-NaN, or near-constant AVDepthData.depthDataMap? Should applications treat these as unavailable depth even when the AVDepthData metadata reports a normal quality or accuracy? What is the recommended recovery procedure: rebuild AVCaptureDepthDataOutput, restart the capture session, switch cameras and switch back, wait for a system notification, or stop retrying until the device state changes? ARKit can report ARError.sensorFailed when a required sensor does not deliver input. Is there an AVFoundation equivalent that identifies which required input failed, without adopting ARKit? If no public signal currently exists, would Apple consider exposing a depth-provider status/error callback that distinguishes at least: captured but dropped; no depth capture for the timestamp; provider temporarily unavailable; and provider disabled or failed until recovery/service? We can provide a minimal sample project, affected-device sysdiagnose, and synchronized timestamp traces in Feedback Assistant if that would help. Please let us know which diagnostics and logging profiles would be most useful.
0
0
79
1w
AVCaptureVideoDataOutput stops zooming while AVCaptureVideoPreviewLayer continues — physical wide / ultra-wide / telephoto only
We use a single AVCaptureSession with AVCaptureVideoPreviewLayer and AVCaptureVideoDataOutput (preview-sized buffers, BGRA). When we increase videoZoomFactor, beyond a certain zoom level the image from AVCaptureVideoDataOutput no longer zooms further, while AVCaptureVideoPreviewLayer continues to zoom with the same zoom control. The preview and the video-data output therefore diverge. This behavior appears when the active camera is a physical lens device — wide, ultra-wide, or telephoto (e.g. builtInWideAngleCamera, builtInUltraWideCamera, builtInTelephotoCamera, or similar). It does not appear when the active input is a virtual / multi-camera (e.g. triple camera, dual-wide, or other system multi-camera). Are there known conditions under which this mismatch between preview and video-data output is expected? Thank you.
2
0
851
1w
AVAssetDownloadConfiguration does not download supplemental tracks on iOS 16
We're downloading HLS content for offline playback using AVAssetDownloadConfiguration. It's critical for our app that all subtitle tracks are downloaded together with the video, so users can watch any subtitle and audio option while offline. On iOS 16, not all tracks are downloaded, and when the user goes offline, playback fails with NSURLErrorNotConnectedToInternet (error -1009) because AVFoundation tries to fetch the missing subtitle segments over the network. On iOS 17 and later, everything works as expected — all subtitle and audio tracks are downloaded. However, on iOS 16 we observe two issues: First issue: Supplemental tracks are not downloaded on iOS 16 (device) Only the default audio track is downloaded. All non-default supplemental tracks — both subtitles and additional audio tracks — exhibit the same problem: Some supplemental tracks listed in the manifest don't even get a directory created in the .movpkg package Some supplemental tracks do get a directory created, but it contains only StreamInfoBoot.xml and StreamInfoRoot.xml — no actual segment data (.frag files) is downloaded. The StreamInfoBoot.xml for these directories shows NO, 0, and 0, with Supplemental Additionally, a Manifest.m3u8 stream directory is created with Complete=NO and no segment data. On iOS 17+ devices, all supplemental track directories are created and all contain segment data. Second issue: Downloads fail entirely on iOS 16 Simulator We receive the following error: Error Domain=NSURLErrorDomain Code=-16090 UserInfo={ NSLocalizedDescription=The operation could not be completed, _NSURLErrorFailingURLSessionTaskErrorKey=BackgroundAVAssetDownloadTask <DEC0D555-6C05-416C-91B8-F490F5A29046>.<3>, _NSURLErrorRelatedURLSessionTaskErrorKey=( "BackgroundAVAssetDownloadTask <DEC0D555-6C05-416C-91B8-F490F5A29046>.<3>" ), NSLocalizedFailureReason=An unknown error occurred (-16090) } On iOS 17+ Simulators, downloads work without issues. Our setup: let urlAsset = AVURLAsset(url: manifestURL) let preferredMediaSelection = try await urlAsset.load(.allMediaSelections) let config = AVAssetDownloadConfiguration(asset: urlAsset, title: title) config.primaryContentConfiguration.mediaSelections = preferredMediaSelection let task = assetDownloadURLSession.makeAssetDownloadTask(downloadConfiguration: config) task.resume() Example HLS manifest: #EXTM3U #EXT-X-VERSION:7 #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="French",LANGUAGE="fr",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_25/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="French (forced)",LANGUAGE="fr-x-cc",DEFAULT=NO,AUTOSELECT=YES,FORCED=YES,URI="s_31/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",LANGUAGE="en",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_37/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Kazakh",LANGUAGE="kk",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_43/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Uzbek",LANGUAGE="uz",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_49/stream.m3u8" #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aud_ec3",NAME="French 5.1",LANGUAGE="fr",DEFAULT=YES,AUTOSELECT=YES,CHANNELS="6",URI="a_18/stream.m3u8" #EXT-X-STREAM-INF:AUDIO="aud_ec3",SUBTITLES="subs",BANDWIDTH=1931408,VIDEO-RANGE=SDR,CODECS="hvc1.1.6.L90.90,ec-3",RESOLUTION=1024x428,FRAME-RATE=25,HDCP-LEVEL=NONE v_6/stream.m3u8 #EXT-X-SESSION-KEY:METHOD=SAMPLE-AES,URI="skd://XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",KEYFORMAT="com.apple.streamingkeydelivery",KEYFORMATVERSIONS="1" Questions: What is the correct way to make sure all supplemental tracks (subtitles, non-default audio) are actually downloaded on an iOS 16 real device? We pass asset.load(.allMediaSelections) to primaryContentConfiguration.mediaSelections — is there a different configuration, or should we use auxiliaryContentConfigurations, or something else entirely? Is there any way to make asset downloads work on the iOS 16 Simulator at all? We always get NSURLErrorDomain error -16090, while iOS 17+ Simulators work fine. If the supplemental tracks download bug on iOS 16 cannot be fixed or worked around from the app side, do you have any recommended approach for handling incomplete .movpkg packages at offline playback time?
1
0
364
2w
Variable aperture on the iPhone 18 Pro
Does iOS 27 provide an API which allows developers to set the iPhone 18 Pro camera's aperture? If it does, is the aperture going to be limited to just 4 discrete settings (like in the Camera app and like mentioned in the Tech Specs for the 18 Pro) or does the hypothetical iOS API allow us to set the aperture to a continuous value in the 0.0–1.0 range (similar to how a camera's focus can be set)?
2
1
600
2w
Capturing 48mp photos with .builtInTripleCamera
I am able to capture 48mp photos using .builtInWideAngleCamera, but it seems like .builtInTripleCamera is capped at 12mp? Is there a way to capture 48mp photos using .builtInTripleCamera? Because .builtInTripleCamera provides smooth transition between cameras during zooming, and I'd like to keep this behavior. New iPhone 17 Pro have all their cameras at 48mp. Is there a chance that their .builtInTripleCamera is capable of capturing 48mp? Or is this an API limitation?
4
0
931
2w
Working with kCVPixelFormatType_96VersatileBayerPacked12
Whilst AVCaptureSession is setup to capture ProRes RAW video, is it possible to get video pixel data which can read and processed, such as using CIImage(cvPixelBuffer: ) AVCaptureVideoDataOutput outputs ProRes RAW in kCVPixelFormatType_96VersatileBayerPacked12 pixel format. Is there a provided way to debayer this pixel format into something more usable?
1
1
827
2w
visionOS Developer Strap: UVC video works, but how can I access the capture device’s USB Audio Class (UAC) audio?
I’m developing a native visionOS application that uses an external HDMI capture device connected to Apple Vision Pro through the Developer Strap. The goal is to use Apple Vision Pro as a low-latency display for external HDMI sources such as an Xbox Series X, including both the HDMI video and the original digital game audio. My current hardware path is: Xbox Series X → HDMI → Elgato 4K X → USB-C → Apple Vision Pro Developer Strap The Elgato 4K X receives both video and audio from the Xbox over HDMI. What is working UVC video capture is working successfully on visionOS. The external Elgato capture device can be used as the video source and the Xbox video is displayed correctly inside the Vision Pro application. I have also verified the Elgato 4K X independently on macOS, where the device is recognized correctly. The problem I cannot find a supported way on visionOS to access the USB Audio Class (UAC) audio interface associated with the capture device. I specifically need the digital game audio originating from: Xbox HDMI → Elgato 4K X → USB/UAC → visionOS I do not want to use the Apple Vision Pro’s built-in microphone. Selecting/capturing the Vision Pro microphone would only record environmental sound and is not a substitute for the digital HDMI audio stream. Apple’s visionOS documentation for external devices through the Developer Strap appears to focus on UVC video capture. I haven’t been able to determine whether the accompanying UAC interface of a UVC/UAC composite capture device is exposed to visionOS applications. What I’ve tried / verified Connected an Xbox Series X to the HDMI input of an Elgato 4K X. Connected the 4K X to Apple Vision Pro through the Developer Strap. Successfully discovered and received the external UVC video stream. Successfully displayed the Xbox video feed in visionOS. Confirmed that the 4K X itself is recognized and functional on macOS. Attempted to obtain audio corresponding to the external capture source. The Vision Pro microphone is available as an audio source, but this is not the audio source required. I have not found a documented visionOS API that exposes the 4K X’s USB/UAC audio endpoint through the Developer Strap. Questions Does visionOS currently support USB Audio Class input devices connected through the Apple Vision Pro Developer Strap? If a device exposes both UVC video and UAC audio interfaces, is there a supported way to associate/access both interfaces from a visionOS application? If UAC is supported, which framework/API should be used to discover the external audio endpoint — AVFoundation, Core Audio, or another API? Does accessing the UAC interface require an entitlement in addition to UVC device access? If generic UAC access through the Developer Strap is currently unavailable to third-party visionOS applications, is there a restricted/managed entitlement that developers can request? If this isn’t currently supported, what is Apple’s recommended architecture for receiving synchronized video and audio from an HDMI capture device on Apple Vision Pro? Ultimately, I’m trying to achieve: HDMI source → UVC video + UAC audio → visionOS application → video displayed in Vision Pro + audio played through Vision Pro speakers/AirPods Since UVC video from this exact hardware is already working, I’m primarily trying to determine whether access to the accompanying USB audio interface is possible with the current visionOS SDK, requires another entitlement/API, or is currently unsupported. Any guidance from Apple or developers who have worked with composite UVC/UAC devices on Vision Pro would be greatly appreciated.
0
1
591
2w
CMLogItem.timestamp: which clock is it on, and how to convert to AVCaptureSession.synchronizationClock / host time?
We receive live relative-altitude updates from CMAltimeter.startRelativeAltitudeUpdates(to:withHandler:) on iPhone. CMAltitudeData inherits CMLogItem.timestamp, which the documentation describes as the time when the item is valid and as seconds since device boot. We need to determine whether a pressure measurement's event time falls inside an application operation bounded by a supported monotonic clock. Could Apple clarify the supported contract for CMAltitudeData.timestamp on iOS? Does CMAltitudeData.timestamp use the same epoch and rate as mach_absolute_time(), DispatchTime.uptimeNanoseconds, CACurrentMediaTime(), or ProcessInfo.systemUptime? If only some are compatible, which ones? Does it advance or pause during device sleep, device lock, application suspension, and background execution? In particular, are its sleep/suspension semantics guaranteed to match any of the clocks above? Is there a supported API for converting CMLogItem.timestamp to a host CMClock/Mach time, or for sampling “now” in the exact same clock domain used by CMAltitudeData.timestamp? Are the answers contractual across supported iPhone hardware and iOS releases, or are they implementation details that applications should not rely on? If direct comparison is not supported, what Apple-supported clock or conversion mechanism should an application use to compare a CMAltitudeData event time with two application-side monotonic operation boundaries? The question concerns clock semantics only. A minimal reproducer can be supplied if requested, but no application identifier, production data, sensor values, or user data is required to answer it.
Replies
4
Boosts
0
Views
333
Activity
12s
iPhone Duo: Does CameraCaptureAccessory accept touch, what counts as an active camera session, and can both front cameras run at once?
After watching "Leverage multiple displays and scenes on iPhone Duo" and "Build a great camera experience for iPhone Duo", we have four questions about the outer display while the device is open: Does a view hosted in CameraCaptureAccessory receive touch input on the outer display, or is it display-only? The accessory is described as available when the app "has an active camera session." Does that require a running preview layer or an attached output (photo, movie or data), or is a running AVCaptureSession with an input enough? Is a session whose main purpose is to host the accessory a supported use? Can AVCaptureMultiCamSession run the inner (under-display) front camera and the outer front camera at the same time on iPhone Duo? If so, which device-type combinations are supported, and are there thermal or power limits? Is CameraCaptureAccessory the only way for an app to show content on the outer display while the inner display is live, or are scene accessories that don't depend on the camera planned? Our use case is not photography. We're exploring a companion view for a second person sitting across from the device owner, so we want to understand whether the camera accessory is the right tool before designing around it.
Replies
4
Boosts
0
Views
40
Activity
15h
Camera direction and rotation in the half-folded pose on iPhone Duo
"Build a great camera experience for iPhone Duo" covers the device open and the device closed, including reconfiguring the session from AVCaptureDeviceDirectionCoordinator's change handler when the forward-facing camera changes. It does not cover the partially open pose, which "Leverage multiple displays and scenes on iPhone Duo" treats as a first-class hinge state with a continuous angle. Consider a camera app that keeps a live preview on screen and adopts AVCaptureDevice.RotationCoordinator — the documented remedy for cases like the square front sensor on iPhone 17, where a hard-coded 90° leaves the preview tilted — applying videoRotationAngleForHorizonLevelPreview to both the preview and the capture connection. In the partially open pose the two halves are at different physical attitudes: one stands up, the other lies flat. Which camera does AVCaptureDeviceDirectionCoordinator report as forward-facing while the hinge status is partially open? Does the hinge angle affect that decision? RotationCoordinator derives its angle from device attitude. With the two halves at different attitudes, which one does videoRotationAngleForHorizonLevelPreview reflect — the half carrying the active camera, the half hosting the preview layer, or a single device-level reference? If an app deliberately keeps the rear camera on the scene while its preview is drawn on the half facing the user, is that a supported configuration, or is the coordinator expected to override that choice? Is any of this modelled in the iPhone Duo simulator, or does the partially open pose require real hardware to test? Question 4 matters for planning: building against a rotation value that only appears on device would be worth knowing about before the hardware ships.
Replies
4
Boosts
0
Views
60
Activity
15h
Simultaneous capture and fold transitions in a live-streaming app on iPhone Duo
I build Lumecast, a live-streaming app (custom AVCapture pipeline feeding WebRTC). Three questions:   1. Does iPhone Duo support AVCaptureMultiCamSession — specifically front + rear simultaneously? Our co-casting feature would let a solo host fill two panes with their face and what they're looking at. How should a continuous capture session (live broadcast, can't pause) respond to camera direction changes as the device opens and closes? Is there a recommended pattern for swapping to the Virtual Front Camera mid-session without a visible gap for viewers?  3. Which cameras are available to third-party apps in each posture (folded / half-open / fully open), and does availability change at a specific hinge state? Goal: a host folds or unfolds mid-broadcast and their audience never sees a dropped frame.
Replies
3
Boosts
0
Views
90
Activity
1d
Is AVCaptureMultiCamSession supported on iPhone Duo
is AVCaptureMultiCamSession supported on iPhone Duo — specifically streaming two cameras that face opposite directions simultaneously (e.g. .builtInInnerUltraWideCamera + .builtInOuterUltraWideCamera, or the Virtual Front Camera plus a backward-facing camera)? I noticed the article recommends single-input reconfiguration over multi-cam; is that a performance recommendation, or is opposite-direction multi-cam unavailable on this device? Our use case is a live co-cast where one host fills two video panes: their face and what they're looking at.
Replies
0
Boosts
0
Views
15
Activity
1d
Where is the API for variable aperture of the iPhone 18 Pro?
According to Apple's press release (https://www.apple.com/pt/newsroom/2026/09/apple-debuts-iphone-18-pro-and-iphone-18-pro-max/): Variable aperture also provides more control for creative pros by giving them the ability to manually adjust any of the four aperture settings in the Camera app, and an API is available to developers for even more control across the aperture range in their apps. Can a DTS engineer specify the quoted API please? I can't see it anywhere in the documentation yet. What does it mean "even more control across the aperture range"? Will it be possible to set the aperture in a 0.0-1.0 range?
Replies
3
Boosts
3
Views
923
Activity
1d
Custom AVVideoCompositing on a composition-backed AVPlayerItem fails with AVErrorUnknown Xcode 27 beta 2 / beta 3
Trivial pass-through compositor fails on Xcode 27 (beta 2, beta 3); error code -11800 underlying error -12784. Repro included https://github.com/BugorBN/avplayer-custom-compositor-repro It works well on Xcode26 and lower
Replies
3
Boosts
3
Views
819
Activity
1d
Picture-In-Picture not starting with Custom Player
Team, I am trying to start PictureInPicture for my video from custom player as mentioned in the following link, but I am getting an error Steps: Created a player using AVPlayer. Initiated AVPictureInPictureController with AVPlayerLayer Added observer on PIP button enable status When PIP button is tapped executed pipController.startPictureInPicture() statement. PIP is not getting started & getting following error Project created in Xcode11 & currently using Xcode13.2 Error Domain=AVKitErrorDomain Code=-1001 "Failed to start picture in picture." UserInfo={NSLocalizedDescription=Failed to start picture in picture., NSLocalizedFailureReason=The UIScene for the content source has an activation state other than UISceneActivationStateForegroundActive, which is not allowed.} I didn't get any solutions online, plz help me on this issue. FYI: PIP button/mode is working if I use AVPlayerViewController(with default playerItem buttons) instead of AVPlayer & AVPictureInPictureController with custom controls Thanks in advance. Adopting Picture in Picture in a Custom Player
Replies
3
Boosts
2
Views
1.6k
Activity
1d
`LockedCameraCaptureManager` practically unusable since iOS 26
Somewhere since iOS 26, the LockedCameraCapture framework gets in an unpredictable state after opening the main app from the LockedCamera extension using LockedCameraCaptureSession.openApplication(for userActivity:). (Feedback with sample code to reproduce: FB21966835) Opening the extension from the lock screen again doesn’t open the extension but puts the lock screen in a state as if it has. Content updated from LockedCameraCaptureManager.shared.sessionContentUpdates comes in inconsistently, usually needs the app to be opened again or the extension to be opened. This makes using this extension impossible for me as I use it to record video files that manually need to be imported when the app is launched (so not through PhotoKit). Does anybody have a suggestion to circumvent this issue or how to get this fixed?
Replies
2
Boosts
3
Views
1.2k
Activity
2d
AVPlayerItemSampleBufferOutput
I am using AVPlayer, but I am unable to retrieve PCM data from .m3u8 streams via the AVPlayerItemSampleBufferOutput API in iOS 27 beta. I have tried both the Objective-C and Swift APIs: the Objective-C delegate methods are not being called, and the Swift methods nextAvailableSampleBuffer() and nextSampleBuffer() are returning no data.
Replies
2
Boosts
1
Views
631
Activity
2d
fail to get HLS realtime stream via AVPlayerItemSampleBufferOutputDelegate
Hi, I'm trying to do some realtime audio processing on audio served from an HLS stream (i.e. an AVPlayer created using an M3U HTTP URL). And I find that new APIs(AVPlayerItemSampleBufferOutput) are available in iOS 27.0 to achieve it. However, I failed to get the available data via the AVPlayerItemSampleBufferOutputDelegate. And I found some error in the system log: I create AVPlayerItemSampleBufferOutput and set the delegate after receiving the AVPlayerItemStatusReadyToPlay event. And here's my code: @interface OCAudioSamplebuffer () <AVPlayerItemSampleBufferOutputDelegate> @property (nonatomic, strong) AVPlayerItemSampleBufferOutput *bufferOutput; @property (nonatomic, strong) dispatch_queue_t bufferOutputQueue; @property (nonatomic, strong) AVPlayerItem *playerItem; @end - (void)playItem:(AVPlayerItem *)item { if (@available(iOS 27, *)) { AVAudioSession *audioSession = [AVAudioSession sharedInstance]; if([NSThread mainThread]){ [audioSession setCategory:AVAudioSessionCategoryPlayback withOptions:AVAudioSessionCategoryPlayback error:nil]; [audioSession setActive:YES error:nil]; }else{ dispatch_async(dispatch_get_main_queue(), ^{ [audioSession setCategory:AVAudioSessionCategoryPlayback withOptions:AVAudioSessionCategoryPlayback error:nil]; [audioSession setActive:YES error:nil]; }); } AVPlayerItemSampleBufferOutputAudioConfiguration *cfg = [[AVPlayerItemSampleBufferOutputAudioConfiguration alloc] init]; CMFormatDescriptionRef formatDescription = [self createPCMFormatDescriptionWithSampleRate:44100.0 channels:2 isFloat:YES]; cfg.requestedAudioFormat = formatDescription; if (formatDescription) { NSLog(@"create PCM Format Description success"); CFRelease(formatDescription); } else { NSLog(@"fail to create PCM Format Description"); } self.bufferOutput = [[AVPlayerItemSampleBufferOutput alloc] initWithConfiguration:cfg]; NSLog(@"create buffer success, PCM Format Description%@---,%@", cfg.requestedAudioFormat, formatDescription); self.bufferOutputQueue = dispatch_queue_create("audioSamplebufferQueue", DISPATCH_QUEUE_CONCURRENT); [self.bufferOutput setDelegate:self queue:self.bufferOutputQueue]; [item addOutput:self.bufferOutput]; self.playerItem = item; } else { // Fallback on earlier versions } } Any help. Thank you
Replies
2
Boosts
0
Views
908
Activity
2d
iOS 27 beta: Opening Notification Center pauses AVPlayer playback
Since iOS 27 beta we are seeing a behavior change in our video streaming app and I would like to know whether others can reproduce it. Behavior on iOS 27 beta: Fully opening the Notification Center pauses playback. Audio continues for about 5 more seconds, then the app is suspended. Closing the Notification Center leaves the player paused. On iOS 26 the same build keeps playing in this situation. Setup: AVQueuePlayer playing video with audio, not muted audiovisualBackgroundPlaybackPolicy = .automatic (default) AVAudioSession category .playback, UIBackgroundModes: audio entitlement AVPictureInPictureController attached with canStartPictureInPictureAutomaticallyFromInline = true Filed as FB23893965 Questions: Can anyone reproduce this on iOS 27 beta? Is this intentional or a regression?
Replies
1
Boosts
3
Views
787
Activity
6d
How can an app detect a TrueDepth pipeline failure when no depth output or drop callback is delivered?
How can an app detect a TrueDepth pipeline failure when no depth output or drop callback is delivered? We are using AVFoundation to stream synchronized RGB and depth data from the front-facing TrueDepth camera. On a small number of physical devices, the capture session and RGB stream remain healthy, but the depth pipeline becomes silent or returns unusable data. We would like to understand the intended public-API behavior and the supported way to detect this state without relying on private system logs or an application-defined timeout. We need an AVFoundation-only solution rather than ARKit. Capture configuration Our setup follows Apple's TrueDepth streaming sample: Discover .builtInTrueDepthCamera. Select an activeFormat with a nonempty supportedDepthDataFormats list. Set a supported activeDepthDataFormat, typically DepthFloat16. Add AVCaptureVideoDataOutput and AVCaptureDepthDataOutput. Enable the .depthData connection. Test both direct depth delegate delivery and AVCaptureDataOutputSynchronizer. Observe capture-session runtime errors, interruptions, connection state, and system pressure. The same code and configuration continuously deliver valid RGB and depth frames on healthy control devices. Observed failure modes 1. No depth callback of any kind The video delegate continues to receive RGB frames at the expected rate. The depth connection reports enabled and active. The capture session remains running. No AVCaptureSessionRuntimeErrorNotification or interruption is reported. System pressure is nominal. depthDataOutput(_:didOutput:timestamp:connection:) is never called. depthDataOutput(_:didDrop:timestamp:connection:reason:) is also never called. With AVCaptureDataOutputSynchronizer, RGB synchronization points may continue, but no AVCaptureSynchronizedDepthData object is present for the depth output. 2. One depth frame, followed by permanent silence On other affected devices, one depth frame is delivered shortly after startup, while hundreds of later RGB frames continue with no further depth output or drop callback. 3. A depth object is delivered, but the map is unusable We have also observed an AVDepthData object whose map is empty or has no usable spatial information, for example: a zero-sized/empty depth representation; an unfiltered map containing only NaN; or a filtered map containing a single constant value across nearly all pixels. Metadata such as depth quality, accuracy, calibration availability, supported formats, and connection state does not reliably distinguish these frames from healthy ones. Relevant documented behavior The documentation for AVCaptureSynchronizedDepthData.depthDataWasDropped says that a dropped depth object is different from a synchronization timestamp for which no depth capture occurred. In the latter case, no AVCaptureSynchronizedDepthData object is present in the collection. The documentation for depthDataOutput(_:didDrop:timestamp:connection:reason:) says that the callback reports captured depth data that wasn't processed, with an empty-shell AVDepthData object. This appears to leave a diagnostic gap: if the TrueDepth pipeline fails before a depth object is produced, the application may receive neither an output callback nor a drop callback. System-log correlation We do not want to depend on these private and undocumented strings, but sysdiagnose/device logs from affected devices correlate the public-API silence with messages such as: Can't activate Pearl projector: no projector token found PROJECTOR ON DENIED Object too close (no depth) GMC status: -2 Projector GMC hasn't completed yet - dropping depth/dx buffers Pulses exceed limit. Projector Off In the first group, depth delivery never starts. In the second group, depth may never start or may stop after one frame. RGB capture can remain active in both cases. These logs suggest that depth or disparity buffers can be rejected before they reach AVCaptureDepthDataOutput, which would explain why the application sees neither didOutput nor didDrop. However, none of these projector, IR, calibration, protection, or token states appears to be exposed by a public API. Questions Is it expected that AVCaptureDepthDataOutput delivers neither didOutput nor didDrop when no depth object is produced upstream? Is the absence of AVCaptureSynchronizedDepthData from a synchronization collection the only public signal for this condition? Is there a supported API, notification, KVO property, or error that reports that the TrueDepth IR/projector/depth provider is unavailable, disabled, protected, stalled, or unable to produce data? Can an app distinguish a healthy but temporarily delayed depth stream from a pipeline that will never deliver depth, other than by implementing its own timeout and frame-count watchdog? Is there a documented validity rule for an empty, all-NaN, or near-constant AVDepthData.depthDataMap? Should applications treat these as unavailable depth even when the AVDepthData metadata reports a normal quality or accuracy? What is the recommended recovery procedure: rebuild AVCaptureDepthDataOutput, restart the capture session, switch cameras and switch back, wait for a system notification, or stop retrying until the device state changes? ARKit can report ARError.sensorFailed when a required sensor does not deliver input. Is there an AVFoundation equivalent that identifies which required input failed, without adopting ARKit? If no public signal currently exists, would Apple consider exposing a depth-provider status/error callback that distinguishes at least: captured but dropped; no depth capture for the timestamp; provider temporarily unavailable; and provider disabled or failed until recovery/service? We can provide a minimal sample project, affected-device sysdiagnose, and synchronized timestamp traces in Feedback Assistant if that would help. Please let us know which diagnostics and logging profiles would be most useful.
Replies
0
Boosts
0
Views
79
Activity
1w
AVCaptureVideoDataOutput stops zooming while AVCaptureVideoPreviewLayer continues — physical wide / ultra-wide / telephoto only
We use a single AVCaptureSession with AVCaptureVideoPreviewLayer and AVCaptureVideoDataOutput (preview-sized buffers, BGRA). When we increase videoZoomFactor, beyond a certain zoom level the image from AVCaptureVideoDataOutput no longer zooms further, while AVCaptureVideoPreviewLayer continues to zoom with the same zoom control. The preview and the video-data output therefore diverge. This behavior appears when the active camera is a physical lens device — wide, ultra-wide, or telephoto (e.g. builtInWideAngleCamera, builtInUltraWideCamera, builtInTelephotoCamera, or similar). It does not appear when the active input is a virtual / multi-camera (e.g. triple camera, dual-wide, or other system multi-camera). Are there known conditions under which this mismatch between preview and video-data output is expected? Thank you.
Replies
2
Boosts
0
Views
851
Activity
1w
AVAssetDownloadConfiguration does not download supplemental tracks on iOS 16
We're downloading HLS content for offline playback using AVAssetDownloadConfiguration. It's critical for our app that all subtitle tracks are downloaded together with the video, so users can watch any subtitle and audio option while offline. On iOS 16, not all tracks are downloaded, and when the user goes offline, playback fails with NSURLErrorNotConnectedToInternet (error -1009) because AVFoundation tries to fetch the missing subtitle segments over the network. On iOS 17 and later, everything works as expected — all subtitle and audio tracks are downloaded. However, on iOS 16 we observe two issues: First issue: Supplemental tracks are not downloaded on iOS 16 (device) Only the default audio track is downloaded. All non-default supplemental tracks — both subtitles and additional audio tracks — exhibit the same problem: Some supplemental tracks listed in the manifest don't even get a directory created in the .movpkg package Some supplemental tracks do get a directory created, but it contains only StreamInfoBoot.xml and StreamInfoRoot.xml — no actual segment data (.frag files) is downloaded. The StreamInfoBoot.xml for these directories shows NO, 0, and 0, with Supplemental Additionally, a Manifest.m3u8 stream directory is created with Complete=NO and no segment data. On iOS 17+ devices, all supplemental track directories are created and all contain segment data. Second issue: Downloads fail entirely on iOS 16 Simulator We receive the following error: Error Domain=NSURLErrorDomain Code=-16090 UserInfo={ NSLocalizedDescription=The operation could not be completed, _NSURLErrorFailingURLSessionTaskErrorKey=BackgroundAVAssetDownloadTask <DEC0D555-6C05-416C-91B8-F490F5A29046>.<3>, _NSURLErrorRelatedURLSessionTaskErrorKey=( "BackgroundAVAssetDownloadTask <DEC0D555-6C05-416C-91B8-F490F5A29046>.<3>" ), NSLocalizedFailureReason=An unknown error occurred (-16090) } On iOS 17+ Simulators, downloads work without issues. Our setup: let urlAsset = AVURLAsset(url: manifestURL) let preferredMediaSelection = try await urlAsset.load(.allMediaSelections) let config = AVAssetDownloadConfiguration(asset: urlAsset, title: title) config.primaryContentConfiguration.mediaSelections = preferredMediaSelection let task = assetDownloadURLSession.makeAssetDownloadTask(downloadConfiguration: config) task.resume() Example HLS manifest: #EXTM3U #EXT-X-VERSION:7 #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="French",LANGUAGE="fr",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_25/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="French (forced)",LANGUAGE="fr-x-cc",DEFAULT=NO,AUTOSELECT=YES,FORCED=YES,URI="s_31/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",LANGUAGE="en",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_37/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Kazakh",LANGUAGE="kk",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_43/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Uzbek",LANGUAGE="uz",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_49/stream.m3u8" #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aud_ec3",NAME="French 5.1",LANGUAGE="fr",DEFAULT=YES,AUTOSELECT=YES,CHANNELS="6",URI="a_18/stream.m3u8" #EXT-X-STREAM-INF:AUDIO="aud_ec3",SUBTITLES="subs",BANDWIDTH=1931408,VIDEO-RANGE=SDR,CODECS="hvc1.1.6.L90.90,ec-3",RESOLUTION=1024x428,FRAME-RATE=25,HDCP-LEVEL=NONE v_6/stream.m3u8 #EXT-X-SESSION-KEY:METHOD=SAMPLE-AES,URI="skd://XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",KEYFORMAT="com.apple.streamingkeydelivery",KEYFORMATVERSIONS="1" Questions: What is the correct way to make sure all supplemental tracks (subtitles, non-default audio) are actually downloaded on an iOS 16 real device? We pass asset.load(.allMediaSelections) to primaryContentConfiguration.mediaSelections — is there a different configuration, or should we use auxiliaryContentConfigurations, or something else entirely? Is there any way to make asset downloads work on the iOS 16 Simulator at all? We always get NSURLErrorDomain error -16090, while iOS 17+ Simulators work fine. If the supplemental tracks download bug on iOS 16 cannot be fixed or worked around from the app side, do you have any recommended approach for handling incomplete .movpkg packages at offline playback time?
Replies
1
Boosts
0
Views
364
Activity
2w
Variable aperture on the iPhone 18 Pro
Does iOS 27 provide an API which allows developers to set the iPhone 18 Pro camera's aperture? If it does, is the aperture going to be limited to just 4 discrete settings (like in the Camera app and like mentioned in the Tech Specs for the 18 Pro) or does the hypothetical iOS API allow us to set the aperture to a continuous value in the 0.0–1.0 range (similar to how a camera's focus can be set)?
Replies
2
Boosts
1
Views
600
Activity
2w
Capturing 48mp photos with .builtInTripleCamera
I am able to capture 48mp photos using .builtInWideAngleCamera, but it seems like .builtInTripleCamera is capped at 12mp? Is there a way to capture 48mp photos using .builtInTripleCamera? Because .builtInTripleCamera provides smooth transition between cameras during zooming, and I'd like to keep this behavior. New iPhone 17 Pro have all their cameras at 48mp. Is there a chance that their .builtInTripleCamera is capable of capturing 48mp? Or is this an API limitation?
Replies
4
Boosts
0
Views
931
Activity
2w
Working with kCVPixelFormatType_96VersatileBayerPacked12
Whilst AVCaptureSession is setup to capture ProRes RAW video, is it possible to get video pixel data which can read and processed, such as using CIImage(cvPixelBuffer: ) AVCaptureVideoDataOutput outputs ProRes RAW in kCVPixelFormatType_96VersatileBayerPacked12 pixel format. Is there a provided way to debayer this pixel format into something more usable?
Replies
1
Boosts
1
Views
827
Activity
2w
AppleTV ContinuityCamera Scan QR Code issue
I found two issues with scan code to connect continuity cameras in tvOS QR code scanning connection timeout Scan the QR code to connect successfully, but the Camera has no data and the Capture Session does not report an exception. And these are not 100% reproduce.
Replies
0
Boosts
0
Views
480
Activity
2w
visionOS Developer Strap: UVC video works, but how can I access the capture device’s USB Audio Class (UAC) audio?
I’m developing a native visionOS application that uses an external HDMI capture device connected to Apple Vision Pro through the Developer Strap. The goal is to use Apple Vision Pro as a low-latency display for external HDMI sources such as an Xbox Series X, including both the HDMI video and the original digital game audio. My current hardware path is: Xbox Series X → HDMI → Elgato 4K X → USB-C → Apple Vision Pro Developer Strap The Elgato 4K X receives both video and audio from the Xbox over HDMI. What is working UVC video capture is working successfully on visionOS. The external Elgato capture device can be used as the video source and the Xbox video is displayed correctly inside the Vision Pro application. I have also verified the Elgato 4K X independently on macOS, where the device is recognized correctly. The problem I cannot find a supported way on visionOS to access the USB Audio Class (UAC) audio interface associated with the capture device. I specifically need the digital game audio originating from: Xbox HDMI → Elgato 4K X → USB/UAC → visionOS I do not want to use the Apple Vision Pro’s built-in microphone. Selecting/capturing the Vision Pro microphone would only record environmental sound and is not a substitute for the digital HDMI audio stream. Apple’s visionOS documentation for external devices through the Developer Strap appears to focus on UVC video capture. I haven’t been able to determine whether the accompanying UAC interface of a UVC/UAC composite capture device is exposed to visionOS applications. What I’ve tried / verified Connected an Xbox Series X to the HDMI input of an Elgato 4K X. Connected the 4K X to Apple Vision Pro through the Developer Strap. Successfully discovered and received the external UVC video stream. Successfully displayed the Xbox video feed in visionOS. Confirmed that the 4K X itself is recognized and functional on macOS. Attempted to obtain audio corresponding to the external capture source. The Vision Pro microphone is available as an audio source, but this is not the audio source required. I have not found a documented visionOS API that exposes the 4K X’s USB/UAC audio endpoint through the Developer Strap. Questions Does visionOS currently support USB Audio Class input devices connected through the Apple Vision Pro Developer Strap? If a device exposes both UVC video and UAC audio interfaces, is there a supported way to associate/access both interfaces from a visionOS application? If UAC is supported, which framework/API should be used to discover the external audio endpoint — AVFoundation, Core Audio, or another API? Does accessing the UAC interface require an entitlement in addition to UVC device access? If generic UAC access through the Developer Strap is currently unavailable to third-party visionOS applications, is there a restricted/managed entitlement that developers can request? If this isn’t currently supported, what is Apple’s recommended architecture for receiving synchronized video and audio from an HDMI capture device on Apple Vision Pro? Ultimately, I’m trying to achieve: HDMI source → UVC video + UAC audio → visionOS application → video displayed in Vision Pro + audio played through Vision Pro speakers/AirPods Since UVC video from this exact hardware is already working, I’m primarily trying to determine whether access to the accompanying USB audio interface is possible with the current visionOS SDK, requires another entitlement/API, or is currently unsupported. Any guidance from Apple or developers who have worked with composite UVC/UAC devices on Vision Pro would be greatly appreciated.
Replies
0
Boosts
1
Views
591
Activity
2w