Ultra-wide camera in world tracking

My app previews cinema lenses by cropping the AR feed, so I can only simulate lenses tighter than the main camera's ~24mm-equivalent, meaning wider cinema lenses are impossible to accurately preview. ARConfiguration.VideoFormat exposes captureDeviceType, but world-tracking formats only ever list the wide-angle module. Is running world tracking on (or alongside) the ultra-wide camera fundamentally infeasible? Or just a case of not enough Feedback Assistant requests?

Answered by Engineer in 891234022

World tracking automatically uses Ultra Wide camera on supported iOS devices. It is unavailable in the API, because it is not configurable.

World tracking automatically uses Ultra Wide camera on supported iOS devices. It is unavailable in the API, because it is not configurable.

Thanks! Since the Ultra Wide is already streaming during world tracking:

  1. Is there any supported way today to get Ultra Wide pixels while an ARSession runs? (My understanding is a concurrent AVCaptureSession gets interrupted once ARKit owns the cameras.)

  2. If not, would read-only access to those frames — e.g. a secondary pixel buffer on ARFrame with its own intrinsics/extrinsics, no configurability, in the same spirit as scene depth — be a reasonable Feedback request? Or is there a deeper reason (calibration, exposure control, bandwidth) that feed can't be surfaced for rendering?

Ultra-wide camera in world tracking
 
 
Q