Will it be possible to get direct access to all the different cameras of Vision Pro via AVFoundation or indirectly via other frameworks?
Which sensors (inner and outer color & IR cameras, LiDAR, TrueDepth camera) will developers have access to on visionOS?
Hi @rd1, AVFoundation is available on visionOS, but apps will not have access to cameras. Check out the session Enhance your iPad and iPhone apps for the Shared Space for more information about how existing iPadOS and iOS apps can interact with the hardware:
However, many of these cameras are not available for app use. It is critical to use discovery sessions to detect which cameras and microphones are available for use. To ensure apps have an excellent capture experience, use AVCaptureDevice discovery session to confirm hardware availability.
You say: "Apps will not have access to cameras.", but the cited text says "many of these cameras are not available for app use." This gives room for at least one camera to be usable. So my question is: Will it be possible to capture a photo or a video using the Vision Pro? I would mainly need this for capturing the real world, without any AR content.
The linked video at 10:17 advices the audience to use AVCaptureDevice.DiscoverySession but XCODE 15.2 (15C500b) error says 'DiscoverySession' is unavailable in visionOS and even though the video doesn't state that no camera is available () to me it seems it is not, at least right now. Any other experiences?
Thank you, @Developer Tools Engineer. It seems that not even ARKit will let developers access any sensor data: https://developer.apple.com/videos/play/wwdc2023/10082?time=262
In the WWDC video you shared, it was mentioned that only a "composite .front camera" will be abailable via AVCapture DiscoverySession. May I ask what is meant by "composite camera"? Does it mean that visionOS will provide stream of RGB-only frames captured by a virtual camera, which is created by internally combining data from several hardware cameras?
Or is the meaning of "composite" meant to be "multiple modalities" (i.e. AVFoundation providing access to RGB and depth output via a virtual composite camera)?
Is AVFoundation going to be the only Framework on visionOS capable of providing access to (composite) camera data, as suggested by the video you shared?
Even if no 3rd party app provides this 2D streaming output, Apple should still supply a 2D streaming capability so college professors can use this thing to lecture to classes via a 2D video projector and social media influencers can use it to stream to youtube.
Demos of how to program it will be MUCH more compelling, and tutorials far more popular, if they can be performed in the 3D space and exported to 2D for others to watch.
Note that this is possible now with the much less expensive iPhone 15 Pro. Just record spatial video (3D) with it.
