App access to Vision Pro's cameras and microphone?

I recommend that you watch this WWDC video: https://developer.apple.com/videos/play/wwdc2023/10094/?time=588

The link is time stamped to the relevant portion of the video, which explains camera availability on Vision Pro.

Note the odd wording of the video you were told to watch: https://developer.apple.com/videos/play/wwdc2023/10094/?time=588

"If no spatial personal is found on this device, then no camera frames will return to apps."

Does this mean that ONLY if a shared session is active and has at least one other "spatial personal" present, that it is possible to obtain video?

That you can't stream yourself by yourself performing ad narrating a task, such as performing a complex calculation or playing a game or giving a lecture or... reporting a bug?

You combine this with the odd wording in this interview:

https://youtu.be/DgLrBSQ6x7E?t=4163

Greg Joswiak: Right, look, because one of the challenges we had in making teh [keynote] video is the fact that we have to take this incredible spatial experience and try to translate it onto a 2D screen. But all the UI you see, all the stuff that we were showing coming out of the device was rendered on device. And its out there, even in the third person view, that's composited onto a scene. So this isn't like us having graphic artists with an M2 ultra coming up with all this stuff this is all coming off of...

Mike Rockwell: It's all rendered real time.

Greg Joswiak: Yeah. Realtime and then that's how we showed it in the film and that's important. That was not fake.

.

In other words, it took a lot of work to get that out of the AVP. Had it been built in as I'm asking for, they would have said: "oh yeah, 2D composite video out is an important usecase, so we used the built-in video-out feature and in fact, the scene of the person wearing the AVP was shot by another person wearing the AVP."

Instead, you have this official description of when cameras can be used by developers: "If no spatial personal is found on this device, then no camera frames will return to apps" combined with a detailed discussion of how important it was to get real video (implication how difficult it was to do) and you'll realize that they left off the single most important use-case for early adopters, both developers AND powerusers:

The power users get to put up a youtube of themselves "in world" doing random stuff to show off their new toy while the developers can fire a 2D video of their own app "in world" doing something unexpected and say "this is what we expected and yet THIS is what we actually saw on the device."

.

So there appears to be a severe limitation to what you can show other people who are NOT wearing an AVP.

IF it is at all possible, this has to change.

Filiming yourself doing something and presenting it to others, both in streaming (to a video projector) or as a youtube video is an extremely important usecase and my impression is that this is NOT supported at all.

Now, I undertant that Netflix doesn't want their movies restreamed, but that is trivial to handle:

have a permission for all apps that is automatically set to false — allow 2D streaming — so only apps whose creators want the content streamable can be streamed.

If any app in a space has "allows 2D streaming" set to false, 2D streaming in shared space cannot happen, and the user will need to quit that app.

Only if ALL running apps (including no apps at all) have set "allows 2D streaming" to true, can the user stream their shared space. If they try and an app is running that disallows this, they must first quit that app and try again. HI stuff to be worked out, of course.

I still don't get it. Will it be possible to capture photos or record videos (of the real world) on the device?