How to display stereo images in Apple Vision Pro?

Hi community,

I have a pair of stereo images, one for each eye. How should I render it on visionOS? I know that for 3D videos, the AVPlayerViewController could display them in fullscreen mode. But I couldn't find any docs relating to 3D stereo images.

I guess my question can be brought up in a more general way: Is there any method that we can render different content for each eye? This could also be helpful to someone who only has sight on one eye.

Replies

I have the same question.😭😭😭 There is nothing about the stereo image in the documents. Is there any solution you found?

Using Reality Composer Pro, you can make a custom material that feeds the left and right images into a GeometrySwitchCameraIndexNode.

  • @mayoff I was able to create custom material like that. I also specified two images for left and right. After I applied it to just a simple cube and ran the app, but no material rendered on a cube.

    Can you please elaborate on how to properly use that material? I would imagine we need to have some sort of a custom plain/canvas that we can apply this material, so it renders the stereoscopic image on it once presented on screen? Also, is there a way to dynamically set those images in code? TY!

  • Hey anyone have any further success? looking to do the same

  • How do you specify the images from the app?

Add a Comment

The above Shader Graph from @mayoff works fine for me, using known good stereo image pairs. However, the preview on the Mac shows only a black material. You can only see the stereo output on actual hardware.

I the shader and named it TestStereo. Then I created a Cube in RCP, and changed the material from DefaultMaterial to TestStereo. When I did this, the cube became not quite opaque (I could now faintly see the grid through the cube). When I preview the scene on an actual Vision Pro, it looks great. Well, the stereo image part looks great, but I now have the image shown on all 6 faces of the cube. So I'll experiment with adding a billboard component (as seen in the SwiftSplash sample game, and build a simple Scene that I can drive with arbitrary left/right images.

The image shows a color difference between Reality Composer Pro and AVP, with AVP appearing noticeably brighter. What could be the reason for this? @mayoff thx:)

@kandao-xr I Believe that you are seeing this increase of brightness, because on your shader, you are adding the left image for both, LEFT camera index and MONO, that could be drawing two times (duplicating the brigthness of the image) when rendering the left Eye. Could you test just remove the Connection between left-image -> [mono]CameraIndex, to see if will solve?

thx for you repley, i remove mono to check, it's not work, when i use the previewsurface the color seem to normal, maybe it's the unlitsuiface problem