With quite some excitement I read about visionOS 2's new feature to automatically turn regular 2D photos into spatial photos, using machine learning. It's briefly mentioned in this WWDC video: https://developer.apple.com/wwdc24/10166 My question is: Can developers use this feature via an API, so we can turn any image into a spatial image, even if it is not in the device photo library? We would like to download an image from our server, convert it on the visionPro on-the-fly and display it as a spatial photo.

Our app needs to scan QR codes (or a similar mechanism) to populate it with content the user wants to see. Is there any update on QR code scanning availability on this platform? I asked this before, but never got any feedback. I know that there is no way to access the camera (which is an issue in itself), but at least the system could provide an API to scan codes. (It would be also cool if we were able to use the same codes Vision Pro uses for detecting the Zeiss glasses, as long as we could create these via server-side JavaScript code.)

How can I remove or hide this part under a SwiftUI panel? What Swift code should I write to control this?

I can get the fully immersive rendering working with metal and composite services but in WWDC 24 rendering metal with passthrough was announced: https://developer.apple.com/wwdc24/10092. I watched the video and downloaded the test project. I noticed that the passthrough was showing up in the demo project but not in my metal project. After debugging I found out it was this key: Preferred Default Scene Session Role in my Info.plist that was set to Compositor Services Immersive Space Application Session Role (like the video said) but it needed to be set to Window Application Session Role for the passthrough to come in. Is this a bug?

I followed the WWDC video to learn Sharplay. I understood the first creation of seats, but I couldn't learn some of the following content very well, so I hope you can give me a list code. The contents are as follows: I have already taken a seat. struct TeamSelectionTemplate: SpatialTemplate { let elements: [any SpatialTemplateElement] = [ .seat(position: .app.offsetBy(x: 0, z: 4)), .seat(position: .app.offsetBy(x: 1, z: 4)), .seat(position: .app.offsetBy(x: -1, z: 4)), .seat(position: .app.offsetBy(x: 2, z: 4)), .seat(position: .app.offsetBy(x: -2, z: 4)), ] } I hope you can give me a SharePlay Button. After pressing it, it will assign all users in Facetime to a seat with elements quantified in TeamSelectionTemplate. Thank you very much.

I tried to show spatial photo on my application by swiftUI's Image but it just show flat version of it even I Use Vision Pro, so, how can I show spatial photo to users, does there any options for this?

I would like to code some RealityViews to run on my Mac first (and then incorporate them in a visionOS project) so that my code/test loop is faster, but I have not been able to find a simple example that supports Mac. Is it possible to have volumes on a Mac? Is there support for using a game controller to move around the RealityView, like in the visionOS simulator?

Has there been an adjustment in the maximum number of DrawableQueues that can be swapped for textures in VisionOS 2? Or an adjustment in the total amount of RAM allowed in a scene? I have been having a difficult time getting more than one DrawableQueue to appear when it worked fine in VisionOS 1.x.

Hi community, I have a pair of stereo images, one for each eye. How should I render it on visionOS? I know that for 3D videos, the AVPlayerViewController could display them in fullscreen mode. But I couldn't find any docs relating to 3D stereo images. I guess my question can be brought up in a more general way: Is there any method that we can render different content for each eye? This could also be helpful to someone who only has sight on one eye.

When using .mov video files for creating video materials in RealityKit, they display correctly on my modelEntity. However, when I tried using a video file in the .mp4 format, I only get a solid black material. Does AVKit support playing .mp4 video files on visionOS 1.2?

How to display the user's own persona in a view

Can a TabletopKit game with a variable number of players also have a variable number of AI, each playing a specific position? For example, a game that can handle 3-7 players, being set up for a specific game where 4 of the players are real, and an AI plays 2 other locations? The Kit appears to be setup to only be able to handle real (human) players. Thank you!

Does the current version of TabletopKit support having two or more game players to be at the same physical location? In these cases, the players would not want to see a Facetime persona around the table but instead should be able to see the physical player. Any other remote players would be able to see the personas of those players since they are not at that location. There are a couple of issues in this scenario (shared position of the board, players' location around the table, etc.), but they should be solvable. Thank you!

In the example https://developer.apple.com/documentation/imageio/writing-spatial-photos, we see that for each image encoded with the photo we include the following information: kCGImagePropertyGroups: [ kCGImagePropertyGroupIndex: 0, kCGImagePropertyGroupType: kCGImagePropertyGroupTypeStereoPair, (isLeft ? kCGImagePropertyGroupImageIsLeftImage : kCGImagePropertyGroupImageIsRightImage): true, kCGImagePropertyGroupImageDisparityAdjustment: encodedDisparityAdjustment ], Which will identify which image is left, and which is right, also information about group type = stereo pair. Now, how do you read those back? I tried to implement a reading simply with CGImageSourceCopyPropertiesAtIndex, and that did not work, getting back "No property groups found." func tryToReadThose() { guard let imageData = try? Data(contentsOf: outputImageURL), let source = CGImageSourceCreateWithData(imageData as NSData, nil) else { print("cannot read") return } for i in 0..<CGImageSourceGetCount(source) { guard let imageProperties = CGImageSourceCopyPropertiesAtIndex(source, i, nil) as? [String: Any] else { print("cannot read options") continue } if let propertyGroups = imageProperties[String(kCGImagePropertyGroups)] as? [Any] { // Process the property groups as needed print(propertyGroups) } else { print("No property groups found.") } //print(imageProperties) } } I assume maybe CGImageSourceCopyPropertiesAtIndex expects something as a 3rd parameter. But in the https://developer.apple.com/documentation/imageio/cgimagesource "Specifying the Read Options" I don't see anything related to that.

I've been trying to get the drag gesture up and running so I can move my 3D model around in my immersive space, but for some reason I am not able to move it around. The model shows up in my visionOS 1.0 Simulator, but I can't seem to get it to move around. Would love some help with this and some resources too that would be helpful. Here's a snippet of my Reality View code import SwiftUI import RealityKit import RealityKitContent struct GearRealityView: View { static var modelEntity = Entity() var body: some View { RealityView { content in if let model = try? await Entity(named: "LandingGear", in: realityKitContentBundle) { GearRealityView.modelEntity = model content.add(model) } }.gesture( DragGesture() .targetedToEntity(GearRealityView.modelEntity) .onChanged({ value in GearRealityView.modelEntity.position = value.convert(value.location3D, from: .local, to: GearRealityView.modelEntity.parent!) }) ) } }

Posting here as I did not see a section for Dev Documentation portal Using the search box in the documentation portal I searched for "frustum" hoping to find any APIs that game me control over frustum culling. https://developer.apple.com/search/?q=frustum&type=Documentation The search came up empty for hits in RealityKit. Hours later I found the boundsMargin API which explains how it affect frustum culling. I went back and tried the search again to verify the documentation search result were incomplete. site:developer.apple.com/documentation/realitykit frustum on google worked fine. Fixing this can save everyone time and stress.

Hello! Is it possible to turn on hand pass-through (hand cutout, not sure what is the correct name, real hands from the camera) in WebXR when in immersive-vr and hand tracking feature is enabled? I only see my hands when I disable hand tracking feature in WebXR, but then I don't get the joints positions and orientations.

Hello. I am trying to load my own Image Based Lighting file in a visionOS RealityView. I used the code you get when creating a new project from scratch and selecting the immersive space to full when creating the project. With the sample file Apple provides, it works. But when I put my image in PNG, HEIC or EXR format in the same location the example file was in, it doesn't load and the error states: Failed to find resource with name "SkyboxUpscaled2" in bundle In this image you can see the file "ImageBasedLight", which is the one that comes with the project and the file "SkyboxUpscaled2" which is my own in the .exr format. if let immersiveContentEntity = try? await Entity(named: "Immersive", in: realityKitContentBundle) { content.add(immersiveContentEntity) do{ let resource = try await EnvironmentResource(named: "SkyboxUpscaled2") let iblComponent = ImageBasedLightComponent(source: .single(resource), intensityExponent: 0.25) immersiveContentEntity.components.set(iblComponent) immersiveContentEntity.components.set(ImageBasedLightReceiverComponent(imageBasedLight: immersiveContentEntity)) }catch{ print(error.localizedDescription) } Does anyone have an idea why the file is not found? Thanks in advance!

I am getting the error "Initializing hosting entity without a context" in the console when I build and run my game in XCode 16.0 beta, targeting Vision Pro OS 2.0 (22N5252n). Not sure where the error is originating.

I was wondering of anyone had guidance on how to “livestream“ MV-HEVC content. More specifically, I have a left and right eye view for stereoscopic content (perhaps, for example, the views were taken from a stereoscopic video being passed through an AVPlayer). I know, based on sample code, that I can convert the stereoscopic video into a MV-HEVC file using AVAssetWriter. However, how would I take the stereoscopic video and encode it, in realtime, to a stream that could then leverage HLS Tools to deliver to clients? Is AVFoundation capable of this directly? Or is there an API within VideoToolbox that can help with this?