Discuss Spatial Computing on Apple Platforms.

Post

Replies

Boosts

Views

Activity

Access main camera on Apple Vision Pro
From visionOS 2.0 we can access Apple Vision Pro's main camera but only for Enterprise account as it is enterprise API only, I have a normal Developer account and I want to use main camera and want to have a video call feature in app by using main camera of AVP, is it possible to do it using developer account only. Currently using that account I am not able to create entitlement certificate as there is no option.
2
0
211
2w
How to present an alert in visionOS immersive space?
My visionOS app uses an immersive view. If the app encounters an error, I want to present an alert. I tried in a demo app to present such an alert, but it is not shown. Nearly the same code on iOS presents an alert window. Here is my demo code, based on Apple's Immersive Environment App template: import SwiftUI import RealityKit import RealityKitContent struct ErrorInfo: LocalizedError, Equatable { var errorDescription: String? var failureReason: String? } struct ImmersiveView: View { @State private var presentAlert = false let error = ErrorInfo( errorDescription: "My error", failureReason: "No reason" ) var body: some View { RealityView { content, attachments in let mesh = MeshResource.generateBox(width: 1.0, height: 0.05, depth: 1.0) var material = UnlitMaterial() material.color.tint = .red let boardEntity = ModelEntity(mesh: mesh, materials: [material]) boardEntity.transform.translation = [0, 0, -3] content.add(boardEntity) } update: { content, attachments in // … } attachments: { // … } .onAppear { presentAlert = true } .alert( isPresented: $presentAlert, error: error, actions: { error in }, message: { error in Text(error.failureReason!) }) } } Since I cannot see any alert, is something wrong with my code? How should an alert be presented in immersive space?
1
0
198
2w
dismissWindow() doesn't dismantle View
We have discovered that our UIViewRepresentable view isn't being dismantled after its window is dismissed via dismissWindow(). This seems to result in a leak of our custom Coordinator class. Every time the user opens a new window, a new Coordinator is created; if the user then dismisses the window manually, or we dismiss it programmatically, the Coordinator remains in memory with no way to destroy it. Is this expected behavior? How can we be sure to clean up our Coordinator when the view's window is closed? Thanks.
0
0
149
2w
Can we prevent Apple Vision Pro from turnning off the display and going to sleep as long as take off Apple Vision Pro?
In some cases, we will put the Apple Vision Pro back on for a short period of time after taking it off, such as less than 1 minute, and want to keep it activte when we take it off so that it can continue to work seamlessly when we put it on again. The current Apple Vision Pro turns off the display and goes to sleep whenever it is been taken off. This feature is also explained in the user guide, to save power, for safety, etc. However, we want a seamless experience! Is it possible to have a screen saver like macOS or iOS, where we can set our own delay time to go sleep? Or is there any API that can be called to prevent going to sleep?
0
0
229
3w
3D display from 3D camera
I want to pursue for a project involving 3D VR visualisation. I would like to know if there is a 3D stereoscopic camrea setup that is able to connect straight to Apple Vision Pro display yet. Of course, with no compatibility issues with MV-HEVC. Any recommendation is appreciated.
0
0
204
3w
visionOS Simulator Rotate and Scale gestures difficult to register (capture)
We were having an issue wrb the system rotate and scale gestures (two-handed gestures / RotateGesture3D and MagnifyGesture) were extremely difficult to register (make work) in the visionOS simulator. The solution we found was to: Launch your app in the simulator Move the pointer on top of the 3D object for which you are testing rotation and scaling gestures. Press and hold the Option key to display touch points (ie: the two-handed gesture points). While maintaining the option key pressed, release the pointer and re-enable it again. I am using a track pad with tap-to-click enabled and three-finger to drag enabled in accessibility, so "release the pointer and re-enable it again" translates simply to removing the three finger and placing them again on the trackpad. If you have maintained the option key pressed, then you should now be able to rotate and scale the 3D object. Context if you are interested: Our issue was also occurring in Apple's own sample project relating to gestures "Transforming RealityKit entities using gestures", at below link. On Apple's article "Interacting with your app in the visionOS simulator" at the below link, for two-handed gestures it states "Press and hold the Option key to display touch points. Move the pointer while pressing the Option key to change the distance between the touch points. Move the pointer and hold the Shift and Option keys to reposition the touch points." This simply did not work anymore for rotation and scaling gestures. These gestures used to be a lot more responsive in Sonoma. Either the article should be updated to what I described above, or there is an issue. Our colleague who is using macOS Sonoma 14.6.1 with the latest release of Xcode is not having these issues. Here is the list of configurations (troubleshooting we tried!) where it is difficult to achieve rotation and scaling gestures in the visionOS simulator: macOS Sequoia 16.1 Beta, Xcode 16.1 RC w visionOS 2.1 macOS Sequoia 16.1 Beta, Xcode 16.1 RC w visionOS 2.0 macOS Sequoia 16.1 Beta, Xcode 16.2 Beta 1 w visionOS 2.1 macOS Sequoia 16.1 Beta, Xcode 16.2 Beta 1 w visionOS 2.0 macOS Sequoia 16.1 Beta, remove all Xcodes and installed the build from AppStore (Xcode 16.1) macOS Sequoia 16.1 Beta, Xcode 16.0 w visionOS 2.0 completely wiped out, and reset entire development machine, re-installed latest releases of sequoia (15.1) and xcode (15.1)) Throughout these troubleshooting I often: restarted both xcode and sim erased all derived data erased all contents and settings from sims performed fresh git clones None of the above worked, only the workaround described above works atm. As you can maybe deduce, it was very time consuming to find the workaround, we also wasted some development effort thinking our gesture development was no-good. Hopefully this will help other devs. Article Link: https://developer.apple.com/documentation/xcode/interacting-with-your-app-in-the-visionos-simulator Gesture sample project link: https://developer.apple.com/documentation/realitykit/transforming-realitykit-entities-with-gestures
2
0
333
3w
VisionOS 2.0 App Crashes Faster with Each Upload from XCode
We developed an app for VisionOS 2.0 Beta in XCode 16 Beta. The development was done in the beta versions since we needed features for our app which were not available in VisionOS 1.0, which was the most recent stable release at the time we developed the app. The app was fully functional on our AVP running VisionOS 2.0 Beta Version 5, and we never had any errors. We did not publish the app to the app store since we are a research lab using the app for teleoperating a custom robot. Last week, we upgraded the AVP from VisionOS 2.0 Beta Version 5 to VisionOS 2.0 (stable release). Unfortunately, once we upgraded to 2.0, we began to have an issue with the app. While the app is running, at seemingly random times, without any new functionality being used within the app (no new buttons being pressed, etc), we encounter the following console error: assertion failure: 'index < m_size' (operator[]:line 1011) Index out of range. index = 18446744073709551615, size = 0 We could re-upload the app to the AVP and successfully operate the app for several minutes until the same error occurred again. We thought to use Apple Configurator to flash VisionOS 2.0 Beta 5 to the AVP since the error wasn't happening on the previous firmware, but we were unable to flash a beta version of VisionOS via Apple Configurator, so we simply performed a factory reset of the device (on VisionOS2.0, by pressing restore in Apple Configurator with the AVP connected via the developer strap) to see if this might fix the issue. After doing a factory reset, we thought the console error completely went away. We were able to operate the app for ~3 hours on Sunday with no issues. Then, yesterday (Monday) we operated the app for another 2 hours, and at the very end of using the app, it crashed with the same error. We re-uploaded the app with XCode, and the error occurred again after about 20 mins of using the app. This cycle repeated, and every time we re-uploaded the app, the time it took for the error to occur decreased, until we uploaded the app and the error occurred in <20 seconds. We decided to test our hypothesis by upgrading VisionOS to 2.1 and using XCode 16. Similarly, we were able to run the app on the AVP for 2 hours, then the error occurred. The next time we ran the app, the error occurred within 20 minutes, then after reloading, occurred within 5 mins, then 2 mins, etc. We are pretty stumped on why the app would work after a factory reset or a firmware upgrade for hours, then fail faster and faster every time we re-upload the app from Xcode. We are not experienced in debugging Swift and ObjC, so we wanted to inquire if this is an issue that you have ran into before to point us in the right direction. We think that it could be a problem with the cached memory that persists on the device across uploads from Xcode, but that's the extent of our understanding. P.S., we also experienced this error during some of the app failures, but the one above is the most common: assertion failure: Index out of range (operatorl]:line 858) index = 576460752303423487, max = 1
3
0
231
3w
[Apple Vision Pro] Issue with startDeviceMotionUpdates in ImmersiveSpace Mode
Hello everyone, I’m developing an app for Apple Vision Pro, and I’m trying to retrieve motion data updates aligned to magnetic north by using the following method: startDeviceMotionUpdates(using: .xMagneticNorthZVertical, to: .main) { ... } The goal is to get motion data oriented to magnetic north while an ImmersiveSpace() with an immersiveStyle set to .mixed is active. However, with this setup, I receive no updates at all. If I switch to: startDeviceMotionUpdates(using: .xArbitraryZVertical, to: .main) { ... } or startDeviceMotionUpdates(to: .main) { ... } then I do receive data, but it’s not aligned as required (I specifically need .xMagneticNorthZVertical). Has anyone experienced a similar issue, or does anyone know how to enable updates aligned to magnetic north in this configuration? Thanks in advance for any insights! SDK: VisionOS 2.0
1
0
180
3w
Detecting collisions between fingertip and world mesh
I'm using hand tracking to detect collisions between fingertips and entities that I have placed in the scene. I'm using the .mixed environment. However, I want to detect when a fingertip touches a real-world object such as a wall. No matter what I try, I can't get the collision to fire. I'm using the SceneReconstructionProvider to give me world meshes, which I use to create ModelEntity objects to which I add a CollisionComponent with the shape of the object. I can render the meshes just fine, but nothing I do seems to allow collisions to work. Surely this is possible, what am I missing?
4
0
210
3w
iOS 18.1 Rc
I’ve updated to iOS 18.1 public beta when it released and I got the problem when I was in public beta 4 to rc and the problem never fixed with the time limit when I ask for more time it never give me more time it’s just says waiting for parent approval I have this problem until ios 18.1 RC can u fix it I’m on iPhone 11 pro max
1
0
322
3w
Synchronizing Physical Properties of EntityEquipment in TableTopKi
I am working on adding synchronized physical properties to EntityEquipment in TableTopKit, allowing seamless coordination during GroupActivities sessions between players. Current Approach and Limitations I have tried setting EntityEquipment's state to DieState and treating it as a TossableRepresentation object. This approach achieves basic physical properties synchronized across players. However, it has several limitations: No Collision Detection Between Dice: Multiple dice do not collide with each other. Shape Limitations: Custom shapes, like parallelepipeds, cannot be configured. Below is my existing code for Base Entity Equipment without physical properties: struct CubeWithPhysics: EntityEquipment { let id: ID let entity: Entity var initialState: BaseEquipmentState init(id: ID, entity: Entity) { self.id = id self.entity = entity initialState = .init(parentID: .tableID, pose: .init(position: .zero, rotation: .zero), entity: self.entity) } } I’d appreciate any guidance on the recommended approach to adding synchronized physical properties to EntityEquipment.
4
4
385
3w
Determining if an ObjectAnchor is currently observed
I'm writing code using ObjectAnchor for Vision OS. If an object is tracked, and then becomes not visible (either because the user looked in a different direction, or because the tracked object was occluded by another object), it is still tracked and you get anchor updates (e.g., object permanence). For my application, it would be very helpful if I could determine if the object is currently being observed, or it is not currently observed and just assumed to be in the same location as seen previously. ObjectAnchor.isTracked just seems to indicate whether it is getting anchor updates. I don't see anything in the ObjectAnchor or AnchorUpdate that would allow me to determine if the object is currently observed. Does anyone know of a way to do this, or would this be a feature request?
2
0
155
3w
Xcode 16 crashes my Vision Pro Object Tracking App
I created an Object & Hand Tracking app based on the sample code released here by Apple. https://developer.apple.com/documentation/visionos/exploring_object_tracking_with_arkit The app worked great and everything was fine, but I realized I was coding on Xcode 16 beta 3, so I installed the latest Xcode 16 from the App Store and tested by app there, and it completely crashed. No idea why. Here is the console dyld[1457]: Symbol not found: _$ss13withTaskGroup2of9returning9isolation4bodyq_xm_q_mScA_pSgYiq_ScGyxGzYaXEtYas8SendableRzr0_lF Referenced from: <3AF14FE4-0A5F-381C-9FC5-E2520728FC65> /private/var/containers/Bundle/Application/F74E88F2-874F-4AF4-9D9A-0EFB51C9B1BD/Hand Tracking.app/Hand Tracking.debug.dylib Expected in: <2F158065-9DC8-33D2-A4BF-CF0C8A32131B> /usr/lib/swift/libswift_Concurrency.dylib It was working perfectly fine on Xcode 16 beta 3, which makes me think it's an Xcode 16 issue, but no idea how to fix this. I also installed Xcode 16.2 beta (the newest beta) but same error. Please help if anyone knows what is wrong!
1
0
257
4w
WebXR Immersive AR Support & DOM Overlays
As mentioned in https://forums.developer.apple.com/forums/thread/756736?answerId=810096022#810096022 Is there any update about the full support to WebXR AR Module, which should enable immersive-ar mode? Are the features such as DOM overlays and WebGPU bindings on the roadmap? Is it possible to capture stereoscopic video either internally or externally or via airplay for debugging purposes? Thanks
0
2
255
4w
Handling user-initiated re-centering in group immersive space?
Hi, currently tinkering with a little shareplay app for the Vision Pro that allows people to facetime and shareplay to play with random 3d models (as well as move them around, which should sync the model positions for everyone in relative space). When the users start their facetime call, then open the immersive space to see the 3d models, the models load in properly in context of the group immersive space's coordinate system, and moving the models reflects the new positions real-time for each participant. The main issue comes if/when users use the digital crown to re-center their view. It appears to re-center the model and view, which is expected. However, it also seems to re-position the model/root entity to match the user's origin. Not sure if this is intentional or not, but this essentially makes it so that it "de-syncs" the model (so me moving the model next to someone does not reflect it 1:1 - it still moves properly, but the new "initial" position after re-centering makes it offset). Is there a potential solution or work-around for this such that re-centering the view doesn't de-sync the model/entity's position? Rough code for my RealityView component is below: RealityView { content, attachments in content.add(appModel.originEntity) appModel.originEntity.addChild(appModel.modelContainerEntity) appModel.setInitialModelPosition() configureGestures(forModel: appModel.modelContainerEntity) configureToolbarAttachment(content: content, attachments: attachments) } update: { content, _ in // I have modified the Apple provided gesture components to // send the app model the new positions/rotations // as well as broadcast the position/rotation to shareplay participants // When user re-centers view, it seems to also re-position the model // so that its origin is at the local user's origin, rather than // the original origin // Can we receive a notification that user has re-centered view? // Or some other work-around? appModel.modelContainerEntity.setPosition(appModel.modelState.position, relativeTo: nil) appModel.modelContainerEntity.setOrientation(.init(appModel.modelState.rotation3d), relativeTo: nil) } attachments: { Attachment(id: "customViewAttachment") { CustomView() } } .installGestures() Please let me know if anything wasn't clear or if more information is needed. Thanks!
4
0
265
Oct ’24