When I've made an animated UDSZ, at what framerate will the animation be rendered in QuickLook? Is it the same across all devices? (iPhone, Apple Vision Pro, etc.) and viewing environments? (QuickLook, inside an ARView, etc.) Suppose I export my file at 30fps and the device draws at 60fps, does the device interpolate between frames automatically, animate at a lower frame rate, or play it at twice the speed? What if it were 24fps? My primary concern with understanding frame rates is a bit of trouble I've had making perfectly looping animations. There always seems to be the slightest stutter between iterations. Thanks in advance for any insights you're able to provide!

Hey I wanted to make an app that tracks changes in the room and room lightning and I was wondering if its possible to use VirtualEnvironmentProbeComponent to obtain the EnvironmentResource image and store it? If so are there any example of similar operation I could use? Thank you!

I have been referencing the Object Tracking Tutorial from WWDC 2024 on Vision OS, how Create ML is used to create a reference object, and we can track them in the ARSession. I am looking forward to building this feature on an AR app for iPhone, I am using iPhone 13 Pro Max. I have created couple of reference objects from the Create ML.

Hello, I am developing a visionOS application and am interested in obtaining detailed data of users’ hands through ARKit, including but not limited to Transform and rotation angle. I have reviewed Happy Beem, but it appears to only introduce the method of identifying the user’s specific gestures. Could you please advise on how to obtain the Transform and rotation angle of the user’s hand? Thank you.

We are using the ARKit image tracking feature on visionOS 2.0 with three pre-registered images. The image tracking works, but only one image is actively tracked at a time. When more than one target image is visible to the camera, it has difficulty detecting and tracking the other images. Is this the expected behavior in visionOS, or is there something we need to do to resolve this issue?

I am using ARKit to detect image in visionPro. However I met some question about adding the reference image. Some of my images can not be added correctly sometimes. (As you can see in the picture above, the 'orange' can not be added correctly, but the 'cup' can). However, sometimes they will be added without any problem. I do not know why it will happen. And I want they all be added steadily.

Hello. I am currently building an app using AR Kit. As for the UI, I am using SwiftUI and NavigationStack + NavigationLink for navigation and screen transitions! Here I need to go back and forth between the AR screen and other screens. If the number of screen transitions is small, this is not a problem. However, if the number of screen transitions increases to 10 or 20, it crashes somewhere. We are struggling with this problem. (The nature of the application requires multiple screen transitions.) The crash log showed the following. error: read memory from 0x1e387f2d4 failed AR_Crash_Sample-2025-03-07-115914.txt Incident Identifier: B23D806E-D578-4A95-8828-2A1E8D6BB7F8 Beta Identifier: 924A85AB-441C-41A7-9BC2-063940BDAF32 Hardware Model: iPhone16,1 Process: AR_Crash_Sample [2375] Path: /private/var/containers/Bundle/Application/FAC3D662-DB10-434E-A006-79B9515D8B7A/AR_Crash_Sample.app/AR_Crash_Sample Identifier: ar.crash.sample.AR.Crash.Sample Version: 1.0 (1) AppStoreTools: 16C7015 AppVariant: 1:iPhone16,1:18 Beta: YES Code Type: ARM-64 (Native) Role: Foreground Parent Process: launchd [1] Coalition: ar.crash.sample.AR.Crash.Sample [1464] Date/Time: 2025-03-07 11:59:14.3691 +0900 Launch Time: 2025-03-07 11:57:47.3955 +0900 OS Version: iPhone OS 18.3.1 (22D72) Release Type: User Baseband Version: 2.40.05 Report Version: 104 Exception Type: EXC_CRASH (SIGABRT) Exception Codes: 0x0000000000000000, 0x0000000000000000 Termination Reason: SIGNAL 6 Abort trap: 6 Terminating Process: AR_Crash_Sample [2375] Triggered by Thread: 7 Application Specific Information: abort() called Thread 7 name: Dispatch queue: com.apple.arkit.depthtechnique Thread 7 Crashed: 0 libsystem_kernel.dylib 0x1e387f2d4 __pthread_kill + 8 1 libsystem_pthread.dylib 0x21cedd59c pthread_kill + 268 2 libsystem_c.dylib 0x199f98b08 abort + 128 3 libc++abi.dylib 0x21ce035b8 abort_message + 132 4 libc++abi.dylib 0x21cdf1b90 demangling_terminate_handler() + 320 5 libobjc.A.dylib 0x18f6c72d4 _objc_terminate() + 172 6 libc++abi.dylib 0x21ce0287c std::__terminate(void (*)()) + 16 7 libc++abi.dylib 0x21ce02820 std::terminate() + 108 8 libdispatch.dylib 0x199edefbc _dispatch_client_callout + 40 9 libdispatch.dylib 0x199ee65cc _dispatch_lane_serial_drain + 768 10 libdispatch.dylib 0x199ee7158 _dispatch_lane_invoke + 432 11 libdispatch.dylib 0x199ee85c0 _dispatch_workloop_invoke + 1744 12 libdispatch.dylib 0x199ef238c _dispatch_root_queue_drain_deferred_wlh + 288 13 libdispatch.dylib 0x199ef1bd8 _dispatch_workloop_worker_thread + 540 14 libsystem_pthread.dylib 0x21ced8680 _pthread_wqthread + 288 15 libsystem_pthread.dylib 0x21ced6474 start_wqthread + 8 Perhaps I am using too much memory! How can I address this phenomenon? For the AR functionality, we are using UIViewRepresentable, which is written in UIKit and can be called from SwiftUI import ARKit import AsyncAlgorithms import AVFoundation import SCNLine import SwiftUI internal struct MeasureARViewContainer: UIViewRepresentable { @Binding var tapCount: Int @Binding var distance: Double? @Binding var currentIndex: Int var focusSquare: FocusSquare = FocusSquare() let coachingOverlay: ARCoachingOverlayView = ARCoachingOverlayView() func makeUIView(context: Context) -> ARSCNView { let arView: ARSCNView = ARSCNView() arView.delegate = context.coordinator let configuration: ARWorldTrackingConfiguration = ARWorldTrackingConfiguration() configuration.planeDetection = [.horizontal, .vertical] if ARWorldTrackingConfiguration.supportsFrameSemantics(.sceneDepth) { configuration.frameSemantics = [.sceneDepth, .smoothedSceneDepth] } arView.session.run(configuration, options: [.resetTracking, .removeExistingAnchors]) context.coordinator.sceneView = arView context.coordinator.scanTarget() coachingOverlay.session = arView.session coachingOverlay.delegate = context.coordinator coachingOverlay.goal = .horizontalPlane coachingOverlay.activatesAutomatically = true coachingOverlay.autoresizingMask = [.flexibleWidth, .flexibleHeight] coachingOverlay.translatesAutoresizingMaskIntoConstraints = false arView.addSubview(coachingOverlay) return arView } func updateUIView(_ _: ARSCNView, context: Context) { context.coordinator.mode = MeasurementMode(rawValue: currentIndex) ?? .width if tapCount == 0 { context.coordinator.resetMeasurement() return } if distance != nil { return } DispatchQueue.main.async { if context.coordinator.distance == nil { context.coordinator.handleTap() } } } static func dismantleUIView(_ uiView: ARSCNView, coordinator: Coordinator) { uiView.session.pause() coordinator.stopScanTarget() coordinator.stopSpeech() DispatchQueue.main.async { uiView.removeFromSuperview() } } func makeCoordinator() -> Coordinator { Coordinator(self) } class Coordinator: NSObject, ARSCNViewDelegate, ARSessionDelegate, ARCoachingOverlayViewDelegate { var parent: MeasureARViewContainer var sceneView: ARSCNView? var startPosition: SCNVector3? var pointedCount: Int = 0 var distance: Float? var mode: MeasurementMode = .width let synthesizer: AVSpeechSynthesizer = AVSpeechSynthesizer() var scanTargetTask: Task<Void, Never>? var currentResult: ARRaycastResult? init(_ parent: MeasureARViewContainer) { self.parent = parent } // ... etc } }

I would like to know if there have been any updates that improved the image tracking speed. I submitted feedback a while ago, mentioning that the image tracking rate was only one frame per second on Vision Pro. Thank you very much. FB15688516 and FB16745373is feedback we previously submitted, but we have not received any response yet.

As I understand it there are two ways I can track a hand, or a joint, in RealityKit: either, create an AnchorEntity, for example AnchorEntity(.hand(.left, location: .palm)) or, set up an ARSession with a HandTrackingProvider ( a lot more code which I haven't repeated here). Assuming this is correct, when would I want to use one over the other?

My app sends users notifications of important messages in the app. How can I make sure the user receives them?

We're developing a VisionOS application, where we would like to do product recognition (like food items). We have enterprise entitlements and therefore also main camera access for VisionOS. We send this live camera frames to a trained CoreML model where we will receive 2D coordinates from the model detection prediction. Now, we would like to create a 3D anchor on the detected items so it can be visible for user. The 3D anchor is going to be the class name of the detected item. How do we transform this 2D coordinate from the model prediction to a 3D anchor?

I am working with MeshAnchors, and I am having troubles getting to the classification of the triangles/faces. This post references the MeshAnchor.Geometry, and that struct does have a property named "classifications", but it is of type GeometrySource. I cannot find any classification information in GeometrySource. Am I missing something there? I think I am looking for something of type MeshAnchor.MeshClassification, but I cannot find any structs with this as a property.

I am developing an visionos app. I load a .usdz file as a Reality Entity(such as a cabbage). And I want such an effect: When I turn on a desk lamp in real world near the Entity, the surface of the Entity will correctly respond to the light in the real world. I want an effect like this: https://www.reddit.com/r/virtualreality/comments/1as01mm/shiny_disco_ball_reflecting_my_room/ I look up the api such as ImageBasedLightComponent andVirtualEnvironmentProbeComponent in RealityKit、EnvironmentLightEstimationProvider in ARKit，but I do not know how to code. Besides, it will be better if the shadow will also respond to the light correctly.

I am developing an ARKit based application that requires plane detection of the tabletop at which the user is seated. Early testing was with an iPhone 8 and iPhone 8+. With those devices, ARKit rapidly detected the plane of the tabletop when it was only 8 to 10 inches away. Using iPhone 15 with the same code, it seems to require me to move the phone more like 15 to 16 inches away before detecting the plane of the table. This is an awkward motion for a user seated at a table. To validate that it was not necessarily a feature of my code, I determined that the same behavior results with Apple's sample AR Interaction application. Has anyone else experienced this, and if so, have suggestions to improve the situation?

Hi, is there any way to use front camera to do the motion capture? I want recognize if the user raised there hands up with the front camera on iPhone. I was able to do it with the back camera, not the front. Also, if there is any sample code, or document, I would be super happy. Waiting for your reply!!

Hello Everyone Sam Francisco here otherwise known as Ian Francis Creative Director at Primus Films Ltd in the UK. I'm excited today to have Apple invite me to the Developer Forums and indeed honoured to have the chance of creating something that is of interest in the App Store and beyond. My question is self explanatory really, I am in the Final Cut of 'Metaphor' which is a 42 minutes 17' second Horror Film about Climate Change and I have a deadline set for it's Premiere on YouTube which is this year on April 22nd which as I'm sure you all know is Earth Day. So although I am quite comfortable and relatively proficient as the film's Writer, Director and Editor I have to confess to being green as tomatoes when it comes to marketing and publicity, and negotiating rights and agreeing deals for example to gain access to the App Store here at Apple with a view to discussing rental or sale options. Even if the film comes away empty handed from the Dorothy Chandler it'll be all set for the Baftas over in the UK and I was wondering how to get discussions going on this and if anyone out there has any knowledge of where I even start I would be very grateful. Thank you for your time Ian Francis

Hi, we would like to create something where you can open multiple volumetric windows and place them in a room, our biggest issue is that we want these windows to be persistent, so when I close and reopen the app, the windows to be in the same position. We can't use immersive spaces because we also want to have the possibility to access the shared space. Is it possible with the current features and capabilities to do that? If yes do you have some advices how can we achieve this? The alternative is if is it possible to open the virtual display in immersive spaces or if we have the possibility to implement our own virtual display.

I am a newby of spatial computing and I am using ARKit and RealityKit to develop a visionPro app. I want to accomplish such a goal: If the user's hand touchs an object(an entity in RealityView) on the table, it will post a Window. But I do not know how to handle the event "the user's hand touchs the object". Should I use hand tracking feature to do some computing by myself? Or is there some api to use directly? Thank you!

I am a newby of spatial computing. Here I am learning how to use ARKit to capture the environment texture and apply it on a ModelEntity of RealityKit on Vision Pro. But I do not find a demo of how to use EnvironmentLightEstimationProvider. After checking the documentation, I also have some questions: EnvironmentProbeAnchor.environmentTexture is a MTLTexture, but EnvironmentResource needs a CGImage. How do I translate MTLTexture to CGImage(Forgive me that I do not know much about Metal or other framework, so It will be better if there is a code that I can copy and paste directly) It seems that the EnvironmentProbeAnchor can only get the light information around the device. But what should I do if I want get the light information around the ModelEntity so that I can apply the environment texture on it. It will be better if you can provide a code demo about how to use the new api. Thank you!

When you click the button in the background of three horizontal lines, when the view is about to appear, add buried event statistics, but click the button to close it, it will repeat the view will appear method API, equivalent to the view method repeated execution twice, resulting in incorrect buried event statistics