Posts under Spatial Computing topic

Post

Replies

Boosts

Views

Activity

Main Camera development request blocked by Individual-to-Organization conversion
We are developing MadnessBot as a business-only Vision Pro automotive inspection app for professional mechanics and repair shops, intended for private business distribution. Main Camera access is essential for analyzing the vehicle and component in front of the technician. A native development app is running on our physical headset, and chat plus OBDLink CX diagnostics have been tested. Our existing Apple Developer Program membership is enrolled as Individual. We contacted Developer Support on October 6, 2026, with a follow-up on October 9, for assistance converting it to Organization. The conversion form asks for the type of tax ID historically associated with the Individual membership; the Account Holder does not know that record and has asked Support to verify it before selecting an answer. The enterprise API request form remains inaccessible until organization enrollment is confirmed. We have not submitted a camera entitlement request yet. Our planned initial request is Development Only Main Camera access for registered test devices. Feedback reference: FB25132939. Could the Enterprise APIs for visionOS team help route this prerequisite blocker to membership support, or advise the correct escalation path so we can complete conversion and apply for Development Only access?
0
0
615
3d
visionOS : any non-nil preferredSurroundingsEffect enables a light-spill halo around VideoPlayerComponent video
Hi all — sharing a platform behavior I isolated while building a custom immersive video player, in case others hit it (and to ask whether it's intended). Finding On visionOS 27 (Xcode 27), an entity playing video via VideoPlayerComponent(videoRenderer:) gets a light-spill "screen glow" halo around its edges whenever the immersive space declares any non-nil preferredSurroundingsEffect. The halo follows the video's colors frame by frame. With no surroundings effect (nil), there is no halo. Quick repro Progressive immersive space + VideoPlayerComponent playing. Add .preferredSurroundingsEffect(.dim(intensity: 0.0)) → halo appears (even though the effect is visually a no-op). Remove it → halo disappears. Same with .colorMultiply(_:) and preset effects. Things I ruled out on device (Vision Pro) ImmersiveContentBrightness (.custom(1.0) vs .automatic): no difference. VideoPlayerComponent configuration (.stereo, spatial video mode, isPassthroughTintingEnabled) and component attach timing: no difference. Presenting the same video via VideoMaterial: no halo. Bright white SwiftUI content: no halo. Questions Is this coupling intended? The preferredSurroundingsEffect docs don't mention any effect on video playback. Is there a public way to keep a surroundings effect (e.g. room dimming) while avoiding the video light spill? I found none on VideoPlayerComponent. Current workaround: don't declare any surroundings effect in the modes where the halo is undesirable. Also filed via Feedback Assistant(FB25130748). Curious if anyone else has seen this or knows a switch.
0
1
868
4d
RoomPlan: placed RealityKit content does not align with the scanned room
I scan a room with RoomPlan API, then keep the same ARSession running (stop(pauseARSession: false)) and place RealityKit entities using the CapturedRoom wall transforms (wall.transform, wall.dimensions), parented to an anchor at the world origin. The camera position stays continuous, so tracking was not reset. But the entities do not sit on the real walls. The whole room appears shifted. What is the correct way to place content in the live AR session so it aligns with the scanned room?
1
0
1.6k
5d
RealityKit VideoMaterial renders 8-bit HEVC HLG/BT.2020 360° video with a purple tint in the visionOS Simulator (device is fine)
We paint a looping 360° video on the inside of a sphere with VideoMaterial(avPlayer:): an AVQueuePlayer plus AVPlayerLooper, a ModelEntity with an inward-facing equirectangular mesh, inside a .mixed ImmersiveSpace. No shaders, no VideoPlayerComponent. We could not find an official Apple sample that puts a VideoMaterial on a 360° sphere; this is the plain pattern from the VideoMaterial(avPlayer:) reference and the one used in threads like 734081. On Apple Vision Pro hardware the video renders correctly. In the visionOS Simulator (27.0, 26.5 and earlier 26.x, Xcode 27.0) the whole frame has a strong purple/magenta cast. Audio and timing are fine. The video (which we cannot share, so here is its metadata, image sample of issue on simulator provided below): MP4, HEVC Main, 8-bit 4:2:0, 3840x1920, tagged BT.2020 primaries, HLG transfer (ARIB STD-B67), BT.2020 non-constant matrix, both in the colr atom and in the SPS VUI. ffprobe -show_format -show_streams (trimmed): [STREAM] index=0 codec_name=hevc profile=Main codec_type=video codec_tag_string=hvc1 width=3840 height=1920 pix_fmt=yuv420p color_range=tv color_space=bt2020nc color_transfer=arib-std-b67 color_primaries=bt2020 r_frame_rate=30000/1001 avg_frame_rate=679200/22663 duration=37.771667 bit_rate=25030607 [SIDE_DATA] side_data_type=Stereo 3D type=2D horizontal_field_of_view=360000/1000 [/SIDE_DATA] [/STREAM] [STREAM] index=1 codec_name=aac profile=LC codec_type=audio codec_tag_string=mp4a sample_fmt=fltp sample_rate=48000 channels=2 channel_layout=stereo duration=37.758333 bit_rate=128176 [/STREAM] [FORMAT] nb_streams=2 format_name=mov,mp4,m4a,3gp,3g2,mj2 duration=37.778333 bit_rate=25160348 TAG:major_brand=mp42 TAG:minor_version=1 TAG:compatible_brands=isommp41mp42 [/FORMAT] AVFoundation's view of the track, from CMFormatDescriptionGetExtension on the loaded video track: primaries=ITU_R_2020 transfer=ITU_R_2100_HLG matrix=ITU_R_2020 bitsPerComponent=8 The playback code, reduced to the essentials: let item = AVPlayerItem(url: fileURL) let looper = AVPlayerLooper(player: player, templateItem: item) // retained player.actionAtItemEnd = .advance let material = VideoMaterial(avPlayer: player) let sphere = ModelEntity( mesh: try equirectSphere(radius: 12), // lat/long grid, normals inward, materials: [material] // UVs (u, 1 - v), inward winding ) sphere.position = [0, 1.5, 0] rootEntity.addChild(sphere) // rootEntity was added in RealityView { } player.play() What we have ruled out in a minimal sample project: Apple's own 360° HLS example (360 Lighthouse, HEVC Main 10, 10-bit, BT.709 SDR) plays with correct colours in the same sphere, same code, same simulator session. Rewriting our file's colr atom to BT.709 (three bytes, no re-encode) is honoured by AVFoundation (the track's CMFormatDescription then reports ITU_R_709_2 for primaries, transfer and matrix) but the tint stays. So the remaining differences between the failing file and Apple's working stream are 8-bit Main vs 10-bit Main 10, HLG/BT.2020 in the bitstream VUI vs BT.709, and local MP4 vs HLS. The minimal sample project (without the video) is available on request. Is this a known limitation of the simulator's VideoMaterial path with 8-bit HLG content, and is there a recommended way to get device-equivalent colour in the simulator? Feedback filed as FB24997147. Here is a sample of the result on the simulator of a cement column, painted white, but rendered purple on simulator:
1
0
3.1k
1w
Mac spatial preview feature: Supported with Blender?
Hello! I am relatively new to dev for Apple Vision Pro, however I have owned one since launch and love it. When I heard that we were going to be able to preview real-time, 3D models from the Mac I couldn’t believe it! This is one of my major workflows and I have been using 3rd party workarounds (that don’t work well). My question is, is there any word on supported 3D modeling apps like Blender or Cinema 4D? Or will there be a way to implement this preview feature with any app via a protocol? Any documentation would be really helpfu! Thank you, and continue the great additions to Apple Vision Pro!
2
0
2.8k
1w
Revealed keyboard drifts in progressive immersion, and no immersion style gives windows-in-front + Crown + passthrough cutouts
Environment: Apple Vision Pro, visionOS 27.0.1 (24M372). Magic Keyboard, with Settings › Awareness & Safety › Reveal Keyboards set to Always. Our app puts the user in an opaque virtual room while they work at their real desk, using app windows and Mac Virtual Display. A virtual desk is placed on the real desk. It is world-locked: positioned once from a PlaneAnchor (PlaneDetectionProvider), then a plain entity at fixed ImmersiveSpace coordinates. No AnchorEntity or WorldAnchor is involved. We offer two modes: .mixed, and .progressive for Digital Crown control. Each gives us only half of what desk work needs. 1. The revealed keyboard drifts in progressive immersion (FB25006789) In .progressive, the physical keyboard shown by keyboard awareness does not stay registered with the virtual desk. After the keyboard has been out of view for a few seconds (for example, while looking at a Mac Virtual Display window), it looks "floaty" when you look back: as the head turns, it slides against the desk. Turning right, the keyboard moves left farther than the desk does. What we checked: It happens in .progressive at any Crown amount, and in .full. With identical content in .mixed (no app passthrough cutout, so again only keyboard awareness shows the keyboard), it is much steadier. On 27.0.1, PlaneAnchor and DeviceAnchor coordinateSpace(correction: .rendered) are exactly equal to their .none transforms: 0 mm, 0°, scale 1. That held over 835 per-frame samples with ±50° yaw and up to 36 cm of head movement. So the rendered correction gives the app nothing to compensate with. The app's desk transform is constant while the drift is visible. upperLimbVisibility(.automatic) vs .visible and a .portrait progressive aspect ratio make no difference. Possibly related: https://developer.apple.com/forums/thread/796557 (FB19666209). 2. No immersion style combines what desk work needs (FB25006837) We need, at the same time: windows (our own and Mac Virtual Display) never covered by the room's walls, furniture or trees; the Digital Crown to bring the real surroundings back in; a passthrough cutout over the real desk, and a well-registered keyboard. .mixed: ✓ OcclusionMaterial cutouts reveal passthrough. ✓ The keyboard stays registered. ✗ RealityKit entities can render in front of windows, including Mac Virtual Display and window chrome. breakthroughEffect doesn't cover whole windows or Mac Virtual Display. ✗ The Crown can't adjust immersion. .progressive: ✓ Windows always stay in front of immersive content (as an Apple engineer confirmed in https://developer.apple.com/forums/thread/764556). ✓ The Crown adjusts immersion. ✗ OcclusionMaterial reveals nothing inside the portal, so there is no desk cutout. ✗ The keyboard drifts (issue 1). Because of issue 1 we have switched our default back to .mixed, and lost windows-in-front. Questions Is keyboard awareness in .progressive/.full expected to be registered less accurately than in .mixed? Is there a supported way to keep it registered with world-locked content, or to get the pose at which the system draws it? In .mixed, is there a supported way to keep windows (including Mac Virtual Display) in front of an app's immersive content? Or, in .progressive, a supported way to show passthrough in an app-defined region inside the portal? Any pointers or workarounds are appreciated.
0
0
1.5k
1w
How to use ClothGrabComponent?
I've simulated a piece of cloth using https://developer.apple.com/documentation/realitykit/physics-cloth-simulation I would like to grab the cloth like what I can in Marvellous Designer. I found the ClothGrabComponent at https://developer.apple.com/documentation/realitykit/clothgrabcomponent However, after I added the ClothGrabComponent to the cloth entity, I am still not able to grab it. Is there any sample code to show me how to grab the cloth? Thank you.
2
0
1.4k
2w
Is there currently a way to import 3D Gaussian Splatting data into Reality Composer Pro?
Hello, As far as I understand, RealityKit now supports 3D Gaussian Splatting through GaussianSplatComponent. I would like to ask whether there is currently a way to use 3D Gaussian Splatting data directly in Reality Composer Pro, for example by importing a .ply file containing Gaussian splats. More specifically: Can Reality Composer Pro import a 3D Gaussian Splatting .ply file? Is there a way to create or assign a GaussianSplatComponent to an entity directly within Reality Composer Pro? If this is not currently supported, is there any recommended workflow for creating a .usdz containing Gaussian Splatting data that can be opened in the Preview app on iPhone/iPad and displayed as an AR object? I tried opening the sample plant.usdz on an iPhone using the Preview app. However, the AR preview appears to be empty, so it seems that the Preview app may not currently support rendering the Gaussian Splatting content contained in that USDZ. My desired workflow would be: Gaussian Splatting .ply → Reality Composer Pro / USDZ → iPhone Preview → AR Is this workflow currently possible? If not, is there an official or recommended way to package a GaussianSplatComponent and its associated Gaussian splat data into a USD/USDZ file for AR preview on iOS? If the above functionality is not currently supported, does the development team have any plans to add or improve support for these features in future versions of Reality Composer Pro or the Preview app? Any clarification regarding the current capabilities, limitations, or future plans would be greatly appreciated. Thank you!
4
1
2.7k
2w
RoomPlan crashes spiking on iOS 27
We're seeing a pretty significant spike in RoomPlan crashes on iOS 27, specifically a RoomScanCore EXC_BREAKPOINT. This crash isn't new, we've seen a low background rate of it across earlier iOS versions, but since iOS 27 shipped, the rate has jumped substantially, well beyond what iOS 27's adoption share alone would explain. We haven't reproduced the crash internally. As with the existing crashes, it's seemingly environment/scenario-dependent in ways that don't always show up in our QA passes. Is anyone else seeing this? Are there known issues that will be patched next month with iOS 27.2?
1
0
1.2k
2w
RealityKit cloth: reducing mesh collider air gap without penetration
I’m testing RealityKit cloth for garment-like cloth over an authored human-shaped triangle-mesh collider. The issue also occurs with a static collider, so it does not appear to depend on character animation.. Contact is stable and runs interactively, but I cannot achieve close surface contact without sacrificing robustness. The cloth mid-surface remains visibly several millimetres from the authored collider, especially around curved or narrow areas such as the head, arms, and hands. Transparent rendering of visualMesh confirms this is not just render thickness. Reducing ClothMeshShape.bias reduces the gap, but eventually causes interpenetration around difficult geometry. Increasing it prevents penetration but makes the cloth visibly float. I’ve also checked scale, winding, normals, reasonably even triangles, timestep, and solver iterations. I want to avoid inflating the collider or adding anatomical proxy shapes. Is bias the only physical-clearance control, or does RealityKit apply an additional contact radius derived from simulation-mesh resolution? Is local conforming refinement the recommended way to achieve roughly 1–2 mm clearance while retaining robust real-time contact? Are there any specific topology requirements beyond evenly sized triangles? I can provide a minimal reproduction and screenshots if helpful although not sure I can share screenshots here?
4
0
5.1k
2w
visionOS 27.0 simulator restarts (backboardd EXC_BREAKPOINT in DRValidateIndices) on MeshResource.generate(from:) for a valid multi-part mesh
Since moving to Xcode 27.0 (27A266a) with the visionOS 27.0 simulator runtime (24M362), generating a mesh with several parts through MeshResource.generate(from: MeshResource.Contents) makes the simulator go black for a couple of seconds and restart. The app does not crash; the simulator's render server does: Thread ... Crashed :: com.apple.CoreRE.ResourceFetchManager-service-queue 0 DirectResource specialized static IndexValidation._validate_neon(_:vertexCount:) 1 DirectResource DRValidateIndices 2 CoreRE re::(anonymous namespace)::validatePrivateData(id<REMeshPayload>, unsigned char const*, unsigned long) 3 CoreRE -[REMultiBufferMeshPayload initWithCoder:] 4 Foundation _decodeObject Environment: MacBook Pro (M1 Pro, 16 GB), macOS 27.0 (26A428), Xcode 27.0 (27A266a), visionOS 27.0 SDK (24M361); failing runtime visionOS 27.0 simulator (24M362); working: the visionOS 26.5 simulator runtime (23O470) under the same Xcode, and an Apple Vision Pro on visionOS 27. What I have established: It reproduces with a fully synthetic mesh: one model, three parts (1330 / 20 / 24 vertices, 9513 / 72 / 120 indices), random positions, normals, UVs and triangle indices. Only the part layout matters. The mesh is valid: every part's indices are within its own vertex count, counts are multiples of three, attribute buffers match. RealityKit's app-side validator agrees and lets it through (it does reject a broken part with REMeshPartDescriptor: index buffer payload range is invalid), then the render server's validator traps on the accepted payload. Single-part meshes of any size, primitives, and compiled .reality scenes are fine. Same mesh, same Xcode: fine on a simulator device created on the 26.5 runtime, fine on a visionOS 27 device. Only the 27.0 simulator runtime traps. FB number: FB24831983. Workarounds so far: a simulator device on the 26.5 runtime, or generating one MeshResource per part. Has anyone seen DRValidateIndices trapping on the 27.0 simulator?
1
0
2.6k
3w
Logitech Muse never appears under Settings > General > Spatial Accessory — spatial tracking permission never granted
I'm building a visionOS 26 app that uses SpatialTrackingSession with .accessory tracking and AnchoringComponent.AccessoryAnchoringSource to track a Logitech Muse, following the pattern from the WWDC25 "Explore spatial accessory input on visionOS" session (and matching two independently published implementations I found). What works: The Muse pairs fine over Bluetooth, connects as a GCStylus (productCategory == .spatialStylus), GCStylusDidConnect fires, and button input (pressedDidChangeHandler) works correctly. What doesn't: SpatialTrackingSession.run() always reports .accessory as unavailable, so AnchoringComponent.AccessoryAnchoringSource(device:) never succeeds. Checking Settings > General > Spatial Accessory on the device shows the Muse isn't listed there at all — not even a battery/status entry. Settings > Privacy & Security > Spatial Accessory also never lists our app. What I've tried: Forgetting the Muse in Bluetooth and re-pairing Full power-cycle of both Bluetooth and the Muse itself Confirmed NSAccessoryTrackingUsageDescription is set and the Accessory Tracking + Game Controllers (Spatial Gamepad) capabilities are enabled in Xcode Confirmed via Apple's own docs, the WWDC session, and two developer write-ups that no special commissioning step beyond standard Bluetooth pairing should be needed For comparison, XR Putt (which also uses the Muse) works fine and does show up with a green checkmark under Spatial Accessory permissions on the same headset. Has anyone hit this specific "device never gets commissioned as a spatial accessory" issue, or knows what actually triggers that commissioning beyond generic Bluetooth pairing? Happy to share more logs/details.
1
0
1.6k
3w
ARKit World Tracking Drift Regression on LiDAR-Equipped Devices - iOS 26.4+
Summary We have identified a reproducible world tracking drift regression in ARKit on LiDAR-equipped iOS devices running iOS 26.4 and later. A static virtual node anchored at the world origin visually drifts from its initial position as the user moves around a real-world scene, despite the scene remaining physically static. The same code produces stable, drift-free results on non-LiDAR devices running identical OS versions. Device & OS Observations Testing was performed across four devices on iOS 26.4 using the same application build and ARWorldTrackingConfiguration settings. Non-LiDAR devices — iPhone 14 and iPhone 15 — produced stable, drift-free tracking in all test runs. No world origin displacement was observed regardless of how long or how far the user walked. LiDAR-equipped devices — iPhone 14 Pro and iPhone 16 Pro — exhibited consistent, reproducible drift. A static node placed at the world origin visually shifted from its initial position as the user moved through the scene. The same devices were stable on earlier iOS versions, confirming this is a regression introduced in iOS 26.4. Technical Observations Nature of drift: A SCNNode placed statically at the ARKit world origin (SCNVector3(0, 0, 0)) visually displaces from its original position as the user walks around a static real-world scene. The displacement is not random — it accumulates directionally as the user moves, consistent with a sensor fusion or coordinate anchoring error. Trigger condition: The drift occurs during normal walking motion around a fixed point of interest, such as circling a parked vehicle. It does not appear when the device is held still. LiDAR specificity: The drift is exclusive to devices with a LiDAR scanner. Identical hardware configurations — same iOS build, same ARWorldTrackingConfiguration settings — on non-LiDAR devices produce no drift whatsoever. This isolates the regression to the LiDAR sensor's contribution to ARKit's internal Visual-Inertial Odometry (VIO) fusion pipeline. No API-level workaround found: There is currently no public ARKit API to selectively disable the LiDAR scanner's contribution to VIO. All available configuration-level options have been evaluated without resolving the drift. ARWorldTrackingConfiguration Options Evaluated The following configuration changes were applied individually and in combination. None resolved the drift on LiDAR devices: isAutoFocusEnabled = false — No improvement videoHDRAllowed = false (disabled) — No improvement planeDetection = [] (disabled) — No improvement sceneReconstruction = .mesh — No improvement worldAlignment: .gravity vs .gravityAndHeading — No improvement Minimal Reproduction Case The drift can be reproduced with a minimal ARKit scene: Create an ARSCNView with ARWorldTrackingConfiguration using default settings. Add a single static SCNNode (e.g., a small sphere or axes geometry) at SCNVector3(0, 0, 0) when the session starts. Run the app on a LiDAR-equipped device (iPhone Pro, iPad Pro with LiDAR) on iOS 26.4 or later. Walk in a circle around the node's approximate real-world position. Expected: The node remains visually fixed at its world position throughout the walkthrough. Actual: The node drifts from its initial position, increasingly displaced from its world origin anchor as walking continues. We want to know if Apple has made any internal updates to ARKit, particularly after the OS 26.4 upgrade. Thanks!
7
8
4.3k
3w
Production LiDAR capture app affected by ARKit world-tracking drift on iOS 26.4+ (re: thread 827240) mitigation guidance and fix timeline?
Hello, and thank you for taking questions this week. We develop DeepWalk, a production App Store application that captures LiDAR depth, RGB, and ARKit camera poses while field technicians walk continuous outdoor routes (sidewalk and infrastructure assessment). Recorded ARCamera.transform values are used downstream for 3D reconstruction, so world-tracking accuracy directly determines our output quality. Problem: Beginning in early April 2026 and coinciding with the iOS 26.4 rollout across our device fleet, we observe directional, accumulating pose drift in recorded trajectories during normal walking motion. The behavior matches the regression reported in forum thread 827240 (world-tracking drift on LiDAR devices on iOS 26.4+, absent on non-LiDAR devices running identical code). Link to existing post here: https://developer.apple.com/forums/thread/827240 Our configuration is minimal: a default ARWorldTrackingConfiguration with frameSemantics = [.sceneDepth, .smoothedSceneDepth] and no other options set, running on a bare ARSession with a Metal renderer (no SceneKit/RealityKit). We measure drift directly from per-frame ARCamera.transform values, so the displacement appears in ARKit's pose output itself rather than in any rendering layer. The drift accumulates with walking distance and does not occur when the device is stationary. We've observed this across a large variety of devices (iPhone 15/16/17 Pro/Pro Max) on all iOS versions 26.4 or greater. We've also observed this on iPad Pros with updated firmware. Thread 827240 additionally reports that disabling autofocus, HDR, and plane detection and varying scene reconstruction and world alignment did not resolve it. Question Because depth capture is our core function, we cannot avoid engaging the LiDAR scanner, and we're not aware of a public API that adjusts its contribution to world tracking. We'd be grateful for guidance on three questions: Is this a known issue in iOS 26.4 and later? Did the LiDAR contribution to world tracking change in that release? For apps that require sceneDepth, is there any recommended session configuration, video format, or capture-side practice that reduces the drift in the interim? Is a fix planned, and if so, should we expect it in an iOS 26.x update or the iOS 27 cycle? This affects how we manage our deployed device fleet. We're happy to provide trajectory datasets, additional sysdiagnoses, or anything else useful. We have quantified drift-per-distance data across OS versions.
7
1
1.6k
3w
WebXR Augmented Reality Module
On visionOS 26.5, I noticed the Safari Extension 'WebXR Augmented Reality Module' option. I've enabled this option in Safari Settings, and restarted Safari, but when I try to launch a WebXR experience that supports immersive-ar as option (for example this one: https://www.viverse.com/dPASDhP I don't see the option to Play in AR (like I do with a Quest 3S headset) Does anyone have any examples of websites that can make use of this new WebXR Augmented Reality Module feature flag in Safari, on visionOS?
3
4
1.7k
3w
Main Camera Error
An incomprehensible error occurs whenever I try to access Main Camera features. I have already acquired the Main Camera entitlement and granted the required permissions, yet the error persists. To make matters worse, even Apple’s provided sample code triggers the exact same issue. The error message is far too uninformative to diagnose the root cause, and I urgently need a resolution as this is completely blocking progress. Multiline [C:1-2] Error received: Invalidated by remote connection. failed to observe with mask <BLSXPCBacklightProxyObserverMask: 0x108accf80; didUpdateToState: YES; eventsArray: YES> error:<XPC error received on message reply handler (3:BSServiceConnectionErrorDomain) "The operation couldn’t be completed. XPC error received on message reply handler"> non-launching port is incompatible with service identifier "com.apple.PointerUI.pointeruid.default-service" nw_socket_copy_info [C1:2] getsockopt TCP_INFO failed [102: Operation not supported on socket] nw_socket_copy_info getsockopt TCP_INFO failed [102: Operation not supported on socket] Unable to find a compatible slice for binary archive at /System/Library/Frameworks/CoreImage.framework/ci_uberwrapper_bin.metallib *** Assertion failure in void ar_session_request_authorization(ar_session_t _Nonnull, ar_authorization_type_t, __strong ar_authorization_results_handler_t _Nonnull)_block_invoke(), session.m:1491, reason: This app failed to request an authorization. The NSWorldSensingUsageDescription / NSHandsTrackingUsageDescription / NSAccessoryTrackingUsageDescription / NSMainCameraUsageDescription keys may be missing from this app's Info.plist. <NSXPCConnection: 0x10e68dcc0> connection to service with pid 86 named com.apple.arkit.service.authorization: Exception caught during invocation of reply block to message 'requestAuthorizations:withReply:'. Exception: This app failed to request an authorization. The NSWorldSensingUsageDescription / NSHandsTrackingUsageDescription / NSAccessoryTrackingUsageDescription / NSMainCameraUsageDescription keys may be missing from this app's Info.plist. ( 0 CoreFoundation 0x0000000198b756d4 686CE4EB-CE52-3DD5-8B23-EF6A6BBFACEA + 1205972 1 libobjc.A.dylib 0x000000018d3a71cc objc_exception_throw + 88 2 CoreFoundation 0x0000000198b755c0 686CE4EB-CE52-3DD5-8B23-EF6A6BBFACEA + 1205696 3 ARKitCore 0x00000002bbd640cc 53FDDDD4-8DD1-3E86-ACB0-8867D29AF781 + 479436 4 CoreFoundation 0x0000000198b7c5b4 686CE4EB-CE52-3DD5-8B23-EF6A6BBFACEA + 1234356 5 CoreFoundation 0x0000000198b79558 686CE4EB-CE52-3DD5-8B23-EF6A6BBFACEA + 1221976 6 Foundation 0x000000019de4220c 61E78A52-3B44-3DCE-AF6D-92C33FA0FACA + 2109964 7 Foundation 0x000000019de41dd0 61E78A52-3B44-3DCE-AF6D-92C33FA0FACA + 2108880 8 Foundation 0x000000019de4651c 61E78A52-3B44-3DCE-AF6D-92C33FA0FACA + 2127132 9 libxpc.dylib 0x00000002c43d5a00 CE0BB604-B358-3F22-B1DA-FFACEFCA20AD + 150016 10 libxpc.dylib 0x00000002c43c5678 CE0BB604-B358-3F22-B1DA-FFACEFCA20AD + 83576 11 libdispatch.dylib 0x00000002c41045fc 4CF750D7-AC3B-3175-9863-2F059D3D3715 + 108028 12 libdispatch.dylib 0x00000002c41083ec 4CF750D7-AC3B-3175-9863-2F059D3D3715 + 123884 13 libdispatch.dylib 0x00000002c40f37c0 4CF750D7-AC3B-3175-9863-2F059D3D3715 + 38848 14 libdispatch.dylib 0x00000002c40f4450 4CF750D7-AC3B-3175-9863-2F059D3D3715 + 42064 15 libdispatch.dylib 0x00000002c40fe3ec 4CF750D7-AC3B-3175-9863-2F059D3D3715 + 82924 16 libdispatch.dylib 0x00000002c40fdd18 4CF750D7-AC3B-3175-9863-2F059D3D3715 + 81176 17 libsystem_pthread.dylib 0x00000002c4364e34 _pthread_wqthread + 292 18 libsystem_pthread.dylib 0x00000002c4362910 start_wqthread + 8 ) *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'This app failed to request an authorization. The NSWorldSensingUsageDescription / NSHandsTrackingUsageDescription / NSAccessoryTrackingUsageDescription / NSMainCameraUsageDescription keys may be missing from this app's Info.plist.' *** First throw call stack: (0x198b756c8 0x18d3a71cc 0x198b755c0 0x2bbd640cc 0x198b7c5b4 0x198b79558 0x19de4220c 0x19de41dd0 0x19de4651c 0x2c43d5a00 0x2c43c5678 0x2c41045fc 0x2c41083ec 0x2c40f37c0 0x2c40f4450 0x2c40fe3ec 0x2c40fdd18 0x2c4364e34 0x2c4362910) *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'This app failed to request an authorization. The NSWorldSensingUsageDescription / NSHandsTrackingUsageDescription / NSAccessoryTrackingUsageDescription / NSMainCameraUsageDescription keys may be missing from this app's Info.plist.' *** First throw call stack: (0x198b756c8 0x18d3a71cc 0x198b755c0 0x2bbd640cc 0x198b7c5b4 0x198b79558 0x19de4220c 0x19de41dd0 0x19de4651c 0x2c43d5a00 0x2c43c5678 0x2c41045fc 0x2c41083ec 0x2c40f37c0 0x2c40f4450 0x2c40fe3ec 0x2c40fdd18 0x2c4364e34 0x2c4362910) libc++abi: terminating due to uncaught exception of type NSException Xcode has killed the LLDB RPC server (pid: 2210) to allow the debugger to detach from your process. BlockQuote visionOS 27 beta 5
5
0
1.2k
3w
BloomComponent lags behind the camera on iOS/macOS but not visionOS
BloomComponent (RealityKit 27) lags behind the geometry that produces it while the camera moves. The halo trails the emissive geometry by roughly one to three frames (eyeballed) and snaps back once the camera stops. Video attached — it's obvious at normal playback speed. visionOS renders the identical scene correctly. Only iOS and macOS / Mac Catalyst lag, which points at the non-visionOS compositing path rather than the effect itself. Filed as FB23960052. This should be fixed before 27 ships — bloom is unusable for anything with a moving camera in its current state on those platforms. Ruled out: Scope. Identical with BloomComponent(scope: .hierarchical) and .unbounded. .unbounded computes no per-entity screen-space bounds, so stale bounds are not the cause. Input handling. The camera is RealityKit's own .realityViewCameraControls(.orbit) — no gesture code of mine involved. Per-frame component writes. BloomOptionsComponent is set once and untouched during the drag. Geometry and camera transform. Both track perfectly; only the glow lags. Reproducer is ~130 lines, no assets — five emissive spheres inside a large inward-facing dark sphere: var material = PhysicallyBasedMaterial() material.baseColor = .init(tint: .black) material.emissiveColor = .init(color: .cyan) material.emissiveIntensity = 4 // ... root.components.set(BloomComponent(scope: .unbounded)) var options = BloomOptionsComponent() options.strength = 1 options.threshold = 1 options.blurRadius = 1 root.components.set(options) shown in: RealityView { content in content.camera = .virtual content.add(root) } .realityViewCameraControls(.orbit) Xcode 27.0 beta 4, iOS 27.0 SDK, Apple Silicon. Affected: iOS 27, macOS 27 / Mac Catalyst 27. Not affected: visionOS 27 (immersive space). Full project and screen recording attached to my radar.
4
0
1.9k
3w
GaussianSplatComponent — stale visibility set after close-range camera approach
Title: GaussianSplatComponent — stale visibility set after close-range camera approach (iOS 27 beta 4, still unfixed) Filed this as a radar during beta 3; still reproduces unchanged on beta 4, so posting here in case anyone from the RealityKit team can confirm it's tracked before this ships. Radar: FB23749559 (filed against beta 3, re-verified on beta 4) Summary On iOS, GaussianSplatComponent keeps a stale visibility set after the camera moves close to and then away from the splat. Moving the camera in close causes a large portion of splats to be culled — expected. But pulling the camera back out does not repopulate them. The visible set stays stale for as long as the current camera transform is held. The missing splats only return once the camera transform changes again — any pan/orbit/zoom delta triggers a recompute and the full splat snaps back. Net effect: after any close inspection, the splat renders permanently partially-culled until the user happens to nudge the camera. The recovery-on-movement / no-recovery-when-stationary signature suggests the iOS render path recomputes splat residency on camera-transform change events rather than every frame. This also reproduces in AR mode, with a completely different camera driver. Since the only thing the two paths share is the splat residency code, that's where the stale set has to be coming from — which also means there's no app-side fix available to me. The camera transform in AR is driven by ARKit, not by my code, so I can't force the perturbation that would otherwise work around it. visionOS 27, running the same component, is unaffected — the visible set is re-evaluated per frame and the splat stays complete. Steps to reproduce Attach a GaussianSplatComponent (~400k splats) to an entity in a bounded RealityView with orbit controls. Move the camera close until a visible portion of the splats is culled. Move the camera back out so the full splat is within the frustum. Hold the camera still. Expected: full splat visible once pulled back. Actual: previously-culled splats stay missing while the camera is stationary; they reappear only on the next camera transform delta. Configuration iOS 27 beta 4 (originally filed against beta 3) iPhone 16 Pro ~400k splats Bounded RealityView + orbit controls; also reproduces in AR Screenshots attached: full splat, stale-culled state after pull-back, and the close-range view that triggers it. This is still present in beta 4 and in the public beta. I'd like to know whether it's tracked, or whether I should plan around splats being unreliable on iOS 27. Thanks!
5
0
1.5k
3w
visionOS 27: allAnchors and anchorUpdates disagree on WorldAnchor transforms (different coordinate frame)
After updating to visionOS 27.0, WorldTrackingProvider.allAnchors returns WorldAnchor.originFromAnchorTransform values that do not match what anchorUpdates delivers for the same anchor, read at the same moment, both isTracked == true, with no .updated event in between. The difference is a rigid frame change, not drift: Same vertical offset on every anchor: -1.173 m. Identical across two apps and two days with the head at 1.13 m and 1.22 m, so a constant, not head height. Horizontal deltas vary per anchor, but inter-anchor distances are preserved, so it is a yaw plus translation. Stable over time, same across launches. anchorUpdates matches the ImmersiveSpace frame RealityKit draws in. allAnchors does not. On visionOS 26 both agreed. Same anchor, same instant: allAnchors (0.1486, -0.7143, 0.2231) anchorUpdates (0.1316, 0.4590, -1.0955) device pose (0.0073, 1.2195, -0.0038) Four other anchors, delta = allAnchors - anchorUpdates: ( 2.6933, -1.1733, -0.2514) (-0.0810, -1.1733, 1.1339) ( 0.1878, -1.1733, 0.9891) ( 2.6289, -1.1733, -0.1809) In our app this showed up as a placed model vanishing after a scene switch: we re-pinned the model root from allAnchors, and it landed below the floor behind the user. A standalone sample (one provider, one anchor, a green sphere from anchorUpdates, a red sphere from allAnchors) shows two spheres on visionOS 27.0, the red one below the floor: added F21942C4-… at (-0.031, 0.712, -0.976) update added F21942C4-… tracked=true at (-0.031, 0.712, -0.976) MISMATCH delta (-0.211, -1.173, 1.444) m allAnchors (-0.243, -0.461, 0.468) anchorUpdates (-0.031, 0.712, -0.976) tracked=true device (0.000, 1.129, -0.001) Questions: Is allAnchors now intentionally expressed in a different frame on visionOS 27, for example related to the new ARKitCoordinateSpaceProviding / coordinateSpace(correction:) API? The documentation for allAnchors does not mention it. If so, which frame is it, and how should it be converted to the ImmersiveSpace frame? Workaround we use: read transforms only from anchorUpdates into a per-id cache and never position from allAnchors. Filed as FB24799304. Minimal repro project attached to the feedback.
1
0
232
3w
Camera specs of world-facing tracking cameras.
We are working on a very space constrained custom spatial accessory. Since the form factor of the device is small, we need to add small LEDs producing small blob diameters with accordingly small distance between them. To better estimate the amount, size, and distance of the LEDs it would help to know which image sensors and field of view the six world‑facing tracking cameras have.
1
0
904
4w
Main Camera development request blocked by Individual-to-Organization conversion
We are developing MadnessBot as a business-only Vision Pro automotive inspection app for professional mechanics and repair shops, intended for private business distribution. Main Camera access is essential for analyzing the vehicle and component in front of the technician. A native development app is running on our physical headset, and chat plus OBDLink CX diagnostics have been tested. Our existing Apple Developer Program membership is enrolled as Individual. We contacted Developer Support on October 6, 2026, with a follow-up on October 9, for assistance converting it to Organization. The conversion form asks for the type of tax ID historically associated with the Individual membership; the Account Holder does not know that record and has asked Support to verify it before selecting an answer. The enterprise API request form remains inaccessible until organization enrollment is confirmed. We have not submitted a camera entitlement request yet. Our planned initial request is Development Only Main Camera access for registered test devices. Feedback reference: FB25132939. Could the Enterprise APIs for visionOS team help route this prerequisite blocker to membership support, or advise the correct escalation path so we can complete conversion and apply for Development Only access?
Replies
0
Boosts
0
Views
615
Activity
3d
visionOS : any non-nil preferredSurroundingsEffect enables a light-spill halo around VideoPlayerComponent video
Hi all — sharing a platform behavior I isolated while building a custom immersive video player, in case others hit it (and to ask whether it's intended). Finding On visionOS 27 (Xcode 27), an entity playing video via VideoPlayerComponent(videoRenderer:) gets a light-spill "screen glow" halo around its edges whenever the immersive space declares any non-nil preferredSurroundingsEffect. The halo follows the video's colors frame by frame. With no surroundings effect (nil), there is no halo. Quick repro Progressive immersive space + VideoPlayerComponent playing. Add .preferredSurroundingsEffect(.dim(intensity: 0.0)) → halo appears (even though the effect is visually a no-op). Remove it → halo disappears. Same with .colorMultiply(_:) and preset effects. Things I ruled out on device (Vision Pro) ImmersiveContentBrightness (.custom(1.0) vs .automatic): no difference. VideoPlayerComponent configuration (.stereo, spatial video mode, isPassthroughTintingEnabled) and component attach timing: no difference. Presenting the same video via VideoMaterial: no halo. Bright white SwiftUI content: no halo. Questions Is this coupling intended? The preferredSurroundingsEffect docs don't mention any effect on video playback. Is there a public way to keep a surroundings effect (e.g. room dimming) while avoiding the video light spill? I found none on VideoPlayerComponent. Current workaround: don't declare any surroundings effect in the modes where the halo is undesirable. Also filed via Feedback Assistant(FB25130748). Curious if anyone else has seen this or knows a switch.
Replies
0
Boosts
1
Views
868
Activity
4d
RoomPlan: placed RealityKit content does not align with the scanned room
I scan a room with RoomPlan API, then keep the same ARSession running (stop(pauseARSession: false)) and place RealityKit entities using the CapturedRoom wall transforms (wall.transform, wall.dimensions), parented to an anchor at the world origin. The camera position stays continuous, so tracking was not reset. But the entities do not sit on the real walls. The whole room appears shifted. What is the correct way to place content in the live AR session so it aligns with the scanned room?
Replies
1
Boosts
0
Views
1.6k
Activity
5d
RealityKit VideoMaterial renders 8-bit HEVC HLG/BT.2020 360° video with a purple tint in the visionOS Simulator (device is fine)
We paint a looping 360° video on the inside of a sphere with VideoMaterial(avPlayer:): an AVQueuePlayer plus AVPlayerLooper, a ModelEntity with an inward-facing equirectangular mesh, inside a .mixed ImmersiveSpace. No shaders, no VideoPlayerComponent. We could not find an official Apple sample that puts a VideoMaterial on a 360° sphere; this is the plain pattern from the VideoMaterial(avPlayer:) reference and the one used in threads like 734081. On Apple Vision Pro hardware the video renders correctly. In the visionOS Simulator (27.0, 26.5 and earlier 26.x, Xcode 27.0) the whole frame has a strong purple/magenta cast. Audio and timing are fine. The video (which we cannot share, so here is its metadata, image sample of issue on simulator provided below): MP4, HEVC Main, 8-bit 4:2:0, 3840x1920, tagged BT.2020 primaries, HLG transfer (ARIB STD-B67), BT.2020 non-constant matrix, both in the colr atom and in the SPS VUI. ffprobe -show_format -show_streams (trimmed): [STREAM] index=0 codec_name=hevc profile=Main codec_type=video codec_tag_string=hvc1 width=3840 height=1920 pix_fmt=yuv420p color_range=tv color_space=bt2020nc color_transfer=arib-std-b67 color_primaries=bt2020 r_frame_rate=30000/1001 avg_frame_rate=679200/22663 duration=37.771667 bit_rate=25030607 [SIDE_DATA] side_data_type=Stereo 3D type=2D horizontal_field_of_view=360000/1000 [/SIDE_DATA] [/STREAM] [STREAM] index=1 codec_name=aac profile=LC codec_type=audio codec_tag_string=mp4a sample_fmt=fltp sample_rate=48000 channels=2 channel_layout=stereo duration=37.758333 bit_rate=128176 [/STREAM] [FORMAT] nb_streams=2 format_name=mov,mp4,m4a,3gp,3g2,mj2 duration=37.778333 bit_rate=25160348 TAG:major_brand=mp42 TAG:minor_version=1 TAG:compatible_brands=isommp41mp42 [/FORMAT] AVFoundation's view of the track, from CMFormatDescriptionGetExtension on the loaded video track: primaries=ITU_R_2020 transfer=ITU_R_2100_HLG matrix=ITU_R_2020 bitsPerComponent=8 The playback code, reduced to the essentials: let item = AVPlayerItem(url: fileURL) let looper = AVPlayerLooper(player: player, templateItem: item) // retained player.actionAtItemEnd = .advance let material = VideoMaterial(avPlayer: player) let sphere = ModelEntity( mesh: try equirectSphere(radius: 12), // lat/long grid, normals inward, materials: [material] // UVs (u, 1 - v), inward winding ) sphere.position = [0, 1.5, 0] rootEntity.addChild(sphere) // rootEntity was added in RealityView { } player.play() What we have ruled out in a minimal sample project: Apple's own 360° HLS example (360 Lighthouse, HEVC Main 10, 10-bit, BT.709 SDR) plays with correct colours in the same sphere, same code, same simulator session. Rewriting our file's colr atom to BT.709 (three bytes, no re-encode) is honoured by AVFoundation (the track's CMFormatDescription then reports ITU_R_709_2 for primaries, transfer and matrix) but the tint stays. So the remaining differences between the failing file and Apple's working stream are 8-bit Main vs 10-bit Main 10, HLG/BT.2020 in the bitstream VUI vs BT.709, and local MP4 vs HLS. The minimal sample project (without the video) is available on request. Is this a known limitation of the simulator's VideoMaterial path with 8-bit HLG content, and is there a recommended way to get device-equivalent colour in the simulator? Feedback filed as FB24997147. Here is a sample of the result on the simulator of a cement column, painted white, but rendered purple on simulator:
Replies
1
Boosts
0
Views
3.1k
Activity
1w
Mac spatial preview feature: Supported with Blender?
Hello! I am relatively new to dev for Apple Vision Pro, however I have owned one since launch and love it. When I heard that we were going to be able to preview real-time, 3D models from the Mac I couldn’t believe it! This is one of my major workflows and I have been using 3rd party workarounds (that don’t work well). My question is, is there any word on supported 3D modeling apps like Blender or Cinema 4D? Or will there be a way to implement this preview feature with any app via a protocol? Any documentation would be really helpfu! Thank you, and continue the great additions to Apple Vision Pro!
Replies
2
Boosts
0
Views
2.8k
Activity
1w
Revealed keyboard drifts in progressive immersion, and no immersion style gives windows-in-front + Crown + passthrough cutouts
Environment: Apple Vision Pro, visionOS 27.0.1 (24M372). Magic Keyboard, with Settings › Awareness & Safety › Reveal Keyboards set to Always. Our app puts the user in an opaque virtual room while they work at their real desk, using app windows and Mac Virtual Display. A virtual desk is placed on the real desk. It is world-locked: positioned once from a PlaneAnchor (PlaneDetectionProvider), then a plain entity at fixed ImmersiveSpace coordinates. No AnchorEntity or WorldAnchor is involved. We offer two modes: .mixed, and .progressive for Digital Crown control. Each gives us only half of what desk work needs. 1. The revealed keyboard drifts in progressive immersion (FB25006789) In .progressive, the physical keyboard shown by keyboard awareness does not stay registered with the virtual desk. After the keyboard has been out of view for a few seconds (for example, while looking at a Mac Virtual Display window), it looks "floaty" when you look back: as the head turns, it slides against the desk. Turning right, the keyboard moves left farther than the desk does. What we checked: It happens in .progressive at any Crown amount, and in .full. With identical content in .mixed (no app passthrough cutout, so again only keyboard awareness shows the keyboard), it is much steadier. On 27.0.1, PlaneAnchor and DeviceAnchor coordinateSpace(correction: .rendered) are exactly equal to their .none transforms: 0 mm, 0°, scale 1. That held over 835 per-frame samples with ±50° yaw and up to 36 cm of head movement. So the rendered correction gives the app nothing to compensate with. The app's desk transform is constant while the drift is visible. upperLimbVisibility(.automatic) vs .visible and a .portrait progressive aspect ratio make no difference. Possibly related: https://developer.apple.com/forums/thread/796557 (FB19666209). 2. No immersion style combines what desk work needs (FB25006837) We need, at the same time: windows (our own and Mac Virtual Display) never covered by the room's walls, furniture or trees; the Digital Crown to bring the real surroundings back in; a passthrough cutout over the real desk, and a well-registered keyboard. .mixed: ✓ OcclusionMaterial cutouts reveal passthrough. ✓ The keyboard stays registered. ✗ RealityKit entities can render in front of windows, including Mac Virtual Display and window chrome. breakthroughEffect doesn't cover whole windows or Mac Virtual Display. ✗ The Crown can't adjust immersion. .progressive: ✓ Windows always stay in front of immersive content (as an Apple engineer confirmed in https://developer.apple.com/forums/thread/764556). ✓ The Crown adjusts immersion. ✗ OcclusionMaterial reveals nothing inside the portal, so there is no desk cutout. ✗ The keyboard drifts (issue 1). Because of issue 1 we have switched our default back to .mixed, and lost windows-in-front. Questions Is keyboard awareness in .progressive/.full expected to be registered less accurately than in .mixed? Is there a supported way to keep it registered with world-locked content, or to get the pose at which the system draws it? In .mixed, is there a supported way to keep windows (including Mac Virtual Display) in front of an app's immersive content? Or, in .progressive, a supported way to show passthrough in an app-defined region inside the portal? Any pointers or workarounds are appreciated.
Replies
0
Boosts
0
Views
1.5k
Activity
1w
How to use ClothGrabComponent?
I've simulated a piece of cloth using https://developer.apple.com/documentation/realitykit/physics-cloth-simulation I would like to grab the cloth like what I can in Marvellous Designer. I found the ClothGrabComponent at https://developer.apple.com/documentation/realitykit/clothgrabcomponent However, after I added the ClothGrabComponent to the cloth entity, I am still not able to grab it. Is there any sample code to show me how to grab the cloth? Thank you.
Replies
2
Boosts
0
Views
1.4k
Activity
2w
Is there currently a way to import 3D Gaussian Splatting data into Reality Composer Pro?
Hello, As far as I understand, RealityKit now supports 3D Gaussian Splatting through GaussianSplatComponent. I would like to ask whether there is currently a way to use 3D Gaussian Splatting data directly in Reality Composer Pro, for example by importing a .ply file containing Gaussian splats. More specifically: Can Reality Composer Pro import a 3D Gaussian Splatting .ply file? Is there a way to create or assign a GaussianSplatComponent to an entity directly within Reality Composer Pro? If this is not currently supported, is there any recommended workflow for creating a .usdz containing Gaussian Splatting data that can be opened in the Preview app on iPhone/iPad and displayed as an AR object? I tried opening the sample plant.usdz on an iPhone using the Preview app. However, the AR preview appears to be empty, so it seems that the Preview app may not currently support rendering the Gaussian Splatting content contained in that USDZ. My desired workflow would be: Gaussian Splatting .ply → Reality Composer Pro / USDZ → iPhone Preview → AR Is this workflow currently possible? If not, is there an official or recommended way to package a GaussianSplatComponent and its associated Gaussian splat data into a USD/USDZ file for AR preview on iOS? If the above functionality is not currently supported, does the development team have any plans to add or improve support for these features in future versions of Reality Composer Pro or the Preview app? Any clarification regarding the current capabilities, limitations, or future plans would be greatly appreciated. Thank you!
Replies
4
Boosts
1
Views
2.7k
Activity
2w
RoomPlan crashes spiking on iOS 27
We're seeing a pretty significant spike in RoomPlan crashes on iOS 27, specifically a RoomScanCore EXC_BREAKPOINT. This crash isn't new, we've seen a low background rate of it across earlier iOS versions, but since iOS 27 shipped, the rate has jumped substantially, well beyond what iOS 27's adoption share alone would explain. We haven't reproduced the crash internally. As with the existing crashes, it's seemingly environment/scenario-dependent in ways that don't always show up in our QA passes. Is anyone else seeing this? Are there known issues that will be patched next month with iOS 27.2?
Replies
1
Boosts
0
Views
1.2k
Activity
2w
RealityKit cloth: reducing mesh collider air gap without penetration
I’m testing RealityKit cloth for garment-like cloth over an authored human-shaped triangle-mesh collider. The issue also occurs with a static collider, so it does not appear to depend on character animation.. Contact is stable and runs interactively, but I cannot achieve close surface contact without sacrificing robustness. The cloth mid-surface remains visibly several millimetres from the authored collider, especially around curved or narrow areas such as the head, arms, and hands. Transparent rendering of visualMesh confirms this is not just render thickness. Reducing ClothMeshShape.bias reduces the gap, but eventually causes interpenetration around difficult geometry. Increasing it prevents penetration but makes the cloth visibly float. I’ve also checked scale, winding, normals, reasonably even triangles, timestep, and solver iterations. I want to avoid inflating the collider or adding anatomical proxy shapes. Is bias the only physical-clearance control, or does RealityKit apply an additional contact radius derived from simulation-mesh resolution? Is local conforming refinement the recommended way to achieve roughly 1–2 mm clearance while retaining robust real-time contact? Are there any specific topology requirements beyond evenly sized triangles? I can provide a minimal reproduction and screenshots if helpful although not sure I can share screenshots here?
Replies
4
Boosts
0
Views
5.1k
Activity
2w
visionOS 27.0 simulator restarts (backboardd EXC_BREAKPOINT in DRValidateIndices) on MeshResource.generate(from:) for a valid multi-part mesh
Since moving to Xcode 27.0 (27A266a) with the visionOS 27.0 simulator runtime (24M362), generating a mesh with several parts through MeshResource.generate(from: MeshResource.Contents) makes the simulator go black for a couple of seconds and restart. The app does not crash; the simulator's render server does: Thread ... Crashed :: com.apple.CoreRE.ResourceFetchManager-service-queue 0 DirectResource specialized static IndexValidation._validate_neon(_:vertexCount:) 1 DirectResource DRValidateIndices 2 CoreRE re::(anonymous namespace)::validatePrivateData(id<REMeshPayload>, unsigned char const*, unsigned long) 3 CoreRE -[REMultiBufferMeshPayload initWithCoder:] 4 Foundation _decodeObject Environment: MacBook Pro (M1 Pro, 16 GB), macOS 27.0 (26A428), Xcode 27.0 (27A266a), visionOS 27.0 SDK (24M361); failing runtime visionOS 27.0 simulator (24M362); working: the visionOS 26.5 simulator runtime (23O470) under the same Xcode, and an Apple Vision Pro on visionOS 27. What I have established: It reproduces with a fully synthetic mesh: one model, three parts (1330 / 20 / 24 vertices, 9513 / 72 / 120 indices), random positions, normals, UVs and triangle indices. Only the part layout matters. The mesh is valid: every part's indices are within its own vertex count, counts are multiples of three, attribute buffers match. RealityKit's app-side validator agrees and lets it through (it does reject a broken part with REMeshPartDescriptor: index buffer payload range is invalid), then the render server's validator traps on the accepted payload. Single-part meshes of any size, primitives, and compiled .reality scenes are fine. Same mesh, same Xcode: fine on a simulator device created on the 26.5 runtime, fine on a visionOS 27 device. Only the 27.0 simulator runtime traps. FB number: FB24831983. Workarounds so far: a simulator device on the 26.5 runtime, or generating one MeshResource per part. Has anyone seen DRValidateIndices trapping on the 27.0 simulator?
Replies
1
Boosts
0
Views
2.6k
Activity
3w
Logitech Muse never appears under Settings > General > Spatial Accessory — spatial tracking permission never granted
I'm building a visionOS 26 app that uses SpatialTrackingSession with .accessory tracking and AnchoringComponent.AccessoryAnchoringSource to track a Logitech Muse, following the pattern from the WWDC25 "Explore spatial accessory input on visionOS" session (and matching two independently published implementations I found). What works: The Muse pairs fine over Bluetooth, connects as a GCStylus (productCategory == .spatialStylus), GCStylusDidConnect fires, and button input (pressedDidChangeHandler) works correctly. What doesn't: SpatialTrackingSession.run() always reports .accessory as unavailable, so AnchoringComponent.AccessoryAnchoringSource(device:) never succeeds. Checking Settings > General > Spatial Accessory on the device shows the Muse isn't listed there at all — not even a battery/status entry. Settings > Privacy & Security > Spatial Accessory also never lists our app. What I've tried: Forgetting the Muse in Bluetooth and re-pairing Full power-cycle of both Bluetooth and the Muse itself Confirmed NSAccessoryTrackingUsageDescription is set and the Accessory Tracking + Game Controllers (Spatial Gamepad) capabilities are enabled in Xcode Confirmed via Apple's own docs, the WWDC session, and two developer write-ups that no special commissioning step beyond standard Bluetooth pairing should be needed For comparison, XR Putt (which also uses the Muse) works fine and does show up with a green checkmark under Spatial Accessory permissions on the same headset. Has anyone hit this specific "device never gets commissioned as a spatial accessory" issue, or knows what actually triggers that commissioning beyond generic Bluetooth pairing? Happy to share more logs/details.
Replies
1
Boosts
0
Views
1.6k
Activity
3w
ARKit World Tracking Drift Regression on LiDAR-Equipped Devices - iOS 26.4+
Summary We have identified a reproducible world tracking drift regression in ARKit on LiDAR-equipped iOS devices running iOS 26.4 and later. A static virtual node anchored at the world origin visually drifts from its initial position as the user moves around a real-world scene, despite the scene remaining physically static. The same code produces stable, drift-free results on non-LiDAR devices running identical OS versions. Device & OS Observations Testing was performed across four devices on iOS 26.4 using the same application build and ARWorldTrackingConfiguration settings. Non-LiDAR devices — iPhone 14 and iPhone 15 — produced stable, drift-free tracking in all test runs. No world origin displacement was observed regardless of how long or how far the user walked. LiDAR-equipped devices — iPhone 14 Pro and iPhone 16 Pro — exhibited consistent, reproducible drift. A static node placed at the world origin visually shifted from its initial position as the user moved through the scene. The same devices were stable on earlier iOS versions, confirming this is a regression introduced in iOS 26.4. Technical Observations Nature of drift: A SCNNode placed statically at the ARKit world origin (SCNVector3(0, 0, 0)) visually displaces from its original position as the user walks around a static real-world scene. The displacement is not random — it accumulates directionally as the user moves, consistent with a sensor fusion or coordinate anchoring error. Trigger condition: The drift occurs during normal walking motion around a fixed point of interest, such as circling a parked vehicle. It does not appear when the device is held still. LiDAR specificity: The drift is exclusive to devices with a LiDAR scanner. Identical hardware configurations — same iOS build, same ARWorldTrackingConfiguration settings — on non-LiDAR devices produce no drift whatsoever. This isolates the regression to the LiDAR sensor's contribution to ARKit's internal Visual-Inertial Odometry (VIO) fusion pipeline. No API-level workaround found: There is currently no public ARKit API to selectively disable the LiDAR scanner's contribution to VIO. All available configuration-level options have been evaluated without resolving the drift. ARWorldTrackingConfiguration Options Evaluated The following configuration changes were applied individually and in combination. None resolved the drift on LiDAR devices: isAutoFocusEnabled = false — No improvement videoHDRAllowed = false (disabled) — No improvement planeDetection = [] (disabled) — No improvement sceneReconstruction = .mesh — No improvement worldAlignment: .gravity vs .gravityAndHeading — No improvement Minimal Reproduction Case The drift can be reproduced with a minimal ARKit scene: Create an ARSCNView with ARWorldTrackingConfiguration using default settings. Add a single static SCNNode (e.g., a small sphere or axes geometry) at SCNVector3(0, 0, 0) when the session starts. Run the app on a LiDAR-equipped device (iPhone Pro, iPad Pro with LiDAR) on iOS 26.4 or later. Walk in a circle around the node's approximate real-world position. Expected: The node remains visually fixed at its world position throughout the walkthrough. Actual: The node drifts from its initial position, increasingly displaced from its world origin anchor as walking continues. We want to know if Apple has made any internal updates to ARKit, particularly after the OS 26.4 upgrade. Thanks!
Replies
7
Boosts
8
Views
4.3k
Activity
3w
Production LiDAR capture app affected by ARKit world-tracking drift on iOS 26.4+ (re: thread 827240) mitigation guidance and fix timeline?
Hello, and thank you for taking questions this week. We develop DeepWalk, a production App Store application that captures LiDAR depth, RGB, and ARKit camera poses while field technicians walk continuous outdoor routes (sidewalk and infrastructure assessment). Recorded ARCamera.transform values are used downstream for 3D reconstruction, so world-tracking accuracy directly determines our output quality. Problem: Beginning in early April 2026 and coinciding with the iOS 26.4 rollout across our device fleet, we observe directional, accumulating pose drift in recorded trajectories during normal walking motion. The behavior matches the regression reported in forum thread 827240 (world-tracking drift on LiDAR devices on iOS 26.4+, absent on non-LiDAR devices running identical code). Link to existing post here: https://developer.apple.com/forums/thread/827240 Our configuration is minimal: a default ARWorldTrackingConfiguration with frameSemantics = [.sceneDepth, .smoothedSceneDepth] and no other options set, running on a bare ARSession with a Metal renderer (no SceneKit/RealityKit). We measure drift directly from per-frame ARCamera.transform values, so the displacement appears in ARKit's pose output itself rather than in any rendering layer. The drift accumulates with walking distance and does not occur when the device is stationary. We've observed this across a large variety of devices (iPhone 15/16/17 Pro/Pro Max) on all iOS versions 26.4 or greater. We've also observed this on iPad Pros with updated firmware. Thread 827240 additionally reports that disabling autofocus, HDR, and plane detection and varying scene reconstruction and world alignment did not resolve it. Question Because depth capture is our core function, we cannot avoid engaging the LiDAR scanner, and we're not aware of a public API that adjusts its contribution to world tracking. We'd be grateful for guidance on three questions: Is this a known issue in iOS 26.4 and later? Did the LiDAR contribution to world tracking change in that release? For apps that require sceneDepth, is there any recommended session configuration, video format, or capture-side practice that reduces the drift in the interim? Is a fix planned, and if so, should we expect it in an iOS 26.x update or the iOS 27 cycle? This affects how we manage our deployed device fleet. We're happy to provide trajectory datasets, additional sysdiagnoses, or anything else useful. We have quantified drift-per-distance data across OS versions.
Replies
7
Boosts
1
Views
1.6k
Activity
3w
WebXR Augmented Reality Module
On visionOS 26.5, I noticed the Safari Extension 'WebXR Augmented Reality Module' option. I've enabled this option in Safari Settings, and restarted Safari, but when I try to launch a WebXR experience that supports immersive-ar as option (for example this one: https://www.viverse.com/dPASDhP I don't see the option to Play in AR (like I do with a Quest 3S headset) Does anyone have any examples of websites that can make use of this new WebXR Augmented Reality Module feature flag in Safari, on visionOS?
Replies
3
Boosts
4
Views
1.7k
Activity
3w
Main Camera Error
An incomprehensible error occurs whenever I try to access Main Camera features. I have already acquired the Main Camera entitlement and granted the required permissions, yet the error persists. To make matters worse, even Apple’s provided sample code triggers the exact same issue. The error message is far too uninformative to diagnose the root cause, and I urgently need a resolution as this is completely blocking progress. Multiline [C:1-2] Error received: Invalidated by remote connection. failed to observe with mask <BLSXPCBacklightProxyObserverMask: 0x108accf80; didUpdateToState: YES; eventsArray: YES> error:<XPC error received on message reply handler (3:BSServiceConnectionErrorDomain) "The operation couldn’t be completed. XPC error received on message reply handler"> non-launching port is incompatible with service identifier "com.apple.PointerUI.pointeruid.default-service" nw_socket_copy_info [C1:2] getsockopt TCP_INFO failed [102: Operation not supported on socket] nw_socket_copy_info getsockopt TCP_INFO failed [102: Operation not supported on socket] Unable to find a compatible slice for binary archive at /System/Library/Frameworks/CoreImage.framework/ci_uberwrapper_bin.metallib *** Assertion failure in void ar_session_request_authorization(ar_session_t _Nonnull, ar_authorization_type_t, __strong ar_authorization_results_handler_t _Nonnull)_block_invoke(), session.m:1491, reason: This app failed to request an authorization. The NSWorldSensingUsageDescription / NSHandsTrackingUsageDescription / NSAccessoryTrackingUsageDescription / NSMainCameraUsageDescription keys may be missing from this app's Info.plist. <NSXPCConnection: 0x10e68dcc0> connection to service with pid 86 named com.apple.arkit.service.authorization: Exception caught during invocation of reply block to message 'requestAuthorizations:withReply:'. Exception: This app failed to request an authorization. The NSWorldSensingUsageDescription / NSHandsTrackingUsageDescription / NSAccessoryTrackingUsageDescription / NSMainCameraUsageDescription keys may be missing from this app's Info.plist. ( 0 CoreFoundation 0x0000000198b756d4 686CE4EB-CE52-3DD5-8B23-EF6A6BBFACEA + 1205972 1 libobjc.A.dylib 0x000000018d3a71cc objc_exception_throw + 88 2 CoreFoundation 0x0000000198b755c0 686CE4EB-CE52-3DD5-8B23-EF6A6BBFACEA + 1205696 3 ARKitCore 0x00000002bbd640cc 53FDDDD4-8DD1-3E86-ACB0-8867D29AF781 + 479436 4 CoreFoundation 0x0000000198b7c5b4 686CE4EB-CE52-3DD5-8B23-EF6A6BBFACEA + 1234356 5 CoreFoundation 0x0000000198b79558 686CE4EB-CE52-3DD5-8B23-EF6A6BBFACEA + 1221976 6 Foundation 0x000000019de4220c 61E78A52-3B44-3DCE-AF6D-92C33FA0FACA + 2109964 7 Foundation 0x000000019de41dd0 61E78A52-3B44-3DCE-AF6D-92C33FA0FACA + 2108880 8 Foundation 0x000000019de4651c 61E78A52-3B44-3DCE-AF6D-92C33FA0FACA + 2127132 9 libxpc.dylib 0x00000002c43d5a00 CE0BB604-B358-3F22-B1DA-FFACEFCA20AD + 150016 10 libxpc.dylib 0x00000002c43c5678 CE0BB604-B358-3F22-B1DA-FFACEFCA20AD + 83576 11 libdispatch.dylib 0x00000002c41045fc 4CF750D7-AC3B-3175-9863-2F059D3D3715 + 108028 12 libdispatch.dylib 0x00000002c41083ec 4CF750D7-AC3B-3175-9863-2F059D3D3715 + 123884 13 libdispatch.dylib 0x00000002c40f37c0 4CF750D7-AC3B-3175-9863-2F059D3D3715 + 38848 14 libdispatch.dylib 0x00000002c40f4450 4CF750D7-AC3B-3175-9863-2F059D3D3715 + 42064 15 libdispatch.dylib 0x00000002c40fe3ec 4CF750D7-AC3B-3175-9863-2F059D3D3715 + 82924 16 libdispatch.dylib 0x00000002c40fdd18 4CF750D7-AC3B-3175-9863-2F059D3D3715 + 81176 17 libsystem_pthread.dylib 0x00000002c4364e34 _pthread_wqthread + 292 18 libsystem_pthread.dylib 0x00000002c4362910 start_wqthread + 8 ) *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'This app failed to request an authorization. The NSWorldSensingUsageDescription / NSHandsTrackingUsageDescription / NSAccessoryTrackingUsageDescription / NSMainCameraUsageDescription keys may be missing from this app's Info.plist.' *** First throw call stack: (0x198b756c8 0x18d3a71cc 0x198b755c0 0x2bbd640cc 0x198b7c5b4 0x198b79558 0x19de4220c 0x19de41dd0 0x19de4651c 0x2c43d5a00 0x2c43c5678 0x2c41045fc 0x2c41083ec 0x2c40f37c0 0x2c40f4450 0x2c40fe3ec 0x2c40fdd18 0x2c4364e34 0x2c4362910) *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'This app failed to request an authorization. The NSWorldSensingUsageDescription / NSHandsTrackingUsageDescription / NSAccessoryTrackingUsageDescription / NSMainCameraUsageDescription keys may be missing from this app's Info.plist.' *** First throw call stack: (0x198b756c8 0x18d3a71cc 0x198b755c0 0x2bbd640cc 0x198b7c5b4 0x198b79558 0x19de4220c 0x19de41dd0 0x19de4651c 0x2c43d5a00 0x2c43c5678 0x2c41045fc 0x2c41083ec 0x2c40f37c0 0x2c40f4450 0x2c40fe3ec 0x2c40fdd18 0x2c4364e34 0x2c4362910) libc++abi: terminating due to uncaught exception of type NSException Xcode has killed the LLDB RPC server (pid: 2210) to allow the debugger to detach from your process. BlockQuote visionOS 27 beta 5
Replies
5
Boosts
0
Views
1.2k
Activity
3w
BloomComponent lags behind the camera on iOS/macOS but not visionOS
BloomComponent (RealityKit 27) lags behind the geometry that produces it while the camera moves. The halo trails the emissive geometry by roughly one to three frames (eyeballed) and snaps back once the camera stops. Video attached — it's obvious at normal playback speed. visionOS renders the identical scene correctly. Only iOS and macOS / Mac Catalyst lag, which points at the non-visionOS compositing path rather than the effect itself. Filed as FB23960052. This should be fixed before 27 ships — bloom is unusable for anything with a moving camera in its current state on those platforms. Ruled out: Scope. Identical with BloomComponent(scope: .hierarchical) and .unbounded. .unbounded computes no per-entity screen-space bounds, so stale bounds are not the cause. Input handling. The camera is RealityKit's own .realityViewCameraControls(.orbit) — no gesture code of mine involved. Per-frame component writes. BloomOptionsComponent is set once and untouched during the drag. Geometry and camera transform. Both track perfectly; only the glow lags. Reproducer is ~130 lines, no assets — five emissive spheres inside a large inward-facing dark sphere: var material = PhysicallyBasedMaterial() material.baseColor = .init(tint: .black) material.emissiveColor = .init(color: .cyan) material.emissiveIntensity = 4 // ... root.components.set(BloomComponent(scope: .unbounded)) var options = BloomOptionsComponent() options.strength = 1 options.threshold = 1 options.blurRadius = 1 root.components.set(options) shown in: RealityView { content in content.camera = .virtual content.add(root) } .realityViewCameraControls(.orbit) Xcode 27.0 beta 4, iOS 27.0 SDK, Apple Silicon. Affected: iOS 27, macOS 27 / Mac Catalyst 27. Not affected: visionOS 27 (immersive space). Full project and screen recording attached to my radar.
Replies
4
Boosts
0
Views
1.9k
Activity
3w
GaussianSplatComponent — stale visibility set after close-range camera approach
Title: GaussianSplatComponent — stale visibility set after close-range camera approach (iOS 27 beta 4, still unfixed) Filed this as a radar during beta 3; still reproduces unchanged on beta 4, so posting here in case anyone from the RealityKit team can confirm it's tracked before this ships. Radar: FB23749559 (filed against beta 3, re-verified on beta 4) Summary On iOS, GaussianSplatComponent keeps a stale visibility set after the camera moves close to and then away from the splat. Moving the camera in close causes a large portion of splats to be culled — expected. But pulling the camera back out does not repopulate them. The visible set stays stale for as long as the current camera transform is held. The missing splats only return once the camera transform changes again — any pan/orbit/zoom delta triggers a recompute and the full splat snaps back. Net effect: after any close inspection, the splat renders permanently partially-culled until the user happens to nudge the camera. The recovery-on-movement / no-recovery-when-stationary signature suggests the iOS render path recomputes splat residency on camera-transform change events rather than every frame. This also reproduces in AR mode, with a completely different camera driver. Since the only thing the two paths share is the splat residency code, that's where the stale set has to be coming from — which also means there's no app-side fix available to me. The camera transform in AR is driven by ARKit, not by my code, so I can't force the perturbation that would otherwise work around it. visionOS 27, running the same component, is unaffected — the visible set is re-evaluated per frame and the splat stays complete. Steps to reproduce Attach a GaussianSplatComponent (~400k splats) to an entity in a bounded RealityView with orbit controls. Move the camera close until a visible portion of the splats is culled. Move the camera back out so the full splat is within the frustum. Hold the camera still. Expected: full splat visible once pulled back. Actual: previously-culled splats stay missing while the camera is stationary; they reappear only on the next camera transform delta. Configuration iOS 27 beta 4 (originally filed against beta 3) iPhone 16 Pro ~400k splats Bounded RealityView + orbit controls; also reproduces in AR Screenshots attached: full splat, stale-culled state after pull-back, and the close-range view that triggers it. This is still present in beta 4 and in the public beta. I'd like to know whether it's tracked, or whether I should plan around splats being unreliable on iOS 27. Thanks!
Replies
5
Boosts
0
Views
1.5k
Activity
3w
visionOS 27: allAnchors and anchorUpdates disagree on WorldAnchor transforms (different coordinate frame)
After updating to visionOS 27.0, WorldTrackingProvider.allAnchors returns WorldAnchor.originFromAnchorTransform values that do not match what anchorUpdates delivers for the same anchor, read at the same moment, both isTracked == true, with no .updated event in between. The difference is a rigid frame change, not drift: Same vertical offset on every anchor: -1.173 m. Identical across two apps and two days with the head at 1.13 m and 1.22 m, so a constant, not head height. Horizontal deltas vary per anchor, but inter-anchor distances are preserved, so it is a yaw plus translation. Stable over time, same across launches. anchorUpdates matches the ImmersiveSpace frame RealityKit draws in. allAnchors does not. On visionOS 26 both agreed. Same anchor, same instant: allAnchors (0.1486, -0.7143, 0.2231) anchorUpdates (0.1316, 0.4590, -1.0955) device pose (0.0073, 1.2195, -0.0038) Four other anchors, delta = allAnchors - anchorUpdates: ( 2.6933, -1.1733, -0.2514) (-0.0810, -1.1733, 1.1339) ( 0.1878, -1.1733, 0.9891) ( 2.6289, -1.1733, -0.1809) In our app this showed up as a placed model vanishing after a scene switch: we re-pinned the model root from allAnchors, and it landed below the floor behind the user. A standalone sample (one provider, one anchor, a green sphere from anchorUpdates, a red sphere from allAnchors) shows two spheres on visionOS 27.0, the red one below the floor: added F21942C4-… at (-0.031, 0.712, -0.976) update added F21942C4-… tracked=true at (-0.031, 0.712, -0.976) MISMATCH delta (-0.211, -1.173, 1.444) m allAnchors (-0.243, -0.461, 0.468) anchorUpdates (-0.031, 0.712, -0.976) tracked=true device (0.000, 1.129, -0.001) Questions: Is allAnchors now intentionally expressed in a different frame on visionOS 27, for example related to the new ARKitCoordinateSpaceProviding / coordinateSpace(correction:) API? The documentation for allAnchors does not mention it. If so, which frame is it, and how should it be converted to the ImmersiveSpace frame? Workaround we use: read transforms only from anchorUpdates into a per-id cache and never position from allAnchors. Filed as FB24799304. Minimal repro project attached to the feedback.
Replies
1
Boosts
0
Views
232
Activity
3w
Camera specs of world-facing tracking cameras.
We are working on a very space constrained custom spatial accessory. Since the form factor of the device is small, we need to add small LEDs producing small blob diameters with accordingly small distance between them. To better estimate the amount, size, and distance of the LEDs it would help to know which image sensors and field of view the six world‑facing tracking cameras have.
Replies
1
Boosts
0
Views
904
Activity
4w