RealityKit

RSS for tag

Simulate and render 3D content for use in your augmented reality apps using RealityKit.

Posts under RealityKit tag

200 Posts

Post

Replies

Boosts

Views

Activity

PhotogrammetrySession(input: [PhotogrammetrySample])
Please I would really appreciate a timely response to this: Providing a response and feedback to this: https://origin-devforums.apple.com/forums/thread/827311 and https://developer.apple.com/forums/thread/827043?page=1#889020022 Feedback Report: I have created a feedback report as well like recommended with this ID: FB22823733, with more elaborate images of my implementation, also see here More clarity: Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I also tried using the PhotogrammetrySession folder run but its still failed with : CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing. Please I would really appreciate a timely response to this
1
0
925
Jun ’26
Object Capture broken on iPhone 17 Pro (iOS 26.x): ObjectCaptureSession sensorFailed + PhotogrammetrySession cv3dapi 4004
The full RealityKit Object Capture stack fails device-specifically on the iPhone 17 Pro family (iPhone18,1 / iPhone18,2) under iOS 26.x. Both the live and offline paths break. The same code and images work on iPhone 12 Pro (iPhone13,3) at the same iOS version. Both ObjectCaptureSession.isSupported and PhotogrammetrySession.isSupported return true on the 17 Pro, so there's no API signal that the operation will fail — it only fails at runtime on real hardware. Defect 1 — ObjectCaptureSession.Error.sensorFailed A guided capture session fails. On iOS 26.4.2 it fails at scan start. On iOS 26.5 it reaches .capturing and renders the live session for several seconds, but auto-capture never fires; attempting a manual capture then fails. The Swift enum (.sensorFailed) hides the cause; the underlying framework log is: <<<< FigXPCUtilities >>>> signalled err=-17281 at <>:308 <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:569) - (err=-17281) ObjectCaptureSession.takeStillImageCapture(isAutoCapture:): Failed to capture high resolution frame! ARSession <0x…>: did fail with error: Error Domain=com.apple.arkit.error Code=102 "Sensor failure." UserInfo={NSUnderlyingError=0x… {Error Domain=AVFoundationErrorDomain Code=-11819 "Cannot complete action" UserInfo={NSLocalizedRecoverySuggestion=Try again later.}} NSLocalizedFailureReason=The sensor did not provide the required data.} Capturing failed with error: objectCaptureSessionError(CoreOC.ObjectCaptureSession.Error.sensorFailed) A second, uncatchable variant aborts instead of returning the error: Abort: …/AppleCVHWA/library/VIO/HWFeatureDetection/src/HwGeneralProcessingAPI.cpp:711 : Original error reported by request utility function: kVisionHWAReturnGeneralProcessingFailed Appears to be mediaserverd resetting (−11819) underneath the live AVCaptureSession, tearing it out from under ARKit (ARError 102). Configuration-independent: tested with isOverCaptureEnabled = false and the identical chain still occurs. Defect 2 — PhotogrammetrySession reconstruction (cv3dapi 4004) Folder-mode reconstruction starts but fails after a few seconds; SfM builds no pose map: ERROR cv3dapi.pg: Internal error codes (1): 4004 // "No SfM map found" WARN cv3dapi.pg: Internal warning codes (2): 2502 3502 Output error with code = -15 requestError: CoreOC.PhotogrammetrySession.Error.processError The input is just a folder of images handed to PhotogrammetrySession in folder mode — no other configuration. The same folder reconstructs fine on the 12 Pro. Affected config Failing: iPhone 17 Pro (iPhone18,1), 17 Pro Max (iPhone18,2), iOS 26.4.2 and 26.5 Working (same code/OS): iPhone 12 Pro (iPhone13,3) Open question Does a 17-Pro image set fail PhotogrammetrySession everywhere, or only on the 17 Pro? Fails everywhere → 17 Pro images / HEIC metadata confuse SfM. Works elsewhere → the on-device reconstruction engine itself is the defect. Sample image set + sysdiagnose available on request. References cv3dapi 4004 = "No SfM map found" (683300) · -15/processError wrapper (794169) · iOS 26 Object Capture regression (810496) · 17 Pro Max sensorFailed (827216)
1
0
1k
Jun ’26
Reality View Preserves Camera Transform when toggling Virtual & Spatial Tracking modes
When switching from RealityView’s .spatialTracking camera mode to .virtual camera mode, the camera’s orientation relative to the scene is preserved permanently with no way to reset to default World-Up orientation. Since .spatialTracking’s camera mode will always have a non-default orientation, switching to .virtual camera mode ensures that the cameras’s ‘UP’ direction will never match the device display’s ‘UP’ direction as is default. This is especially noticeable when using .orbit camera controls, as the orbit’s UP direction matches the scene, not camera, and all rotation directions give unexpected results. Expected: When setting virtual camera mode after using spatialTracking camera mode, either 1. The Virtual Camera orientation returns to default (world up). Or 2. A 'content.camera.resetOrientation()' call is made available which resets the RealityView camera to default orientation. Reality: Switching from .spatialTracking -> .virtual camera mode permanently locks the .virtual camera’s orientation the final frame of the .spatialTracking camera’s rotation (relative to the RealityView content scene). One imperfect workaround is to reset / rebuild the entire RealityView after changing modes (by resetting .id() or otherwise. This is not ideal as it causes everything inside the make closure to rerun, which not only is a performance & time cost, visually incurs a flicker and can also be problematic with managing increasingly complicated views. Another imperfect alternative is to use more than one RealityView - which is not ideal as it incurs double the base ram usage, significantly increases code, and seemingly goes against the intent of being able to change the camera .virtual/.spatatialTracking mode at will. Code Sample: import SwiftUI import RealityKit struct RKSpatialVirtualToggle: View { @State var showAR: Bool = false var body: some View { RealityView { content in let cube = ModelEntity(mesh: .generateBox(size: 0.25), materials: [SimpleMaterial()]) cube.position.z = -1 content.add(cube) content.camera = showAR ? .spatialTracking : .virtual content.cameraTarget = cube } update: { content in content.camera = showAR ? .spatialTracking : .virtual } .realityViewCameraControls(.orbit) VStack{ Spacer() Button("Toggle AR"){ showAR.toggle() } .buttonStyle(.borderedProminent) } } } Xcode Version: Version 26.0 (17A324) iOS Version: iOS 26.5 (23F75) Tested on devices, iPhone 12 Pro, iPhone 15 Pro
1
0
667
Jun ’26
PhotogrammetrySession(input: [PhotogrammetrySample]) Hangs or terminates
Providing clarity and response to this: https://origin-devforums.apple.com/forums/thread/827311 and https://developer.apple.com/forums/thread/827043?page=1#889020022 Feedback Report: I have created a feedback report as well like recommended with this ID: FB22823733, with more elaborate images of my implementation, also see here More clarity: Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I also tried using the PhotogrammetrySession folder run but its still failed with : CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing.
0
0
1.2k
May ’26
`FigAudioSession(AV) err=-19224` triggered by empty Button tap on visionOS 26.5, breaking subsequent AVAudioSession configuration
Environment Device: Apple Vision Pro (real device) OS: visionOS 26.5 Xcode: 26.5 Framework: AVFAudio / AVFoundation Summary On visionOS 26.5, tapping an empty Button consistently emits the following internal warning before the action closure executes: <<<< FigAudioSession(AV) >>>> signalled err=-19224 (<>:612) After this warning is emitted, any subsequent call to configure AVAudioSession silently stops working — audio input and output become non-functional for the lifetime of the session. If the same configuration is performed without a preceding button tap (e.g., inside View.task {}), it succeeds and audio works correctly. Reproduction Due to a dependency on LiveKitWebRTC (livekit/webrtc-xcframework) for WebRTC-based Realtime API audio, we are unable to provide a full self-contained sample project. However, the AVAudioSession configuration code involved is as follows: static func configureAudioSession() { #if !os(macOS) do { let audioSession = AVAudioSession.sharedInstance() #if os(tvOS) try audioSession.setCategory(.playAndRecord, options: []) #else try audioSession.setCategory(.playAndRecord, options: [.defaultToSpeaker]) #endif try audioSession.setMode(.videoChat) try audioSession.setActive(true, options: .notifyOthersOnDeactivation) } catch { print("Failed to configure AVAudioSession: \(error)") } #endif } Scenario A — Button tap (fails): Button("Start") { configureAudioSession() // FigAudioSession err=-19224 appears; audio stops working } Scenario B — View.task (succeeds): .task { configureAudioSession() // No warning; audio works correctly } The only difference is whether a user gesture (Button tap) precedes the call. Observed Behavior Tapping any Button on visionOS 26.5 causes FigAudioSession(AV) err=-19224 to be signalled at <>:612, even before the action closure runs. After this warning, AVAudioSession configuration appears to have no effect — setActive(true) does not throw, but audio appears to stop functioning. Configuring the session prior to any button interaction (e.g., in View.task {}) works correctly. Expected Behavior A Button tap should not implicitly interfere with the audio session state. AVAudioSession configuration should succeed regardless of the UI event context that triggers it. Questions What does FigAudioSession(AV) err=-19224 mean? Does it correspond to a documented AVAudioSession.ErrorCode? Why does a Button tap trigger a FigAudioSession signal on visionOS? Is the system performing implicit audio session management when detecting user interaction? Is there a recommended pattern for configuring AVAudioSession in response to a user gesture on visionOS? Our current workaround (View.task {}) is not suitable for on-demand audio start triggered by the user. Is err=-19224 causally responsible for the subsequent audio issue? Since setActive(true) does not throw after the warning, it is unclear whether this signal is the direct cause of the apparent audio failure or a symptom of a deeper conflict. Are there UI components or APIs on visionOS that do not trigger this signal, while still being user-interaction driven? Additional Notes Reproducible only on physical Apple Vision Pro hardware; not observed in Simulator. AirPlay mirroring is not in use during testing. No other apps are playing audio in the background at the time of reproduction. We use LiveKitWebRTC (livekit/webrtc-xcframework, revision 94ce1c9) for WebRTC audio. However, the FigAudioSession warning appears independently of the WebRTC layer — it is emitted on Button tap even before configureAudioSession() is called. We have verified that calling configureAudioSession() before performHandshake() (i.e., before WebRTC initializes its audio pipeline) does not resolve the issue when a Button tap precedes the call.
0
0
529
May ’26
PhotogrammetrySession(input: [PhotogrammetrySample]) Hangs or terminates
Providing a response and feedback to this: https://origin-devforums.apple.com/forums/thread/827311 and https://developer.apple.com/forums/thread/827043?page=1#889020022 I have a created a feedback/bug report with ID FB22823733 Feedback Report: I have created a feedback report as well like recommended with this ID: FB22823733, with more elaborate images of my implementation, also see here More clarity: Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I also tried using the PhotogrammetrySession folder run but its still failed with : CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing.
0
0
1.7k
May ’26
RealityView AR - anchored to the screen not the floor
This started out as a plea for help, but in preparing this post I discovered the root cause. I'm posting it as a lesson learned in hopes it will help someone. I've spent a good chunk of March trying to get AR-mode working again in my unreleased game. I had it working with SceneKit and ARView 5 years ago, but since 2024 I've been converting the game to use RealityKit and RealityView on iOS, macOS, visionOS, and tvOS. I've been having no joy getting AR mode to work on iOS. I get the pass-through device video but the game content isn't anchored to the floor but rather anchored to the screen. I made a simple project with just a simple shape in the middle of a RealityView and an overlay with a SwiftUI toggle to go in and out of AR-mode. At first, my simple project worked, and I couldn't figure out what was different in the logic. Both projects used the same logic: func transitionToXR(_ content: inout RealityViewCameraContent) { content.remove(gameBoard.rootEntity) content.add(xrAnchor) content.camera = .spatialTracking Self.anchorStateChangedSubscription = content.subscribe(to: SceneEvents.AnchoredStateChanged.self) { event in if event.anchor == xrAnchor, event.isAnchored { xrAnchor.addChild(gameBoard.rootEntity) } } } Then I made an alternate version of my view, and reproduced the same "anchored to the screen not the floor" issue. I compared the code side-by-side and finally saw the difference! The one that didn't work, like my game, had a property 'cameraEntity' which is initialized with PerspectiveCamera(), position and look-at configured, then added as a child of the root entity. So, the simple fix was to remove 'cameraEntity' from the root entity before adding it to the detected AnchorEntity when going into AR-mode. Then when leaving AR-mode, I add back 'cameraEntity' as a child of the root entity and configure it again. So the lesson learned is: make sure there isn't a PerspectiveCamera in the tree of Entities added to an AnchorEntity with a .spatialTracking content camera. Apple: let me know if you think this is a bug or if I was being dumb. If a bug, I can use Feedback Assistant to report this. If I was being dumb, it wouldn't be the first time. :-)
5
0
1.2k
May ’26
RealityView Camera Target Error when set while Orbiting
When interacting with RealityView’s realityViewCameraControls .orbit and setting a new RealityViewCameraContent .cameraTarget, the resulting camera target and camera orbit is incorrect. This can be demonstrated where one finger is orbiting the RealityView, and another pushes a button which changes the camera target. Instead of the camera facing the new target, some point in the scene is the new effective camera target and orbit point. This only occurs when an orbit interaction is currently taking place. If you stop interacting with the orbit, change target, then start orbit interacting again, everything works as expected. Though this example uses two-touches, any change of the camera target has this conflict with orbit interaction. This means interacting with orbit will result in the wrong camera view which is unexpected for users and difficult to reconcile or detect, for developers. Expected: Interacting (orbiting) the scene while setting a new camera target with the buttons on screen (at the same time), the camera’s new target shows centred in view the orbit revolves the new target and continues to match my gestures. Reality: Interacting (orbiting) the scene while setting a new camera target with the buttons on screen (at the same time), the camera’s new target is not centred in view, and camera is now orbiting an unexpected point in the scene, that is not my expected target. One imperfect workaround is to force a rebuild of the view after setting a new cameraTarget. This sets all targets correctly but results in a flicker, loss of orbit controls until re-touch and ultimately is a poor user experience, but is better than the wrong target being shown unexpectedly. Code Sample: import SwiftUI import RealityKit struct RKOribtTarget: View { @State private var target: Int = 0 @State private var rcContent: RealityViewCameraContent? @State private var rkID: UUID = UUID() let root = Entity() let center = ModelEntity(mesh: .generateSphere(radius: 0.05), materials: [UnlitMaterial(color: UIColor(.gray.opacity(0.5)))]) let red = ModelEntity(mesh: .generateBox(size: 0.1), materials: [SimpleMaterial(color: .red, isMetallic: false)]) let blue = ModelEntity(mesh: .generateBox(size: 0.1), materials: [SimpleMaterial(color: .blue, isMetallic: false)]) let green = ModelEntity(mesh: .generateBox(size: 0.1), materials: [SimpleMaterial(color: .green, isMetallic: false)]) var body: some View { VStack{ RealityView { content in red.position.x = 0.5 blue.position.z = 0.5 green.position.y = 0.5 center.position = .init(repeating: 0.25) content.cameraTarget = target == 0 ? root : blue root.addChild(red) root.addChild(blue) root.addChild(green) root.addChild(center) content.add(root) } update: { content in switch target{ case 0: content.cameraTarget = root case 1: content.cameraTarget = blue case 2: content.cameraTarget = red case 3: content.cameraTarget = green default: content.cameraTarget = root } } .id(rkID) .realityViewCameraControls(.orbit) VStack{ Text("Target") Button("Default") { target = 0 // Force rebuilding view resets orbit target and rotation // But shows a flicker, interaction requires touch reset // Not an ideal workaround // rkID = UUID() } .buttonStyle(.bordered) Button("Blue") { target = 1 // rkID = UUID() } .buttonStyle(.bordered) .tint(.blue) Button("Red") { target = 2 // rkID = UUID() } .buttonStyle(.bordered) .tint(.red) Button("Green") { target = 3 // rkID = UUID() } .buttonStyle(.bordered) .tint(.green) } } } } Xcode Version: Version 26.0 (17A324) iOS Version: iOS 26.5 (23F75) Tested on devices, iPhone 12 Pro, iPhone 15 Pro
2
0
1.1k
May ’26
RealityKit custom component: `has()` returns `true` but typed subscript returns `nil` in SwiftPM test runner
swift test (SwiftPM CLI) fails to decode RealityKit custom components from USD files, even though entity.components.has(MyComponent.self) returns true. Typed access via entity.components[MyComponent.self] returns nil. This forces projects that use RealityKit custom components to use xcodebuild test exclusively. Minimal repro: github.com/mesqueeb/swiftpm-realitykit-custom-component-repro Repro steps git clone https://github.com/mesqueeb/swiftpm-realitykit-custom-component-repro cd swiftpm-realitykit-custom-component-repro swift test --filter componentsPresentButNotDecodableInSwiftTest Observed ✅ entity.components.has(ReproComponent.self) returns true ❌ entity.components[ReproComponent.self] returns nil Expected If has(...) returns true for a registered custom component, typed lookup should decode and return non-nil. Notes Running the same test via xcodebuild test works correctly The component is properly registered and the USDA file correctly references it This affects any project that relies on custom RealityKit components in tests — there is no swift test workaround Feedback ID: FB22099519 Environment: macOS 15.5, Xcode 16.4, Swift 6.1
1
0
1.2k
May ’26
RealityView content disappears when selecting Lock In Place on visionOS
I'm experiencing an issue where all RealityView content disappears when the user selects "Lock In Place" from the window management menu (long press on close button). "Follow Me" works correctly and this happens in Testflight builds only not reproducible when I run locally I have reproduced this with a minimal project containing nothing but a simple red cube — no custom anchors, no app state, no dependencies. Steps to Reproduce: Open an ImmersiveSpace A red cube is placed 1m in front of the user via RealityView Long press the X button on any floating window Select "Lock In Place" The cube disappears immediately Expected: Cube remains visible after window is locked Actual: Cube disappears. Note: "Follow Me" does NOT reproduce this issue. Minimal reproducible code: struct ImmersiveView: View { var body: some View { RealityView { content in let cube = ModelEntity( mesh: .generateBox(size: 0.3), materials: [SimpleMaterial(color: .red, isMetallic: false)] ) cube.setPosition(SIMD3(0, 1.5, -1), relativeTo: nil) content.add(cube) } } } Device: Apple Vision Pro visionOS version: Vision OS 26.2 (23N301) Xcode version: Version 26.3 (17C529) Is this a known issue? Is there a recommended workaround to preserve RealityView content during Lock In Place transitions? Thank you!
1
0
1.9k
Apr ’26
Real world anchors
I’m trying to build a persistent world map of my college campus using ARKit, but it’s not very reliable. Anchors don’t consistently appear in the same place across sessions. I’ve tried using image anchors, but they didn’t improve accuracy much. How can I create a stable world map for a larger area and reliably relocalize anchors? Are there better approaches or recommended resources for this?
1
0
1.5k
Apr ’26
RealityKit crashes when rendering SpriteKit scene with SKShapeNode in postProcess callback
I'm converting my game from SceneKit to RealityKit. It has a SpriteKit overlay that according to Explore advanced rendering with RealityKit 2 I can add with the code below. The code runs fine if the SKScene only contains a SKSpriteNode (see the commented out line), but when I add a SKShapeNode with a fillColor instead, the app crashes with this error: -[MTLDebugRenderCommandEncoder validateCommonDrawErrors:]:5970: failed assertion `Draw Errors Validation MTLDepthStencilDescriptor uses frontFaceStencil but MTLRenderPassDescriptor has a nil stencilAttachment texture MTLDepthStencilDescriptor uses backFaceStencil but MTLRenderPassDescriptor has a nil stencilAttachment texture ' I don't know enough about low-level graphics and stencils yet to figure out a quick solution, so I would appreciate if anyone could share an easy fix or explanation of what's wrong. Thanks! class ViewController: NSViewController { var device: MTLDevice! var renderer: SKRenderer! override func loadView() { let arView = ARView(frame: NSScreen.main!.frame) view = arView arView.renderCallbacks.prepareWithDevice = { [weak self] device in guard let self = self else { return } self.device = device renderer = SKRenderer(device: MTLCreateSystemDefaultDevice()!) let scene = SKScene() let shape = SKShapeNode(rectOf: CGSize(width: 10, height: 10)) shape.fillColor = .red scene.addChild(shape) // scene.addChild(SKSpriteNode(color: .red, size: CGSize(width: 10, height: 10))) renderer.scene = scene } arView.renderCallbacks.postProcess = { [weak self] context in guard let self = self else { return } let encoder = context.commandBuffer.makeBlitCommandEncoder() encoder?.copy(from: context.sourceColorTexture, to: context.targetColorTexture) encoder?.endEncoding() renderer.update(atTime: context.time) let descriptor = MTLRenderPassDescriptor() descriptor.colorAttachments[0].loadAction = .load descriptor.colorAttachments[0].storeAction = .store descriptor.colorAttachments[0].texture = context.targetColorTexture renderer.render(withViewport: CGRect(x: 0, y: 0, width: context.targetColorTexture.width, height: context.targetColorTexture.height), commandBuffer: context.commandBuffer, renderPassDescriptor: descriptor) } } }
9
0
3.2k
Apr ’26
RealityView attachment draw order
My visionOS 26.3 app displays a diorama-like scene in a RealityView in a mixed immersive space, about 1 meter square, with view attachments floating above the scene. Each view attachment fades out after user interaction, by animating the view's opacity. What I'm observing is that depending on the position of a view attachment relative to the scene and the camera, an unwanted cutout effect is observed (presumably because of draw order issues), as shown in the right column in the screenshots below. YouTube video link of these sequences: https://youtu.be/oTuo0okKCkc (19 seconds) My question: How does visionOS determine the view attachment draw order relative to the RealityView scene? If I better understood how the draw order is determined, I could modify my scene to ensure that the view attachments were always drawn after the scene, fixing the unwanted cutout effect. I've successfully used ModelSortGroupComponent to control the draw order of entities within the RealityView scene, but my understanding is that this approach cannot be used with view attachments. I've submitted FB22014370 about this issue. Thank you.
4
0
2.1k
Apr ’26
LowLevelInstanceData & animation
AppleOS 26 introduces LowLevelInstanceData that can reduce CPU draw calls significantly by instancing. However, I have noticed trouble with animating each individual instance. As I wanted low-level control, I'm using a custom system and LowLevelInstanceData.replace(using:) to update the transform each frame. The update closure itself is extremely efficient (Xcode Instruments reports nearly no cost). But I noticed extremely high runloop time, reach around 20ms. Time Profiler shows that the CPU is blocked by kernel.release.t6401. I think it is caused by synchronization between CPU and GPU, however, as I am already using a MTLCommandBuffer to coordinate it, I don't understand why I am still seeing large CPU time.
3
0
1.2k
Apr ’26
ParticleEmitterComponent Position Offset Issue After iOS 26.1 Update – Seeking Solutions & Workarounds
Problem Summary After upgrading to iOS 26.1 and 26.2, I'm experiencing a particle positioning bug in RealityKit where ParticleEmitterComponent particles render at an incorrect offset relative to their parent entity. This behavior does not occur on iOS 18.6.2 or earlier versions, suggesting a regression introduced in the newer OS builds. Environment Details Operating System: iOS 26.1 & iOS 26.2 Framework: RealityKit Xcode Version: 16.2 (16C5032a) Expected vs. Actual Behavior Expected: Particles should render at the position of the entity to which the ParticleEmitterComponent is attached, matching the behavior on iOS 18.6.2 and earlier. Actual: Particles appear away from their parent entity, creating a visual misalignment that breaks the intended AR experience. Steps to Reproduce Create or open an AR application with RealityKit that uses particle components Attach a ParticleEmitterComponent to an entity via a custom system Run the application on iOS 26.1 or iOS 26.2 Observe that particles render at an offset position away from the entity Minimal Code Example Here's the setup from my test case: Custom Component & System: struct SparkleComponent4: Component {} class SparkleSystem4: System { static let query = EntityQuery(where: .has(SparkleComponent4.self)) required init(scene: Scene) {} func update(context: SceneUpdateContext) { for entity in context.scene.performQuery(Self.query) { // Only add once if entity.components.has(ParticleEmitterComponent.self) { continue } var newEmitter = ParticleEmitterComponent() newEmitter.mainEmitter.color = .constant(.single(.red)) entity.components.set(newEmitter) } } } AR Setup: let material = SimpleMaterial(color: .gray, roughness: 0.15, isMetallic: true) let model = Entity() model.components.set(ModelComponent(mesh: boxMesh, materials: [material])) model.components.set(SparkleComponent4()) model.position = [0, 0.05, 0] model.name = "MyCube" let anchor = AnchorEntity(.plane(.horizontal, classification: .any, minimumBounds: [0.2, 0.2])) anchor.addChild(model) arView.scene.addAnchor(anchor) Questions for the Community Has anyone else encountered this particle positioning issue after updating to iOS 26.1/26.2? Are there known workarounds or configuration changes to ParticleEmitterComponent that restore correct positioning? Is this a confirmed bug, or could there be a change in coordinate system handling or transform inheritance that I'm missing? Additional Information I've already submitted this issue via Feedback Assistant(FB21346746)
3
1
1.9k
Apr ’26
RealityView content disappears when selecting Lock In Place on visionOS
Hi, I'm experiencing an issue where all RealityView content disappears when the user selects "Lock In Place" from the window management menu (long press on close button). "Follow Me" works correctly and this happens in Testflight builds only not reproducible when I run locally I have reproduced this with a minimal project containing nothing but a simple red cube — no custom anchors, no app state, no dependencies. Steps to Reproduce: Open an ImmersiveSpace. A red cube is placed 1m in front of the user via RealityView. Long press the X button on any floating window Select "Lock In Place". The cube disappears immediately. Expected: Cube remains visible after window is locked Actual: Cube disappears. Minimal reproducible code: var body: some View { RealityView { content in let cube = ModelEntity( mesh: .generateBox(size: 0.3), materials: [SimpleMaterial(color: .red, isMetallic: false)] ) cube.setPosition(SIMD3<Float>(0, 1.5, -1), relativeTo: nil) content.add(cube) } } } Device: Apple Vision Pro visionOS version: Vision OS 26.2 (23N301) Xcode version: Version 26.3 (17C529) Is this a known issue? Is there a recommended workaround to preserve RealityView content during Lock In Place transitions? Thank you!
5
0
2.1k
Apr ’26
RealityView camera feed not shown
I have two RealityView: ParentView and When click the button in ParentView, ChildView will be shown as full screen cover, but the camera feed in ChildView will not be shown, only black screen. If I show ChildView directly, it works with camera feed. Please help me on this issue? Thanks. import RealityKit import SwiftUI struct ParentView: View{ @State private var showIt = false var body: some View{ ZStack{ RealityView{content in content.camera = .virtual let box = ModelEntity(mesh: MeshResource.generateSphere(radius: 0.2),materials: [createSimpleMaterial(color: .red)]) content.add(box) } Button("Click here"){ showIt = true } } .fullScreenCover(isPresented: $showIt){ ChildView() .overlay( Button("Close"){ showIt = false }.padding(20), alignment: .bottomLeading ) } .ignoresSafeArea(.all) } } import ARKit import RealityKit import SwiftUI struct ChildView: View{ var body: some View{ RealityView{content in content.camera = .spatialTracking } } }
5
1
2.6k
Apr ’26
RealityKit fill the background environment
I am new to RealityKit and Metal and I am building a RealityKit app that renders a procedural LowLevelMesh road. But the left and right side of the road is a complete green terrain mesh object and it doesn't look great. What I want is to add some rocks, tall trees and dence bushes (or weed) to make it look like the player is in the woods. But when I add many of those objects then the performance drains. What is the best approach to fill background empty spaces in the scene?
3
0
1.4k
Apr ’26
Blending walk and run animations in RealityKit
Hi everybody, I have 2 separate animations run.usdz and walk.usdz animation files which are loaded perfectly in Reality Composer Pro and in the RealityKit application. I want to gradually increase the speed of my player by switching blending weight values from 0.0 (walking) to 1.0 (full speed running). let rabbit = await RabbitBuilder.loadWalkingRabbit() let runningRabbit = await RabbitBuilder.loadRunningRabbit() rabbit.scale = SIMD3(0.05, 0.05, 0.05) runningRabbit.scale = SIMD3(0.05, 0.05, 0.05) let walkAnimation = rabbit.availableAnimations let runAnimation = runningRabbit.availableAnimations RabbitWalker.walkAnim = walkAnimation.first! RabbitWalker.runAnim = runAnimation.first! guard let walk = RabbitWalker.walkAnim, let run = RabbitWalker.runAnim else { return } let blendTree = BlendTreeAnimation<JointTransforms>( BlendTreeBlendNode(sources: [ BlendTreeSourceNode(source: walk.definition, name: "walk", weight: .value(1 - weight)), BlendTreeSourceNode(source: run.definition, name: "run", weight: .value(weight)) ]), name: "rabbitLocomotion", repeatMode: .repeat, offset: TimeInterval(elapsed) ) // I have runtime error after executing this line: "Cannot add incompatible timeline type to blend tree." guard let resource = try? AnimationResource.generate(with: blendTree) else { return } entity.playAnimation(resource) static func loadWalkingRabbit() async -> Entity? { do { let scene = try await Entity(named: "Scene", in: realityKitEnvironmentBundle) guard let rabbit = await scene.findEntity(named: "RabbitWalk") else { return nil } await rabbit.removeFromParent() return rabbit } catch { return nil } } static func loadRunningRabbit() async -> Entity? { do { let scene = try await Entity(named: "Scene", in: realityKitEnvironmentBundle) guard let rabbit = await scene.findEntity(named: "RabbitRun") else { return nil } await rabbit.removeFromParent() return rabbit } catch { return nil } } But when I run this code I have this error; Cannot add incompatible timeline type to blend tree. By the way I have looked to developer's sample codes from here but I couldn't find any relevant BlendTreeAnimation sample which blends 2 animations. I would very happy if someone could direct me to a solution. Regards.
4
0
1.1k
Apr ’26
PhotogrammetrySession(input: [PhotogrammetrySample])
Please I would really appreciate a timely response to this: Providing a response and feedback to this: https://origin-devforums.apple.com/forums/thread/827311 and https://developer.apple.com/forums/thread/827043?page=1#889020022 Feedback Report: I have created a feedback report as well like recommended with this ID: FB22823733, with more elaborate images of my implementation, also see here More clarity: Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I also tried using the PhotogrammetrySession folder run but its still failed with : CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing. Please I would really appreciate a timely response to this
Replies
1
Boosts
0
Views
925
Activity
Jun ’26
Object Capture broken on iPhone 17 Pro (iOS 26.x): ObjectCaptureSession sensorFailed + PhotogrammetrySession cv3dapi 4004
The full RealityKit Object Capture stack fails device-specifically on the iPhone 17 Pro family (iPhone18,1 / iPhone18,2) under iOS 26.x. Both the live and offline paths break. The same code and images work on iPhone 12 Pro (iPhone13,3) at the same iOS version. Both ObjectCaptureSession.isSupported and PhotogrammetrySession.isSupported return true on the 17 Pro, so there's no API signal that the operation will fail — it only fails at runtime on real hardware. Defect 1 — ObjectCaptureSession.Error.sensorFailed A guided capture session fails. On iOS 26.4.2 it fails at scan start. On iOS 26.5 it reaches .capturing and renders the live session for several seconds, but auto-capture never fires; attempting a manual capture then fails. The Swift enum (.sensorFailed) hides the cause; the underlying framework log is: <<<< FigXPCUtilities >>>> signalled err=-17281 at <>:308 <<<< FigCaptureSourceRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSourceRemote.m:569) - (err=-17281) ObjectCaptureSession.takeStillImageCapture(isAutoCapture:): Failed to capture high resolution frame! ARSession <0x…>: did fail with error: Error Domain=com.apple.arkit.error Code=102 "Sensor failure." UserInfo={NSUnderlyingError=0x… {Error Domain=AVFoundationErrorDomain Code=-11819 "Cannot complete action" UserInfo={NSLocalizedRecoverySuggestion=Try again later.}} NSLocalizedFailureReason=The sensor did not provide the required data.} Capturing failed with error: objectCaptureSessionError(CoreOC.ObjectCaptureSession.Error.sensorFailed) A second, uncatchable variant aborts instead of returning the error: Abort: …/AppleCVHWA/library/VIO/HWFeatureDetection/src/HwGeneralProcessingAPI.cpp:711 : Original error reported by request utility function: kVisionHWAReturnGeneralProcessingFailed Appears to be mediaserverd resetting (−11819) underneath the live AVCaptureSession, tearing it out from under ARKit (ARError 102). Configuration-independent: tested with isOverCaptureEnabled = false and the identical chain still occurs. Defect 2 — PhotogrammetrySession reconstruction (cv3dapi 4004) Folder-mode reconstruction starts but fails after a few seconds; SfM builds no pose map: ERROR cv3dapi.pg: Internal error codes (1): 4004 // "No SfM map found" WARN cv3dapi.pg: Internal warning codes (2): 2502 3502 Output error with code = -15 requestError: CoreOC.PhotogrammetrySession.Error.processError The input is just a folder of images handed to PhotogrammetrySession in folder mode — no other configuration. The same folder reconstructs fine on the 12 Pro. Affected config Failing: iPhone 17 Pro (iPhone18,1), 17 Pro Max (iPhone18,2), iOS 26.4.2 and 26.5 Working (same code/OS): iPhone 12 Pro (iPhone13,3) Open question Does a 17-Pro image set fail PhotogrammetrySession everywhere, or only on the 17 Pro? Fails everywhere → 17 Pro images / HEIC metadata confuse SfM. Works elsewhere → the on-device reconstruction engine itself is the defect. Sample image set + sysdiagnose available on request. References cv3dapi 4004 = "No SfM map found" (683300) · -15/processError wrapper (794169) · iOS 26 Object Capture regression (810496) · 17 Pro Max sensorFailed (827216)
Replies
1
Boosts
0
Views
1k
Activity
Jun ’26
Reality View Preserves Camera Transform when toggling Virtual & Spatial Tracking modes
When switching from RealityView’s .spatialTracking camera mode to .virtual camera mode, the camera’s orientation relative to the scene is preserved permanently with no way to reset to default World-Up orientation. Since .spatialTracking’s camera mode will always have a non-default orientation, switching to .virtual camera mode ensures that the cameras’s ‘UP’ direction will never match the device display’s ‘UP’ direction as is default. This is especially noticeable when using .orbit camera controls, as the orbit’s UP direction matches the scene, not camera, and all rotation directions give unexpected results. Expected: When setting virtual camera mode after using spatialTracking camera mode, either 1. The Virtual Camera orientation returns to default (world up). Or 2. A 'content.camera.resetOrientation()' call is made available which resets the RealityView camera to default orientation. Reality: Switching from .spatialTracking -> .virtual camera mode permanently locks the .virtual camera’s orientation the final frame of the .spatialTracking camera’s rotation (relative to the RealityView content scene). One imperfect workaround is to reset / rebuild the entire RealityView after changing modes (by resetting .id() or otherwise. This is not ideal as it causes everything inside the make closure to rerun, which not only is a performance & time cost, visually incurs a flicker and can also be problematic with managing increasingly complicated views. Another imperfect alternative is to use more than one RealityView - which is not ideal as it incurs double the base ram usage, significantly increases code, and seemingly goes against the intent of being able to change the camera .virtual/.spatatialTracking mode at will. Code Sample: import SwiftUI import RealityKit struct RKSpatialVirtualToggle: View { @State var showAR: Bool = false var body: some View { RealityView { content in let cube = ModelEntity(mesh: .generateBox(size: 0.25), materials: [SimpleMaterial()]) cube.position.z = -1 content.add(cube) content.camera = showAR ? .spatialTracking : .virtual content.cameraTarget = cube } update: { content in content.camera = showAR ? .spatialTracking : .virtual } .realityViewCameraControls(.orbit) VStack{ Spacer() Button("Toggle AR"){ showAR.toggle() } .buttonStyle(.borderedProminent) } } } Xcode Version: Version 26.0 (17A324) iOS Version: iOS 26.5 (23F75) Tested on devices, iPhone 12 Pro, iPhone 15 Pro
Replies
1
Boosts
0
Views
667
Activity
Jun ’26
PhotogrammetrySession(input: [PhotogrammetrySample]) Hangs or terminates
Providing clarity and response to this: https://origin-devforums.apple.com/forums/thread/827311 and https://developer.apple.com/forums/thread/827043?page=1#889020022 Feedback Report: I have created a feedback report as well like recommended with this ID: FB22823733, with more elaborate images of my implementation, also see here More clarity: Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I also tried using the PhotogrammetrySession folder run but its still failed with : CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing.
Replies
0
Boosts
0
Views
1.2k
Activity
May ’26
`FigAudioSession(AV) err=-19224` triggered by empty Button tap on visionOS 26.5, breaking subsequent AVAudioSession configuration
Environment Device: Apple Vision Pro (real device) OS: visionOS 26.5 Xcode: 26.5 Framework: AVFAudio / AVFoundation Summary On visionOS 26.5, tapping an empty Button consistently emits the following internal warning before the action closure executes: <<<< FigAudioSession(AV) >>>> signalled err=-19224 (<>:612) After this warning is emitted, any subsequent call to configure AVAudioSession silently stops working — audio input and output become non-functional for the lifetime of the session. If the same configuration is performed without a preceding button tap (e.g., inside View.task {}), it succeeds and audio works correctly. Reproduction Due to a dependency on LiveKitWebRTC (livekit/webrtc-xcframework) for WebRTC-based Realtime API audio, we are unable to provide a full self-contained sample project. However, the AVAudioSession configuration code involved is as follows: static func configureAudioSession() { #if !os(macOS) do { let audioSession = AVAudioSession.sharedInstance() #if os(tvOS) try audioSession.setCategory(.playAndRecord, options: []) #else try audioSession.setCategory(.playAndRecord, options: [.defaultToSpeaker]) #endif try audioSession.setMode(.videoChat) try audioSession.setActive(true, options: .notifyOthersOnDeactivation) } catch { print("Failed to configure AVAudioSession: \(error)") } #endif } Scenario A — Button tap (fails): Button("Start") { configureAudioSession() // FigAudioSession err=-19224 appears; audio stops working } Scenario B — View.task (succeeds): .task { configureAudioSession() // No warning; audio works correctly } The only difference is whether a user gesture (Button tap) precedes the call. Observed Behavior Tapping any Button on visionOS 26.5 causes FigAudioSession(AV) err=-19224 to be signalled at <>:612, even before the action closure runs. After this warning, AVAudioSession configuration appears to have no effect — setActive(true) does not throw, but audio appears to stop functioning. Configuring the session prior to any button interaction (e.g., in View.task {}) works correctly. Expected Behavior A Button tap should not implicitly interfere with the audio session state. AVAudioSession configuration should succeed regardless of the UI event context that triggers it. Questions What does FigAudioSession(AV) err=-19224 mean? Does it correspond to a documented AVAudioSession.ErrorCode? Why does a Button tap trigger a FigAudioSession signal on visionOS? Is the system performing implicit audio session management when detecting user interaction? Is there a recommended pattern for configuring AVAudioSession in response to a user gesture on visionOS? Our current workaround (View.task {}) is not suitable for on-demand audio start triggered by the user. Is err=-19224 causally responsible for the subsequent audio issue? Since setActive(true) does not throw after the warning, it is unclear whether this signal is the direct cause of the apparent audio failure or a symptom of a deeper conflict. Are there UI components or APIs on visionOS that do not trigger this signal, while still being user-interaction driven? Additional Notes Reproducible only on physical Apple Vision Pro hardware; not observed in Simulator. AirPlay mirroring is not in use during testing. No other apps are playing audio in the background at the time of reproduction. We use LiveKitWebRTC (livekit/webrtc-xcframework, revision 94ce1c9) for WebRTC audio. However, the FigAudioSession warning appears independently of the WebRTC layer — it is emitted on Button tap even before configureAudioSession() is called. We have verified that calling configureAudioSession() before performHandshake() (i.e., before WebRTC initializes its audio pipeline) does not resolve the issue when a Button tap precedes the call.
Replies
0
Boosts
0
Views
529
Activity
May ’26
PhotogrammetrySession(input: [PhotogrammetrySample]) Hangs or terminates
Providing a response and feedback to this: https://origin-devforums.apple.com/forums/thread/827311 and https://developer.apple.com/forums/thread/827043?page=1#889020022 I have a created a feedback/bug report with ID FB22823733 Feedback Report: I have created a feedback report as well like recommended with this ID: FB22823733, with more elaborate images of my implementation, also see here More clarity: Yes, this is for an iOS app(iOS 18+) but testing and debugging on Xcode(MacOS 15.7.7, M4 24GB), which traps or hangs on getting to [PhotogrammetrySample] even while using the lazy sequence and the contentsOf as specified in your docs. I also tried using the PhotogrammetrySession folder run but its still failed with : CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing.
Replies
0
Boosts
0
Views
1.7k
Activity
May ’26
RealityView AR - anchored to the screen not the floor
This started out as a plea for help, but in preparing this post I discovered the root cause. I'm posting it as a lesson learned in hopes it will help someone. I've spent a good chunk of March trying to get AR-mode working again in my unreleased game. I had it working with SceneKit and ARView 5 years ago, but since 2024 I've been converting the game to use RealityKit and RealityView on iOS, macOS, visionOS, and tvOS. I've been having no joy getting AR mode to work on iOS. I get the pass-through device video but the game content isn't anchored to the floor but rather anchored to the screen. I made a simple project with just a simple shape in the middle of a RealityView and an overlay with a SwiftUI toggle to go in and out of AR-mode. At first, my simple project worked, and I couldn't figure out what was different in the logic. Both projects used the same logic: func transitionToXR(_ content: inout RealityViewCameraContent) { content.remove(gameBoard.rootEntity) content.add(xrAnchor) content.camera = .spatialTracking Self.anchorStateChangedSubscription = content.subscribe(to: SceneEvents.AnchoredStateChanged.self) { event in if event.anchor == xrAnchor, event.isAnchored { xrAnchor.addChild(gameBoard.rootEntity) } } } Then I made an alternate version of my view, and reproduced the same "anchored to the screen not the floor" issue. I compared the code side-by-side and finally saw the difference! The one that didn't work, like my game, had a property 'cameraEntity' which is initialized with PerspectiveCamera(), position and look-at configured, then added as a child of the root entity. So, the simple fix was to remove 'cameraEntity' from the root entity before adding it to the detected AnchorEntity when going into AR-mode. Then when leaving AR-mode, I add back 'cameraEntity' as a child of the root entity and configure it again. So the lesson learned is: make sure there isn't a PerspectiveCamera in the tree of Entities added to an AnchorEntity with a .spatialTracking content camera. Apple: let me know if you think this is a bug or if I was being dumb. If a bug, I can use Feedback Assistant to report this. If I was being dumb, it wouldn't be the first time. :-)
Replies
5
Boosts
0
Views
1.2k
Activity
May ’26
VirtualEnvironmentProbeComponent VS ImageBasedLightComponent
Hi. I want to know what's the difference between VirtualEnvironmentProbeComponent and ImageBasedLightComponent? It seems they both can achieve the same light and reflection effect of environment.
Replies
1
Boosts
0
Views
1k
Activity
May ’26
RealityView Camera Target Error when set while Orbiting
When interacting with RealityView’s realityViewCameraControls .orbit and setting a new RealityViewCameraContent .cameraTarget, the resulting camera target and camera orbit is incorrect. This can be demonstrated where one finger is orbiting the RealityView, and another pushes a button which changes the camera target. Instead of the camera facing the new target, some point in the scene is the new effective camera target and orbit point. This only occurs when an orbit interaction is currently taking place. If you stop interacting with the orbit, change target, then start orbit interacting again, everything works as expected. Though this example uses two-touches, any change of the camera target has this conflict with orbit interaction. This means interacting with orbit will result in the wrong camera view which is unexpected for users and difficult to reconcile or detect, for developers. Expected: Interacting (orbiting) the scene while setting a new camera target with the buttons on screen (at the same time), the camera’s new target shows centred in view the orbit revolves the new target and continues to match my gestures. Reality: Interacting (orbiting) the scene while setting a new camera target with the buttons on screen (at the same time), the camera’s new target is not centred in view, and camera is now orbiting an unexpected point in the scene, that is not my expected target. One imperfect workaround is to force a rebuild of the view after setting a new cameraTarget. This sets all targets correctly but results in a flicker, loss of orbit controls until re-touch and ultimately is a poor user experience, but is better than the wrong target being shown unexpectedly. Code Sample: import SwiftUI import RealityKit struct RKOribtTarget: View { @State private var target: Int = 0 @State private var rcContent: RealityViewCameraContent? @State private var rkID: UUID = UUID() let root = Entity() let center = ModelEntity(mesh: .generateSphere(radius: 0.05), materials: [UnlitMaterial(color: UIColor(.gray.opacity(0.5)))]) let red = ModelEntity(mesh: .generateBox(size: 0.1), materials: [SimpleMaterial(color: .red, isMetallic: false)]) let blue = ModelEntity(mesh: .generateBox(size: 0.1), materials: [SimpleMaterial(color: .blue, isMetallic: false)]) let green = ModelEntity(mesh: .generateBox(size: 0.1), materials: [SimpleMaterial(color: .green, isMetallic: false)]) var body: some View { VStack{ RealityView { content in red.position.x = 0.5 blue.position.z = 0.5 green.position.y = 0.5 center.position = .init(repeating: 0.25) content.cameraTarget = target == 0 ? root : blue root.addChild(red) root.addChild(blue) root.addChild(green) root.addChild(center) content.add(root) } update: { content in switch target{ case 0: content.cameraTarget = root case 1: content.cameraTarget = blue case 2: content.cameraTarget = red case 3: content.cameraTarget = green default: content.cameraTarget = root } } .id(rkID) .realityViewCameraControls(.orbit) VStack{ Text("Target") Button("Default") { target = 0 // Force rebuilding view resets orbit target and rotation // But shows a flicker, interaction requires touch reset // Not an ideal workaround // rkID = UUID() } .buttonStyle(.bordered) Button("Blue") { target = 1 // rkID = UUID() } .buttonStyle(.bordered) .tint(.blue) Button("Red") { target = 2 // rkID = UUID() } .buttonStyle(.bordered) .tint(.red) Button("Green") { target = 3 // rkID = UUID() } .buttonStyle(.bordered) .tint(.green) } } } } Xcode Version: Version 26.0 (17A324) iOS Version: iOS 26.5 (23F75) Tested on devices, iPhone 12 Pro, iPhone 15 Pro
Replies
2
Boosts
0
Views
1.1k
Activity
May ’26
RealityKit custom component: `has()` returns `true` but typed subscript returns `nil` in SwiftPM test runner
swift test (SwiftPM CLI) fails to decode RealityKit custom components from USD files, even though entity.components.has(MyComponent.self) returns true. Typed access via entity.components[MyComponent.self] returns nil. This forces projects that use RealityKit custom components to use xcodebuild test exclusively. Minimal repro: github.com/mesqueeb/swiftpm-realitykit-custom-component-repro Repro steps git clone https://github.com/mesqueeb/swiftpm-realitykit-custom-component-repro cd swiftpm-realitykit-custom-component-repro swift test --filter componentsPresentButNotDecodableInSwiftTest Observed ✅ entity.components.has(ReproComponent.self) returns true ❌ entity.components[ReproComponent.self] returns nil Expected If has(...) returns true for a registered custom component, typed lookup should decode and return non-nil. Notes Running the same test via xcodebuild test works correctly The component is properly registered and the USDA file correctly references it This affects any project that relies on custom RealityKit components in tests — there is no swift test workaround Feedback ID: FB22099519 Environment: macOS 15.5, Xcode 16.4, Swift 6.1
Replies
1
Boosts
0
Views
1.2k
Activity
May ’26
RealityView content disappears when selecting Lock In Place on visionOS
I'm experiencing an issue where all RealityView content disappears when the user selects "Lock In Place" from the window management menu (long press on close button). "Follow Me" works correctly and this happens in Testflight builds only not reproducible when I run locally I have reproduced this with a minimal project containing nothing but a simple red cube — no custom anchors, no app state, no dependencies. Steps to Reproduce: Open an ImmersiveSpace A red cube is placed 1m in front of the user via RealityView Long press the X button on any floating window Select "Lock In Place" The cube disappears immediately Expected: Cube remains visible after window is locked Actual: Cube disappears. Note: "Follow Me" does NOT reproduce this issue. Minimal reproducible code: struct ImmersiveView: View { var body: some View { RealityView { content in let cube = ModelEntity( mesh: .generateBox(size: 0.3), materials: [SimpleMaterial(color: .red, isMetallic: false)] ) cube.setPosition(SIMD3(0, 1.5, -1), relativeTo: nil) content.add(cube) } } } Device: Apple Vision Pro visionOS version: Vision OS 26.2 (23N301) Xcode version: Version 26.3 (17C529) Is this a known issue? Is there a recommended workaround to preserve RealityView content during Lock In Place transitions? Thank you!
Replies
1
Boosts
0
Views
1.9k
Activity
Apr ’26
Real world anchors
I’m trying to build a persistent world map of my college campus using ARKit, but it’s not very reliable. Anchors don’t consistently appear in the same place across sessions. I’ve tried using image anchors, but they didn’t improve accuracy much. How can I create a stable world map for a larger area and reliably relocalize anchors? Are there better approaches or recommended resources for this?
Replies
1
Boosts
0
Views
1.5k
Activity
Apr ’26
RealityKit crashes when rendering SpriteKit scene with SKShapeNode in postProcess callback
I'm converting my game from SceneKit to RealityKit. It has a SpriteKit overlay that according to Explore advanced rendering with RealityKit 2 I can add with the code below. The code runs fine if the SKScene only contains a SKSpriteNode (see the commented out line), but when I add a SKShapeNode with a fillColor instead, the app crashes with this error: -[MTLDebugRenderCommandEncoder validateCommonDrawErrors:]:5970: failed assertion `Draw Errors Validation MTLDepthStencilDescriptor uses frontFaceStencil but MTLRenderPassDescriptor has a nil stencilAttachment texture MTLDepthStencilDescriptor uses backFaceStencil but MTLRenderPassDescriptor has a nil stencilAttachment texture ' I don't know enough about low-level graphics and stencils yet to figure out a quick solution, so I would appreciate if anyone could share an easy fix or explanation of what's wrong. Thanks! class ViewController: NSViewController { var device: MTLDevice! var renderer: SKRenderer! override func loadView() { let arView = ARView(frame: NSScreen.main!.frame) view = arView arView.renderCallbacks.prepareWithDevice = { [weak self] device in guard let self = self else { return } self.device = device renderer = SKRenderer(device: MTLCreateSystemDefaultDevice()!) let scene = SKScene() let shape = SKShapeNode(rectOf: CGSize(width: 10, height: 10)) shape.fillColor = .red scene.addChild(shape) // scene.addChild(SKSpriteNode(color: .red, size: CGSize(width: 10, height: 10))) renderer.scene = scene } arView.renderCallbacks.postProcess = { [weak self] context in guard let self = self else { return } let encoder = context.commandBuffer.makeBlitCommandEncoder() encoder?.copy(from: context.sourceColorTexture, to: context.targetColorTexture) encoder?.endEncoding() renderer.update(atTime: context.time) let descriptor = MTLRenderPassDescriptor() descriptor.colorAttachments[0].loadAction = .load descriptor.colorAttachments[0].storeAction = .store descriptor.colorAttachments[0].texture = context.targetColorTexture renderer.render(withViewport: CGRect(x: 0, y: 0, width: context.targetColorTexture.width, height: context.targetColorTexture.height), commandBuffer: context.commandBuffer, renderPassDescriptor: descriptor) } } }
Replies
9
Boosts
0
Views
3.2k
Activity
Apr ’26
RealityView attachment draw order
My visionOS 26.3 app displays a diorama-like scene in a RealityView in a mixed immersive space, about 1 meter square, with view attachments floating above the scene. Each view attachment fades out after user interaction, by animating the view's opacity. What I'm observing is that depending on the position of a view attachment relative to the scene and the camera, an unwanted cutout effect is observed (presumably because of draw order issues), as shown in the right column in the screenshots below. YouTube video link of these sequences: https://youtu.be/oTuo0okKCkc (19 seconds) My question: How does visionOS determine the view attachment draw order relative to the RealityView scene? If I better understood how the draw order is determined, I could modify my scene to ensure that the view attachments were always drawn after the scene, fixing the unwanted cutout effect. I've successfully used ModelSortGroupComponent to control the draw order of entities within the RealityView scene, but my understanding is that this approach cannot be used with view attachments. I've submitted FB22014370 about this issue. Thank you.
Replies
4
Boosts
0
Views
2.1k
Activity
Apr ’26
LowLevelInstanceData & animation
AppleOS 26 introduces LowLevelInstanceData that can reduce CPU draw calls significantly by instancing. However, I have noticed trouble with animating each individual instance. As I wanted low-level control, I'm using a custom system and LowLevelInstanceData.replace(using:) to update the transform each frame. The update closure itself is extremely efficient (Xcode Instruments reports nearly no cost). But I noticed extremely high runloop time, reach around 20ms. Time Profiler shows that the CPU is blocked by kernel.release.t6401. I think it is caused by synchronization between CPU and GPU, however, as I am already using a MTLCommandBuffer to coordinate it, I don't understand why I am still seeing large CPU time.
Replies
3
Boosts
0
Views
1.2k
Activity
Apr ’26
ParticleEmitterComponent Position Offset Issue After iOS 26.1 Update – Seeking Solutions & Workarounds
Problem Summary After upgrading to iOS 26.1 and 26.2, I'm experiencing a particle positioning bug in RealityKit where ParticleEmitterComponent particles render at an incorrect offset relative to their parent entity. This behavior does not occur on iOS 18.6.2 or earlier versions, suggesting a regression introduced in the newer OS builds. Environment Details Operating System: iOS 26.1 & iOS 26.2 Framework: RealityKit Xcode Version: 16.2 (16C5032a) Expected vs. Actual Behavior Expected: Particles should render at the position of the entity to which the ParticleEmitterComponent is attached, matching the behavior on iOS 18.6.2 and earlier. Actual: Particles appear away from their parent entity, creating a visual misalignment that breaks the intended AR experience. Steps to Reproduce Create or open an AR application with RealityKit that uses particle components Attach a ParticleEmitterComponent to an entity via a custom system Run the application on iOS 26.1 or iOS 26.2 Observe that particles render at an offset position away from the entity Minimal Code Example Here's the setup from my test case: Custom Component & System: struct SparkleComponent4: Component {} class SparkleSystem4: System { static let query = EntityQuery(where: .has(SparkleComponent4.self)) required init(scene: Scene) {} func update(context: SceneUpdateContext) { for entity in context.scene.performQuery(Self.query) { // Only add once if entity.components.has(ParticleEmitterComponent.self) { continue } var newEmitter = ParticleEmitterComponent() newEmitter.mainEmitter.color = .constant(.single(.red)) entity.components.set(newEmitter) } } } AR Setup: let material = SimpleMaterial(color: .gray, roughness: 0.15, isMetallic: true) let model = Entity() model.components.set(ModelComponent(mesh: boxMesh, materials: [material])) model.components.set(SparkleComponent4()) model.position = [0, 0.05, 0] model.name = "MyCube" let anchor = AnchorEntity(.plane(.horizontal, classification: .any, minimumBounds: [0.2, 0.2])) anchor.addChild(model) arView.scene.addAnchor(anchor) Questions for the Community Has anyone else encountered this particle positioning issue after updating to iOS 26.1/26.2? Are there known workarounds or configuration changes to ParticleEmitterComponent that restore correct positioning? Is this a confirmed bug, or could there be a change in coordinate system handling or transform inheritance that I'm missing? Additional Information I've already submitted this issue via Feedback Assistant(FB21346746)
Replies
3
Boosts
1
Views
1.9k
Activity
Apr ’26
RealityView content disappears when selecting Lock In Place on visionOS
Hi, I'm experiencing an issue where all RealityView content disappears when the user selects "Lock In Place" from the window management menu (long press on close button). "Follow Me" works correctly and this happens in Testflight builds only not reproducible when I run locally I have reproduced this with a minimal project containing nothing but a simple red cube — no custom anchors, no app state, no dependencies. Steps to Reproduce: Open an ImmersiveSpace. A red cube is placed 1m in front of the user via RealityView. Long press the X button on any floating window Select "Lock In Place". The cube disappears immediately. Expected: Cube remains visible after window is locked Actual: Cube disappears. Minimal reproducible code: var body: some View { RealityView { content in let cube = ModelEntity( mesh: .generateBox(size: 0.3), materials: [SimpleMaterial(color: .red, isMetallic: false)] ) cube.setPosition(SIMD3<Float>(0, 1.5, -1), relativeTo: nil) content.add(cube) } } } Device: Apple Vision Pro visionOS version: Vision OS 26.2 (23N301) Xcode version: Version 26.3 (17C529) Is this a known issue? Is there a recommended workaround to preserve RealityView content during Lock In Place transitions? Thank you!
Replies
5
Boosts
0
Views
2.1k
Activity
Apr ’26
RealityView camera feed not shown
I have two RealityView: ParentView and When click the button in ParentView, ChildView will be shown as full screen cover, but the camera feed in ChildView will not be shown, only black screen. If I show ChildView directly, it works with camera feed. Please help me on this issue? Thanks. import RealityKit import SwiftUI struct ParentView: View{ @State private var showIt = false var body: some View{ ZStack{ RealityView{content in content.camera = .virtual let box = ModelEntity(mesh: MeshResource.generateSphere(radius: 0.2),materials: [createSimpleMaterial(color: .red)]) content.add(box) } Button("Click here"){ showIt = true } } .fullScreenCover(isPresented: $showIt){ ChildView() .overlay( Button("Close"){ showIt = false }.padding(20), alignment: .bottomLeading ) } .ignoresSafeArea(.all) } } import ARKit import RealityKit import SwiftUI struct ChildView: View{ var body: some View{ RealityView{content in content.camera = .spatialTracking } } }
Replies
5
Boosts
1
Views
2.6k
Activity
Apr ’26
RealityKit fill the background environment
I am new to RealityKit and Metal and I am building a RealityKit app that renders a procedural LowLevelMesh road. But the left and right side of the road is a complete green terrain mesh object and it doesn't look great. What I want is to add some rocks, tall trees and dence bushes (or weed) to make it look like the player is in the woods. But when I add many of those objects then the performance drains. What is the best approach to fill background empty spaces in the scene?
Replies
3
Boosts
0
Views
1.4k
Activity
Apr ’26
Blending walk and run animations in RealityKit
Hi everybody, I have 2 separate animations run.usdz and walk.usdz animation files which are loaded perfectly in Reality Composer Pro and in the RealityKit application. I want to gradually increase the speed of my player by switching blending weight values from 0.0 (walking) to 1.0 (full speed running). let rabbit = await RabbitBuilder.loadWalkingRabbit() let runningRabbit = await RabbitBuilder.loadRunningRabbit() rabbit.scale = SIMD3(0.05, 0.05, 0.05) runningRabbit.scale = SIMD3(0.05, 0.05, 0.05) let walkAnimation = rabbit.availableAnimations let runAnimation = runningRabbit.availableAnimations RabbitWalker.walkAnim = walkAnimation.first! RabbitWalker.runAnim = runAnimation.first! guard let walk = RabbitWalker.walkAnim, let run = RabbitWalker.runAnim else { return } let blendTree = BlendTreeAnimation<JointTransforms>( BlendTreeBlendNode(sources: [ BlendTreeSourceNode(source: walk.definition, name: "walk", weight: .value(1 - weight)), BlendTreeSourceNode(source: run.definition, name: "run", weight: .value(weight)) ]), name: "rabbitLocomotion", repeatMode: .repeat, offset: TimeInterval(elapsed) ) // I have runtime error after executing this line: "Cannot add incompatible timeline type to blend tree." guard let resource = try? AnimationResource.generate(with: blendTree) else { return } entity.playAnimation(resource) static func loadWalkingRabbit() async -> Entity? { do { let scene = try await Entity(named: "Scene", in: realityKitEnvironmentBundle) guard let rabbit = await scene.findEntity(named: "RabbitWalk") else { return nil } await rabbit.removeFromParent() return rabbit } catch { return nil } } static func loadRunningRabbit() async -> Entity? { do { let scene = try await Entity(named: "Scene", in: realityKitEnvironmentBundle) guard let rabbit = await scene.findEntity(named: "RabbitRun") else { return nil } await rabbit.removeFromParent() return rabbit } catch { return nil } } But when I run this code I have this error; Cannot add incompatible timeline type to blend tree. By the way I have looked to developer's sample codes from here but I couldn't find any relevant BlendTreeAnimation sample which blends 2 animations. I would very happy if someone could direct me to a solution. Regards.
Replies
4
Boosts
0
Views
1.1k
Activity
Apr ’26