RealityKit

RSS for tag

Simulate and render 3D content for use in your augmented reality apps using RealityKit.

Posts under RealityKit tag

200 Posts

Post

Replies

Boosts

Views

Activity

Best practice for connecting Mac-side USDZ preparation with exact-asset Vision Pro verification?
I’m developing two complementary developer tools: Reality Prep Pro on macOS and Reality Prep Preview on visionOS. The workflow I’m trying to solve is: Mac: prepare and validate a USDZ Vision Pro: verify that exact USDZ on-device Mac: bring the device evidence back and associate it with the exact asset revision I currently use the USDZ’s SHA-256 as the authoritative identity. If the asset bytes later change, the previous Vision Pro evidence is treated as stale and a re-test is recommended. Existing USDZ→USDZ preparation is package-preserving, so non-texture payloads such as animation/rigging data are not unnecessarily re-authored. I’d be interested in hearing from other RealityKit / Reality Composer Pro developers: Is this similar to how you currently handle asset provenance between desktop preparation and real-device testing? Are there any Apple frameworks or established visionOS practices you would recommend for strengthening this kind of revision-to-device-evidence workflow?
0
0
664
Sep ’26
Physical space lighting (SurroundingsLight) never lands on sloped ceiling surfaces that are present in the scene reconstruction mesh
Summary A SpotLightComponent with SpotLightComponent.SurroundingsLight() (and a ProjectiveTexture) lights the flat part of my ceiling and the walls, but never the two sloped ceiling surfaces of a converted attic room, at any angle, intensity, attenuation radius or falloff. The lit area stops in a hard straight line exactly where the flat ceiling meets each slope, even though the beam clearly extends past that line. The sloped surfaces are fully present in SceneReconstructionProvider's mesh, and their faces point into the room. So this doesn't look like a scene understanding coverage problem. It looks as if physical space lighting only considers surfaces classified as horizontal ceiling, wall or floor, and skips slanted ones. Steps to reproduce Open a mixed immersive space in a room with a sloped ceiling (a converted attic, or any vaulted ceiling). Add an entity with SpotLightComponent(color: .white, intensity: 40_000, innerAngleInDegrees: 44.5, outerAngleInDegrees: 45, attenuationRadius: 10), SpotLightComponent.SurroundingsLight(), and a SpotLightComponent.ProjectiveTexture carrying a labelled grid texture so the beam's extent is visible. Place it near the floor under the wearer and aim it straight up. Run SceneReconstructionProvider alongside and draw each MeshAnchor as a wireframe (or as a solid material with faceCulling = .back) to confirm the mesh covers the slopes and faces inward. Expected The projected pattern conforms to every surface in the scene understanding mesh inside the cone, including the sloped ceiling, as it does for the flat ceiling and walls. Actual The pattern appears only on the flat ceiling strip and on the walls. Both slopes stay completely dark, with a hard straight cut at the crease. Tilting the light to aim directly at a slope, raising intensity to 200,000, and changing attenuation radius and falloff make no difference. The wireframe shows continuous mesh across the crease and onto the slopes; the back-face-culled solid view shows the slope faces pointing into the room. Screenshots attached: (1) the projected grid stopping at the crease, (2) the reconstruction wireframe covering the slopes, (3) the solid front-faces-only mesh showing the slopes are front-facing. (4) an additional image of the projected grid with both ceiling and walls in view Code let spot = Entity() spot.components.set(SpotLightComponent.SurroundingsLight()) var light = SpotLightComponent(color: .white, intensity: 40_000, innerAngleInDegrees: 44.5, outerAngleInDegrees: 45, attenuationRadius: 10) light.attenuationFalloffExponent = 0.5 spot.components.set(light) spot.components.set(SpotLightComponent.ProjectiveTexture(texture: gridTexture)) spot.look(at: from + SIMD3(0, 1, 0), from: from, upVector: SIMD3(0, 0, -1), relativeTo: nil) root.addChild(spot) Questions Is physical space lighting limited to horizontally and vertically classified surfaces by design? If so, could this be documented on SurroundingsLight, and is there a way to include slanted surfaces? If not by design, is there anything an app can do to have the light use the full reconstruction mesh?
2
0
1.3k
Sep ’26
PresentationComponent loses gaze targeting or dismisses in a volumetric window
On visionOS 26.6 and 27 latest beta: PresentationComponent in a volumetric WindowGroup, when looking at the volume from a specific lower/back region create two bugs: buttons inside the presentation have their gaze obstructed by hand presence, no button shows any hover highlight hand movement in that specific area auto dismisses the presentation The focused public repro uses a thin ModelEntity with InputTargetComponent and ordinary SwiftUI buttons. Its README includes two clearer videos: https://github.com/mesqueeb/PresentationAutoCloseInBackOfVolume 1. Gaze obstructed on buttons when hand is at specific location (see "Test Action" has no hover highlight, even though i look at it) 2. Hover state restored on buttons when hand is gone (see "Test Action" has hover highlight) 3. Presentation auto closed when hand comes back The weirdest part: This only happens at a specific location, when walking to the back of the volume, then turning around and interacting with the volume from the lower right back side. Any workaround or fix guidance would be appreciated. Feedback Assistant: FB24569418.
2
0
1.6k
Sep ’26
Reality Composer Pro is freezing/not working
Hi all! I am struggling to make any app because reality composer pro freezes when I click on create new project, as well as it does not appear in Xcode at all. I have tried deleting and redownloading all versions of both Reality Composer Pro & Xcode, as well as restarting my computer. Kindly advise what else I can try. Thank you!
0
0
407
Aug ’26
OpacityComponent does not blend a GaussianSplatComponent on iOS/macOS, behaves correct on visionOS
I have a RealityKit scene with one container entity and two children: an entity carrying a GaussianSplatComponent, and a plain Entity with a ModelComponent (an unlit white cube). A single OpacityComponent goes on the container, so one value covers both children in the same frame. container.components.set(OpacityComponent(opacity: value)) On visionOS 27 this does exactly what I expect: both children fade together, continuously, across the whole range. On iOS 27 and Mac Catalyst 27 the cube fades continuously but the splat does not. It renders at full opacity for every value above 0 and disappears at exactly 0. At 0.01 the cube beside it is already almost invisible while the splat is still drawing at full strength, so entity opacity behaves as a visibility switch on a splat rather than as a blend factor. Things I have already ruled out: Not a propagation issue. Moving the OpacityComponent off the container and onto the splat's own entity (and the cube's own entity, to keep the A/B) changes nothing. Not the container failing to receive the value. The cube under the same component, in the same frame, fades correctly. Not the splat data. Per-splat alpha through GaussianSplatResource's opacity buffer is honoured, and projectionMode, sortingMode, scaleActivation and opacityActivation all take effect. Not a difference in scene construction per platform. The same file builds the container on all three platforms in my sample. Confirmed on: iOS 27.0 (24A5418b), iPhone 16 Pro — reproduces macOS 27.0 (26A5416b), Mac Catalyst — reproduces visionOS 27.0 (24M5326g), Apple Vision Pro — correct Filed as FB24431214 with a sample project that builds for all three from one target. Would be great to see this fixed!
1
0
659
Aug ’26
PhotogrammetrySession(input: [PhotogrammetrySample]) Hangs or terminates
Xcode hangs when I call PhotogrammetrySession(input: [PhotogrammetrySample]) with objectMask set and traps on some devices, see the attached screenshot, it gets to the function and hangs. Even the folder reconstruction also doesn't complete as it can't find alignment and displays the CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. In this case it failed while object masking was ON, so RealityKit could not find enough consistent feature tracks inside the masked pixels across the image set. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing. I will appreciate a timely response and willing to provide more clarity and informations, thank you so much for your understanding
4
0
2.9k
Aug ’26
Immersive space closes if immersive environment was fully open
If the immersive environment is fully open, then an immersive space is closed right after attempting to open it. The issue specifically impacts a mixed immersive space which replaces the immersive environment while the immersive environment is at 100%. This issue does not occur if immersive environment is any less than 100%, or if the immersionStyle is .full, or if immersiveEnvirionmentBehavior is .coexist. This can be reproduced in the template for an immersive visionOS app by removing the line .immersionStyle(selection: .constant(.full), in: .full) Then run the app and open the immersive space, and the space then closes after attempting to open it. I’m using visionOS 27.0 beta 5 and Xcode 27.0 beta 5. I've submitted feedback as FB24353599 Video of the behavior: https://youtu.be/wafDXRZVyog
1
0
1.1k
Aug ’26
ViewAttachmentComponent Resolution Low After Moving Into Frame
If a ViewAttachmentComponent moves into frame, it is low resolution until something changes the view while it is in frame. Video demonstrating the behavior: https://youtu.be/KXEFFiAnv1s I am on visionOS 27 beta 4. This did not occur when I was on visionOS 26.5. Also using Xcode 27.0 beta 4 and macOS 27.0 beta 4. To reproduce, have a ViewAttachmentComponent in an immersive space, look away, then look back, and it'll be low resolution. Anything which would change the view while it's in frame will then cause it to update in full resolution. Screenshot of low-resolution view after it moves back into frame from being out of frame: Screenshot after updating the view, making it high-resolution again: I've submitted feedback as FB24116473.
2
0
1.4k
Aug ’26
A ClothGrabComponent movement problem
I'm building a Wacom tablet-driven cloth editing rig in visionOS: a persistent grab sphere (one Entity with ClothGrabComponent, volume mode) parented to the simulation root, toggled with isGrabbing on each pen-down. A single grab behaves correctly — the cloth follows the sphere, nothing else moves. On the second grab at a different position (same cloth, no rebuild), the component keeps dragging the particles from the FIRST grab to the new sphere position — its internal grab coordinates are not updated by the new activation. The vertex that was just bound is pushed away from the sphere at the same time. Instrumented demo is (0.5 × 0.5 m grid, 289 vertices, four corners pinned, gravity = 0, sphere radius 0.06, falloff = .disabled), grab A at vertex #294 (−0.118, +0.118), release, then grab B at vertex #105 (+0.118, −0.118) — 0.335 m apart: t = 345.9 GRAB_B starts: previous #294 disp = 0.000 dist-to-ball = 0.335 current #105 disp = 0.000 dist-to-ball = 0.000 t = 346.4 (+0.5 s): previous #294 disp = 0.335 dist-to-ball = 0.000 ← dragged to the NEW ball current #105 disp = 0.191 dist-to-ball = 0.191 ← pushed AWAY from the ball t = 347.0 … 349.6 previous #294 pinned at ball (dist 0.000), #105 held away at 0.16–0.19 t = 351.9 after release: #294 back to rest (0.012), cloth flat again The 0.335 m displacement of #294 happens within half a second and equals exactly the distance from #294's rest position to the new sphere — the previous grab's particle set is being pulled toward the new sphere location, as if the component re-applied the old grab selection with the new transform. this can repeat with identical numbers. The docs for isGrabbing only say "Indicates whether particles are currently being grabbed" — they don't describe what happens on a false→true transition after the entity has moved, which is the case the official sample never demonstrates. The setup is the one shown in the official sample — a persistent entity carrying the grab component, isGrabbing toggled per interaction: // makeCloth — once let dragBall = makeBall(radius: 0.015, parent: simRoot) var grab = ClothGrabComponent(mode: .volume(shape: .sphere(ClothSphereShape(radius: 0.06)))) grab.falloff = .disabled dragBall.components.set(grab) // grab: move ball to new vertex, activate dragBall.position = body.convert(position: vertexPos, to: simRoot) var g = dragBall.components[ClothGrabComponent.self]! g.isGrabbing = true dragBall.components.set(g) // release: deactivate only var g = dragBall.components[ClothGrabComponent.self]! g.isGrabbing = false dragBall.components.set(g) some tries: Remove and re-add ClothGrabComponent on each grab — the simulator crashes within 1–2 frames with Assertion failed Rebuild the whole ClothBodyComponent between grabs Environment: Xcode 27 beta (build 24M5316i), xrOS 27.0 SDK, simulator runtime com.apple.CoreSimulator.SimRuntime.xrOS-27-0 (avp1). All Cloth* APIs are Beta on visionOS 27
1
0
1.1k
Aug ’26
RealityKit portal crossing leaves a hard PBR lighting seam despite environmentLightingWeight being 0
I’m seeing a device-only lighting discontinuity on PBR entities that intersect a RealityKit portal plane in visionOS 27. A sharp brightness boundary appears exactly where the portal plane cuts through the entity. The portion outside the portal receives a different physical environment-probe contribution from the portion inside. The behavior is reproducible with two independent cases: A completely static blue sphere positioned across the portal plane. An orange sphere moved through the portal using ManipulationComponent and the standard visionOS pinch gesture. Both spheres use: var material = PhysicallyBasedMaterial() material.baseColor = .init(tint: .orange) material.metallic = 0 material.roughness = 1 Each sphere has the crossing and lighting components attached directly: sphere.components.set(PortalCrossingComponent()) sphere.components.set( ImageBasedLightReceiverComponent( imageBasedLight: portalImageBasedLight ) ) var lightingConfiguration = EnvironmentLightingConfigurationComponent() lightingConfiguration.environmentLightingWeight = 0 sphere.components.set(lightingConfiguration) The portal is created using the visionOS 27 factory API: let portal = PortalComponent.makePortal( surfaceStyle: .init(width: 1.2, height: 0.8), boundaryStyle: .infinitePlane(), boundaryMode: .clippingAndCrossing ) if var component = portal.portalEntity.components[PortalComponent.self] { component.lightingBlendDistance = 0.2 portal.portalEntity.components.set(component) } For the moving sphere, environmentLightingWeight stays at 0 for the entire interval in which any part of the sphere intersects the portal. The realtime environment-probe contribution only begins fading from 0 to 1 after the complete sphere has cleared the plane. However, a spatially sharp edge remains: In a bright physical environment, the outside portion is brighter. In a dark physical environment, the outside portion is darker. With UnlitMaterial, the edge disappears completely. With PBR—even metallic 0 and roughness 1—the edge returns. The static sphere reproduces the issue, so it does not appear to be caused by gesture or lighting-update timing. Changing lightingBlendDistance does not soften this remaining edge. The issue is visible on Apple Vision Pro but not in visionOS Simulator. I watched the portal-lighting section of WWDC24 session 10103 and implemented the recommended EnvironmentLightingConfigurationComponent solution. I have also reviewed the documentation for PortalComponent, PortalCrossingComponent, PortalComponent.makePortal, ImageBasedLightReceiverComponent, and environmentLightingWeight. Is there another component or entity-hierarchy requirement needed to fully suppress the physical environment probe on the host-side fragments of a crossing PBR entity? Or is this a RealityKit rendering issue on physical hardware? Environment: Apple Vision Pro (M5) visionOS 27.0 (24M5326g) Xcode 27.0 (27A5228h) visionOS SDK 27.0 (24M5326e) Here is a minimal standalone Xcode reproduction containing both the static and pinch-draggable spheres: https://drive.google.com/file/d/13qqOlDJVCCtlFcMaHCBpDkVJgRzmNjBK/view?usp=sharing
3
0
2k
Aug ’26
ManipulationComponent + Warning messages in RealityView
Hi guys! I wanted to study this new ManipulationComponent(), but I keep getting a warning that I don’t understand, even in a very simple scenario. i don't have any collisions just binding the Manipulation the warning message is : ** Entity returned from EntityWrapper.makeEntity(context:) was already parented to another entity. This is not supported and may lead to unexpected behavior. SwiftUI adds entities to internally-managed entity hierarchies.** RealityView { content, attachments in if let loadedModel = try? await Entity(named: "cloud_glb", in: realityKitContentBundle) { content.add(loadedModel) loadedModel.components.set(ManipulationComponent()) } Thanks !
4
0
1.1k
Aug ’26
Pinch gesture not recognized on MTKView when attaching it to a RealityView using a ViewAttachmentComponent in a immersive space
Hello! We are seeing a problem with a SwiftUI view that wraps an MTKView and that MTKView uses gesture recognizers from UIKit. One of those gestures we are using is UIPinchGestureRecognizer. And that gesture isn’t recognized at all when the SwiftUI view is attached to a RealityView using the ViewAttachmentComponent AND the RealityView is being shown in an ImmersiveSpace. If the SwiftUI view is attached to the RealityView using the init that has an attachment closure then pinching works fine there. So this definitely seems like a bug. Here is some code to help you reproduce the problem. Run this on a Vision Pro device. A simple red square will be rendered and if a single tap or pinch gesture is recognized on the red square, it will print to the console. App Code: import SwiftUI @main struct VisionPinchProblemsApp: App { var body: some Scene { WindowGroup { MenuView() } ImmersiveSpace(id: "RedSquare") { RedSquareView() } } } View code: import MetalKit import RealityKit import SwiftUI import UIKit struct MenuView: View { @Environment(\.openImmersiveSpace) private var openImmersiveSpace @Environment(\.dismissImmersiveSpace) private var dismissImmersiveSpace @State private var showImmersiveSpace = false @State private var immersiveSpaceIsOpen = false var body: some View { Form { Toggle("Show red square", isOn: $showImmersiveSpace) .task(id: showImmersiveSpace) { if showImmersiveSpace { await openImmersiveSpace(id: "RedSquare") immersiveSpaceIsOpen = true } else { if immersiveSpaceIsOpen { await dismissImmersiveSpace() immersiveSpaceIsOpen = false } } } } .onDisappear { // Attempt to close the immersive space on the way out. Task { if immersiveSpaceIsOpen { await dismissImmersiveSpace() } } } } } struct RedSquareView: View { let metalViewAttachmentID = "metalID" var body: some View { // Adds SwiftUI view using attachments closure. // Pinching and single taps are recognized here! // RealityView { content, attachments in // if let metalViewEntity = attachments.entity(for: metalViewAttachmentID) { // metalViewEntity.position = [0, 1, -1.25] // content.add(metalViewEntity) // } // } placeholder: { // ProgressView() // } attachments: { // Attachment(id: metalViewAttachmentID) { // MetalView() // } // } // Add SwiftUI view using ViewAttachmentComponent. // Pinching is not recognized here! // Single tapping is recognized ! // Why doesn't the red square show up in the Vision Pro simulator? RealityView { content in let metalViewEntity = Entity() let metalView = MetalView() .frame(width: 500, height: 500) let component = ViewAttachmentComponent(rootView: metalView) metalViewEntity.components.set(component) metalViewEntity.position = [0, 1, -1.25] content.add(metalViewEntity) } placeholder: { ProgressView() } } } struct MetalView: UIViewRepresentable { var device: MTLDevice? init() { self.device = MTLCreateSystemDefaultDevice() } func makeUIView(context: Context) -> MTKView { let mtkView = MTKView() mtkView.device = device mtkView.clearColor = MTLClearColor(red: 1.0, green: 0.0, blue: 0.0, alpha: 1.0) mtkView.delegate = context.coordinator let pinchGesture = UIPinchGestureRecognizer(target: context.coordinator, action: #selector(context.coordinator.handlePinch(_:))) mtkView.addGestureRecognizer(pinchGesture) let tapGesture = UITapGestureRecognizer(target: context.coordinator, action: #selector(context.coordinator.handleTap(_:))) mtkView.addGestureRecognizer(tapGesture) return mtkView } func updateUIView(_ uiView: MTKView, context: Context) { } func makeCoordinator() -> Coordinator { Coordinator(self) } class Coordinator: NSObject, MTKViewDelegate { var parent: MetalView init(_ parent: MetalView) { self.parent = parent } func mtkView(_ view: MTKView, drawableSizeWillChange size: CGSize) { } func draw(in view: MTKView) { guard let drawable = view.currentDrawable else { return } guard let descriptor = view.currentRenderPassDescriptor else { return } let commandQueue = parent.device?.makeCommandQueue() let commandBuffer = commandQueue?.makeCommandBuffer() let renderEncoder = commandBuffer?.makeRenderCommandEncoder(descriptor: descriptor) renderEncoder?.endEncoding() commandBuffer?.present(drawable) commandBuffer?.commit() } @objc func handlePinch(_ sender: UIPinchGestureRecognizer) { print("Pinch detected") } @objc func handleTap(_ sender: UITapGestureRecognizer) { print("Tap detected") } } }
1
0
1.3k
Aug ’26
ManipulationComponent causes makeUIView(context:) to get called twice
Here I have some demo code that is rendering a cylinder "platter" using RealityKit and there is a red circle rendered on top of it which uses Metal and SwiftUI. When the platter appears you will see in the console that makeUIView(context:) is called twice while it is documented that it will only be called once when the view appears for the first time. So this seems like a bug. If you remove ManipulationComponent from the platter's components you will see that this problem goes away so it seems like that is the cause of the problem. Any insight here would be appreciated! Thank you. Here is what is printed in the console: Entity returned from EntityWrapper.makeEntity(context:) was already parented to another entity. This is not supported and may lead to unexpected behavior. SwiftUI adds entities to internally-managed entity hierarchies. Make UI View! This should be called once. Make UI View! This should be called once. Here is the app code: import SwiftUI @main struct SomeApp: App { var body: some Scene { WindowGroup { ContentView() } ImmersiveSpace(id: "TableTop") { TableTopPlatterView() } } } Here is the view code: import MetalKit import RealityKit import SwiftUI struct ContentView: View { @Environment(\.openImmersiveSpace) private var openImmersiveSpace @Environment(\.dismissImmersiveSpace) private var dismissImmersiveSpace @State private var showImmersiveSpace = false @State private var immersiveSpaceIsOpen = false var body: some View { Form { Toggle("Show table top", isOn: $showImmersiveSpace) .task(id: showImmersiveSpace) { if showImmersiveSpace { await openImmersiveSpace(id: "TableTop") immersiveSpaceIsOpen = true } else { if immersiveSpaceIsOpen { await dismissImmersiveSpace() immersiveSpaceIsOpen = false } } } } .onDisappear { // Attempt to close the immersive space on the way out. Task { if immersiveSpaceIsOpen { await dismissImmersiveSpace() } } } } } struct TableTopPlatterView: View { private var attachmentID: String { "RedCircle" } var body: some View { RealityView { content, attachments in if let redCircleEntity = attachments.entity(for: attachmentID) { // Lays the red circle in the platter. let rotation = Rotation3D(redCircleEntity.orientation) .rotated(by: .init(angle: .degrees(-90), axis: .x)) redCircleEntity.setOrientation(.init(rotation), relativeTo: nil) redCircleEntity.position.y = 0.026 platterEntity.addChild(redCircleEntity) content.add(platterEntity) } } placeholder: { ProgressView() } attachments: { Attachment(id: attachmentID) { MetalView() .clipShape(.circle) } } } /// The platter entity that the red circle lays on top of. private let platterEntity: ModelEntity = { let anchor = AnchorEntity( .plane( .horizontal, classification: .table, minimumBounds: [0.01, 0.01] ) ) let material = SimpleMaterial( color: .lightGray, roughness: 0.5, isMetallic: false ) let platter = ModelEntity( mesh: .generateCylinder(height: 0.05, radius: 0.475), materials: [material] ) platter.generateCollisionShapes(recursive: false) let components: [any Component] = [ InputTargetComponent(), GroundingShadowComponent(castsShadow: true), ManipulationComponent() // MARK: This is causing makeUIView to get called twice! ] platter.components.set(components) // Placed closer to the user when booted up. platter.position = [0, 1, -1.25] anchor.addChild(platter) return platter }() } // Metal view that renders a red square. struct MetalView: UIViewRepresentable { var device: MTLDevice? init() { self.device = MTLCreateSystemDefaultDevice() } func makeUIView(context: Context) -> MTKView { print("Make UI View! This should be called once.") let mtkView = MTKView() mtkView.device = device mtkView.clearColor = MTLClearColor(red: 1.0, green: 0.0, blue: 0.0, alpha: 1.0) mtkView.delegate = context.coordinator return mtkView } func updateUIView(_ uiView: MTKView, context: Context) { } func makeCoordinator() -> Coordinator { Coordinator(self) } class Coordinator: NSObject, MTKViewDelegate { var metalView: MetalView init(_ metalView: MetalView) { self.metalView = metalView } func mtkView(_ view: MTKView, drawableSizeWillChange size: CGSize) { } func draw(in view: MTKView) { guard let drawable = view.currentDrawable else { return } guard let descriptor = view.currentRenderPassDescriptor else { return } let commandQueue = metalView.device?.makeCommandQueue() let commandBuffer = commandQueue?.makeCommandBuffer() let renderEncoder = commandBuffer?.makeRenderCommandEncoder(descriptor: descriptor) renderEncoder?.endEncoding() commandBuffer?.present(drawable) commandBuffer?.commit() } } }
3
0
2.1k
Aug ’26
Adding a ManipulationComponent detaches the whole RealityView content hierarchy on visionOS 27
On visionOS 27 (27.0 Seed 4, 24M5326g), adding a ManipulationComponent to a single entity makes RealityKit remove the RealityView's ENTIRE content hierarchy from the scene and re-add it about 30ms later. Not just the manipulated entity — everything. Same app binary on visionOS 26: never happens. FB24092291. Bisected on the same build and device: two entities, neither with manipulation -> no detach add one entity with a ManipulationComponent -> detach, every time same entity, manipulation swapped for my own drag/rotate/ scale gesture handling -> no detach Everything else is identical between the last two cases — same entity, same collision shape, same InputTargetComponent, same ViewAttachmentComponent, same scene, same view hierarchy. The only variable is whether the component is installed. Setup: an immersive space containing a SwiftUI RealityView. One root entity added to the RealityViewContent in the make closure; all app content built as its descendants. The detach is brief but not harmless: every entity in the hierarchy gets SceneEvents.WillRemoveEntity and then DidAddEntity, so anything keyed to those events runs a full teardown and re-add for content that was never meant to go anywhere. In my app that cascade is what makes the scene visibly empty and rebuild. Nothing in my code removes it. Stack captured from a WillRemoveEntity subscription on the root — frame 0 is my callback, everything above it is framework: SwiftUI (AttributeGraph StatefulRule.withObservation ...) -> RealityFoundation -> CoreRE -> entity removal Ruled out before landing on the component: make runs exactly once, the root re-enters a scene with the same ObjectIdentifier (not a re-host), the hosting controller is never deallocated, there is exactly one RealityViewContent.add(_:) in the whole app, and with Self._printChanges() in the view's body the detach happens in an update pass where the body is not re-evaluated at all. Also ruled out: transient overlay UI (suppressed entirely, still detached) and entity count. Workaround if this is biting you: skip ManipulationComponent on 27 and handle drag/rotate/scale yourself. That is what I am doing for now. If you are debugging something similar: SceneEvents.WillRemoveEntity also fires when an entity merely leaves a scene, so a teardown-looking log is not proof anything was destroyed. Check whether make ran twice and whether the scene identity changed before suspecting your own code.
1
2
1.6k
Aug ’26
How Did Apple Create the Fitness App Awards Animation?
Open your Apple Fitness app, I am facinated by the animation of the Awards screen, it is so smooth. Any idea how this was transiton created from a Grid/CollectionView to a RealityView? I assume the interative badge is a RealityView with a 3D model in there, but how the hell the transition was so smooth from the parent list screen to the detail screen.
0
0
649
Jul ’26
Why is this framework marketed as AR/VR ?
Its an honest question, while VR and AR are a cool thing, they are a bit far from what the vast majority of gaming is. (wither mobile or none mobile) Im only a few weeks with the system, but it think its insanely powerful as a game dev tool.Data orient based with ECS, fast, seamless, frictionless, well thought, optimized and packed everything needed. Just my humble opinion, Maybe consider naming it Apple Game Engine / Apple Game Framework - it will attract a lot of developers who just want to make games and are tired of the bloated and messy unreal and unity, tired of being chained into a bloated OOP hierarchy. "Reality" within the name of the kit, as a concept scares away a lot of developers, they think its just a tool only for VR, they dont know its actually a complete powerful and optimized framework to create any game you want. It have so many advantages over unreal. Unreal wraps everything in proprietary objects, forcing you learn mountains of made up keywords even if you already know native C++ A lot of developers are attracted to rayLib because of how bloated commercial game engines are, they dont know Reality Kit is actually a better solution then a lot of framework libraries out there. Just my thoughts,Maybe it finds its way to some folks in marketing. Either way thank you for creating this framework. its absolutely amazing to use
0
1
688
Jul ’26
Is GaussianSplatComponent different than other Components?
I have been analyzing Apple's Gaussian-Splats-on-visionOS demo project, and it appears that the GaussianSplatComponent is treated differently than other Components. In the function makeSplatEntity(), a GaussianSplatComponent is created and added to an Entity's components: let comp = try assembleSplatComponent(from: buffers, isLinear: isLinear) let entity = Entity() entity.components.set(comp) But when I check the entity's components collection before and after the add/set, nothing has changed. The entity still has the same children. Is the GaussianSplatComponent stored in a different location in the Entity? If so, is it a public property I can inspect? (Or is my interpretation of the components system wrong?) Here are the results of print statements before and after the set() BEFORE ADDING COMPONENT Entity number of components: 2 Component: Transform(scale: SIMD3<Float>(1.0, 1.0, 1.0), rotation: simd_quatf(real: 1.0, imag: SIMD3<Float>(0.0, 0.0, 0.0)), translation: SIMD3<Float>(0.0, 0.0, 0.0)) Component: RealityKit.SynchronizationComponent AFTER ADDING COMPONENT Entity number of components: 2 Component: Transform(scale: SIMD3<Float>(1.0, 1.0, 1.0), rotation: simd_quatf(real: 1.0, imag: SIMD3<Float>(0.0, 0.0, 0.0)), translation: SIMD3<Float>(0.0, 0.0, 0.0)) Component: RealityKit.SynchronizationComponent
1
0
809
Jul ’26
Viewing Gaussian splats while inside the model
I am trying to view immersive environments that are Gaussian Splats created by marble [dot] worldlabs [dot] ai. For example, here is one named "Steampunk City Night View": (Note: the low-res version is 500,000 splats, so I am just testing the first 200,000 splats). But when I try to view it at scale, nothing appears. I tried shrinking it down (scale factor of 0.00595458) and positioning it in front of me, and I could see the content, so the parsing of the PLY file is working. Example: But when I move into the model, it goes clear, also hiding virtual content behind it that is also in the splats' space (like the Mac virtual screen). Example: It is as if when I move inside the Gaussian Splats' bounding box, it sets an opacity material on the splats. This prevents me from using these Gaussian Splats as immersive environments. Is there any way to disable this behavior?
1
0
809
Jul ’26
Reality Kit 3 large scale practices - what is under the hood
Hello guys I want to use Reality Kit 3 for none AR games. Coming from unreal (7y) I have a few technical questions if i may please: (all questions are assuming my game will handle 50-500 enemies on screen) How does RTK3 handle everything under the hood? its easy to create a struct that hold multiple floats as a data blocks for the system to handle (like hp, dmg, ect), but, what happen when i want generic ASSET data (materials, meshes) ? What if i want to construct a material and hand a cheap id to the processor (system) ? while make sure i packed it nicely to the cpu's cache line . There is not much in the documentation to explain the under the hood architecture of the system for me to make educated decision abut my code. i try to avoid reference bloat in my game, and make sure the system does not coupled new material or new mesh every time it want to perform a generic command on the entity (in 60 fps) For example, in unreal mass, as bloated and over complicated it is - its pretty clear to me what is construction phase, what is the recommended way to generate my data blocks (called fragments there) and how exactly to use them in the system so its all tightly packed At this moment my only assumption is to create a 3 layers custom system to make sure its all running properly: The builder of the pool - it will do the " let shinyMetal = SimpleMaterial(color: ... ect" and save it on some dictionary map the data block - just a simple struct that hold a thin reference to the pool that can be use as a component in the ECS the system - The RTK3 built is system real time . can use the struct as a component and even swap the id in real time is this safe consider how RTK3 build under the hood? or is this even slower ? i have very little to go by. if reality kit engineer can reply and help me here it would be awesome :) Thank you so much for reading guys ! was kinda long. cheers
1
1
945
Jul ’26
Reality Composer Pro 3 is unable to properly import my .usdz file
Hi, I’m running into an issue when importing a .usdz asset into Reality Composer Pro 3. The exact same asset was working perfectly in Reality Composer Pro 2, and it also works correctly when previewed in Xcode 27.0 beta or loaded directly in RealityKit from code. However, in Reality Composer Pro 3, the asset does not import correctly: The material appears completely broken and looks different from the original asset. The animations are not working. The same .usdz works as expected in Reality Composer Pro 2, including materials and animations. The same .usdz also works as expected in Xcode / RealityKit. I also noticed that the Preview app does not seem to import the asset correctly either. In Preview, the asset has the wrong orientation and it looks like the skeleton and animations may not be recognized properly. Here are a few screenshots showing how the asset appears in each app: Reality Composer Pro 2: asset works correctly, including materials and animations. Xcode 27.0 beta preview: asset works correctly, including materials and animations. Reality Composer Pro 3: material is broken and animations are not working. Preview app: wrong orientation, possible skeleton / animation import issue. I filed a ticket FB23214120 that contains an example Xcode project and RCP2 & RCP3 projects. Is there anything specific that changed in the USDZ import pipeline in Composer Pro 3 that could explain this behavior? Thanks!
3
2
1.3k
Jun ’26
Best practice for connecting Mac-side USDZ preparation with exact-asset Vision Pro verification?
I’m developing two complementary developer tools: Reality Prep Pro on macOS and Reality Prep Preview on visionOS. The workflow I’m trying to solve is: Mac: prepare and validate a USDZ Vision Pro: verify that exact USDZ on-device Mac: bring the device evidence back and associate it with the exact asset revision I currently use the USDZ’s SHA-256 as the authoritative identity. If the asset bytes later change, the previous Vision Pro evidence is treated as stale and a re-test is recommended. Existing USDZ→USDZ preparation is package-preserving, so non-texture payloads such as animation/rigging data are not unnecessarily re-authored. I’d be interested in hearing from other RealityKit / Reality Composer Pro developers: Is this similar to how you currently handle asset provenance between desktop preparation and real-device testing? Are there any Apple frameworks or established visionOS practices you would recommend for strengthening this kind of revision-to-device-evidence workflow?
Replies
0
Boosts
0
Views
664
Activity
Sep ’26
Physical space lighting (SurroundingsLight) never lands on sloped ceiling surfaces that are present in the scene reconstruction mesh
Summary A SpotLightComponent with SpotLightComponent.SurroundingsLight() (and a ProjectiveTexture) lights the flat part of my ceiling and the walls, but never the two sloped ceiling surfaces of a converted attic room, at any angle, intensity, attenuation radius or falloff. The lit area stops in a hard straight line exactly where the flat ceiling meets each slope, even though the beam clearly extends past that line. The sloped surfaces are fully present in SceneReconstructionProvider's mesh, and their faces point into the room. So this doesn't look like a scene understanding coverage problem. It looks as if physical space lighting only considers surfaces classified as horizontal ceiling, wall or floor, and skips slanted ones. Steps to reproduce Open a mixed immersive space in a room with a sloped ceiling (a converted attic, or any vaulted ceiling). Add an entity with SpotLightComponent(color: .white, intensity: 40_000, innerAngleInDegrees: 44.5, outerAngleInDegrees: 45, attenuationRadius: 10), SpotLightComponent.SurroundingsLight(), and a SpotLightComponent.ProjectiveTexture carrying a labelled grid texture so the beam's extent is visible. Place it near the floor under the wearer and aim it straight up. Run SceneReconstructionProvider alongside and draw each MeshAnchor as a wireframe (or as a solid material with faceCulling = .back) to confirm the mesh covers the slopes and faces inward. Expected The projected pattern conforms to every surface in the scene understanding mesh inside the cone, including the sloped ceiling, as it does for the flat ceiling and walls. Actual The pattern appears only on the flat ceiling strip and on the walls. Both slopes stay completely dark, with a hard straight cut at the crease. Tilting the light to aim directly at a slope, raising intensity to 200,000, and changing attenuation radius and falloff make no difference. The wireframe shows continuous mesh across the crease and onto the slopes; the back-face-culled solid view shows the slope faces pointing into the room. Screenshots attached: (1) the projected grid stopping at the crease, (2) the reconstruction wireframe covering the slopes, (3) the solid front-faces-only mesh showing the slopes are front-facing. (4) an additional image of the projected grid with both ceiling and walls in view Code let spot = Entity() spot.components.set(SpotLightComponent.SurroundingsLight()) var light = SpotLightComponent(color: .white, intensity: 40_000, innerAngleInDegrees: 44.5, outerAngleInDegrees: 45, attenuationRadius: 10) light.attenuationFalloffExponent = 0.5 spot.components.set(light) spot.components.set(SpotLightComponent.ProjectiveTexture(texture: gridTexture)) spot.look(at: from + SIMD3(0, 1, 0), from: from, upVector: SIMD3(0, 0, -1), relativeTo: nil) root.addChild(spot) Questions Is physical space lighting limited to horizontally and vertically classified surfaces by design? If so, could this be documented on SurroundingsLight, and is there a way to include slanted surfaces? If not by design, is there anything an app can do to have the light use the full reconstruction mesh?
Replies
2
Boosts
0
Views
1.3k
Activity
Sep ’26
PresentationComponent loses gaze targeting or dismisses in a volumetric window
On visionOS 26.6 and 27 latest beta: PresentationComponent in a volumetric WindowGroup, when looking at the volume from a specific lower/back region create two bugs: buttons inside the presentation have their gaze obstructed by hand presence, no button shows any hover highlight hand movement in that specific area auto dismisses the presentation The focused public repro uses a thin ModelEntity with InputTargetComponent and ordinary SwiftUI buttons. Its README includes two clearer videos: https://github.com/mesqueeb/PresentationAutoCloseInBackOfVolume 1. Gaze obstructed on buttons when hand is at specific location (see "Test Action" has no hover highlight, even though i look at it) 2. Hover state restored on buttons when hand is gone (see "Test Action" has hover highlight) 3. Presentation auto closed when hand comes back The weirdest part: This only happens at a specific location, when walking to the back of the volume, then turning around and interacting with the volume from the lower right back side. Any workaround or fix guidance would be appreciated. Feedback Assistant: FB24569418.
Replies
2
Boosts
0
Views
1.6k
Activity
Sep ’26
Reality Composer Pro is freezing/not working
Hi all! I am struggling to make any app because reality composer pro freezes when I click on create new project, as well as it does not appear in Xcode at all. I have tried deleting and redownloading all versions of both Reality Composer Pro & Xcode, as well as restarting my computer. Kindly advise what else I can try. Thank you!
Replies
0
Boosts
0
Views
407
Activity
Aug ’26
OpacityComponent does not blend a GaussianSplatComponent on iOS/macOS, behaves correct on visionOS
I have a RealityKit scene with one container entity and two children: an entity carrying a GaussianSplatComponent, and a plain Entity with a ModelComponent (an unlit white cube). A single OpacityComponent goes on the container, so one value covers both children in the same frame. container.components.set(OpacityComponent(opacity: value)) On visionOS 27 this does exactly what I expect: both children fade together, continuously, across the whole range. On iOS 27 and Mac Catalyst 27 the cube fades continuously but the splat does not. It renders at full opacity for every value above 0 and disappears at exactly 0. At 0.01 the cube beside it is already almost invisible while the splat is still drawing at full strength, so entity opacity behaves as a visibility switch on a splat rather than as a blend factor. Things I have already ruled out: Not a propagation issue. Moving the OpacityComponent off the container and onto the splat's own entity (and the cube's own entity, to keep the A/B) changes nothing. Not the container failing to receive the value. The cube under the same component, in the same frame, fades correctly. Not the splat data. Per-splat alpha through GaussianSplatResource's opacity buffer is honoured, and projectionMode, sortingMode, scaleActivation and opacityActivation all take effect. Not a difference in scene construction per platform. The same file builds the container on all three platforms in my sample. Confirmed on: iOS 27.0 (24A5418b), iPhone 16 Pro — reproduces macOS 27.0 (26A5416b), Mac Catalyst — reproduces visionOS 27.0 (24M5326g), Apple Vision Pro — correct Filed as FB24431214 with a sample project that builds for all three from one target. Would be great to see this fixed!
Replies
1
Boosts
0
Views
659
Activity
Aug ’26
PhotogrammetrySession(input: [PhotogrammetrySample]) Hangs or terminates
Xcode hangs when I call PhotogrammetrySession(input: [PhotogrammetrySample]) with objectMask set and traps on some devices, see the attached screenshot, it gets to the function and hangs. Even the folder reconstruction also doesn't complete as it can't find alignment and displays the CoreOC.PhotogrammetrySession.Error 6 and I understand to mean alignment failed. In this case it failed while object masking was ON, so RealityKit could not find enough consistent feature tracks inside the masked pixels across the image set. What can be done or do you guys expose any functions that can be used to aid, or handle these internally, can't find any internally. The ObjectMasks are actually segmentation masks from an ML algorithm . To replicate try calling PhotogrammetrySession(input: [PhotogrammetrySample]) with contentsOf as captured on your documentation, even with like 30 image set or is there something I'm missing. I will appreciate a timely response and willing to provide more clarity and informations, thank you so much for your understanding
Replies
4
Boosts
0
Views
2.9k
Activity
Aug ’26
Immersive space closes if immersive environment was fully open
If the immersive environment is fully open, then an immersive space is closed right after attempting to open it. The issue specifically impacts a mixed immersive space which replaces the immersive environment while the immersive environment is at 100%. This issue does not occur if immersive environment is any less than 100%, or if the immersionStyle is .full, or if immersiveEnvirionmentBehavior is .coexist. This can be reproduced in the template for an immersive visionOS app by removing the line .immersionStyle(selection: .constant(.full), in: .full) Then run the app and open the immersive space, and the space then closes after attempting to open it. I’m using visionOS 27.0 beta 5 and Xcode 27.0 beta 5. I've submitted feedback as FB24353599 Video of the behavior: https://youtu.be/wafDXRZVyog
Replies
1
Boosts
0
Views
1.1k
Activity
Aug ’26
ViewAttachmentComponent Resolution Low After Moving Into Frame
If a ViewAttachmentComponent moves into frame, it is low resolution until something changes the view while it is in frame. Video demonstrating the behavior: https://youtu.be/KXEFFiAnv1s I am on visionOS 27 beta 4. This did not occur when I was on visionOS 26.5. Also using Xcode 27.0 beta 4 and macOS 27.0 beta 4. To reproduce, have a ViewAttachmentComponent in an immersive space, look away, then look back, and it'll be low resolution. Anything which would change the view while it's in frame will then cause it to update in full resolution. Screenshot of low-resolution view after it moves back into frame from being out of frame: Screenshot after updating the view, making it high-resolution again: I've submitted feedback as FB24116473.
Replies
2
Boosts
0
Views
1.4k
Activity
Aug ’26
A ClothGrabComponent movement problem
I'm building a Wacom tablet-driven cloth editing rig in visionOS: a persistent grab sphere (one Entity with ClothGrabComponent, volume mode) parented to the simulation root, toggled with isGrabbing on each pen-down. A single grab behaves correctly — the cloth follows the sphere, nothing else moves. On the second grab at a different position (same cloth, no rebuild), the component keeps dragging the particles from the FIRST grab to the new sphere position — its internal grab coordinates are not updated by the new activation. The vertex that was just bound is pushed away from the sphere at the same time. Instrumented demo is (0.5 × 0.5 m grid, 289 vertices, four corners pinned, gravity = 0, sphere radius 0.06, falloff = .disabled), grab A at vertex #294 (−0.118, +0.118), release, then grab B at vertex #105 (+0.118, −0.118) — 0.335 m apart: t = 345.9 GRAB_B starts: previous #294 disp = 0.000 dist-to-ball = 0.335 current #105 disp = 0.000 dist-to-ball = 0.000 t = 346.4 (+0.5 s): previous #294 disp = 0.335 dist-to-ball = 0.000 ← dragged to the NEW ball current #105 disp = 0.191 dist-to-ball = 0.191 ← pushed AWAY from the ball t = 347.0 … 349.6 previous #294 pinned at ball (dist 0.000), #105 held away at 0.16–0.19 t = 351.9 after release: #294 back to rest (0.012), cloth flat again The 0.335 m displacement of #294 happens within half a second and equals exactly the distance from #294's rest position to the new sphere — the previous grab's particle set is being pulled toward the new sphere location, as if the component re-applied the old grab selection with the new transform. this can repeat with identical numbers. The docs for isGrabbing only say "Indicates whether particles are currently being grabbed" — they don't describe what happens on a false→true transition after the entity has moved, which is the case the official sample never demonstrates. The setup is the one shown in the official sample — a persistent entity carrying the grab component, isGrabbing toggled per interaction: // makeCloth — once let dragBall = makeBall(radius: 0.015, parent: simRoot) var grab = ClothGrabComponent(mode: .volume(shape: .sphere(ClothSphereShape(radius: 0.06)))) grab.falloff = .disabled dragBall.components.set(grab) // grab: move ball to new vertex, activate dragBall.position = body.convert(position: vertexPos, to: simRoot) var g = dragBall.components[ClothGrabComponent.self]! g.isGrabbing = true dragBall.components.set(g) // release: deactivate only var g = dragBall.components[ClothGrabComponent.self]! g.isGrabbing = false dragBall.components.set(g) some tries: Remove and re-add ClothGrabComponent on each grab — the simulator crashes within 1–2 frames with Assertion failed Rebuild the whole ClothBodyComponent between grabs Environment: Xcode 27 beta (build 24M5316i), xrOS 27.0 SDK, simulator runtime com.apple.CoreSimulator.SimRuntime.xrOS-27-0 (avp1). All Cloth* APIs are Beta on visionOS 27
Replies
1
Boosts
0
Views
1.1k
Activity
Aug ’26
RealityKit portal crossing leaves a hard PBR lighting seam despite environmentLightingWeight being 0
I’m seeing a device-only lighting discontinuity on PBR entities that intersect a RealityKit portal plane in visionOS 27. A sharp brightness boundary appears exactly where the portal plane cuts through the entity. The portion outside the portal receives a different physical environment-probe contribution from the portion inside. The behavior is reproducible with two independent cases: A completely static blue sphere positioned across the portal plane. An orange sphere moved through the portal using ManipulationComponent and the standard visionOS pinch gesture. Both spheres use: var material = PhysicallyBasedMaterial() material.baseColor = .init(tint: .orange) material.metallic = 0 material.roughness = 1 Each sphere has the crossing and lighting components attached directly: sphere.components.set(PortalCrossingComponent()) sphere.components.set( ImageBasedLightReceiverComponent( imageBasedLight: portalImageBasedLight ) ) var lightingConfiguration = EnvironmentLightingConfigurationComponent() lightingConfiguration.environmentLightingWeight = 0 sphere.components.set(lightingConfiguration) The portal is created using the visionOS 27 factory API: let portal = PortalComponent.makePortal( surfaceStyle: .init(width: 1.2, height: 0.8), boundaryStyle: .infinitePlane(), boundaryMode: .clippingAndCrossing ) if var component = portal.portalEntity.components[PortalComponent.self] { component.lightingBlendDistance = 0.2 portal.portalEntity.components.set(component) } For the moving sphere, environmentLightingWeight stays at 0 for the entire interval in which any part of the sphere intersects the portal. The realtime environment-probe contribution only begins fading from 0 to 1 after the complete sphere has cleared the plane. However, a spatially sharp edge remains: In a bright physical environment, the outside portion is brighter. In a dark physical environment, the outside portion is darker. With UnlitMaterial, the edge disappears completely. With PBR—even metallic 0 and roughness 1—the edge returns. The static sphere reproduces the issue, so it does not appear to be caused by gesture or lighting-update timing. Changing lightingBlendDistance does not soften this remaining edge. The issue is visible on Apple Vision Pro but not in visionOS Simulator. I watched the portal-lighting section of WWDC24 session 10103 and implemented the recommended EnvironmentLightingConfigurationComponent solution. I have also reviewed the documentation for PortalComponent, PortalCrossingComponent, PortalComponent.makePortal, ImageBasedLightReceiverComponent, and environmentLightingWeight. Is there another component or entity-hierarchy requirement needed to fully suppress the physical environment probe on the host-side fragments of a crossing PBR entity? Or is this a RealityKit rendering issue on physical hardware? Environment: Apple Vision Pro (M5) visionOS 27.0 (24M5326g) Xcode 27.0 (27A5228h) visionOS SDK 27.0 (24M5326e) Here is a minimal standalone Xcode reproduction containing both the static and pinch-draggable spheres: https://drive.google.com/file/d/13qqOlDJVCCtlFcMaHCBpDkVJgRzmNjBK/view?usp=sharing
Replies
3
Boosts
0
Views
2k
Activity
Aug ’26
ManipulationComponent + Warning messages in RealityView
Hi guys! I wanted to study this new ManipulationComponent(), but I keep getting a warning that I don’t understand, even in a very simple scenario. i don't have any collisions just binding the Manipulation the warning message is : ** Entity returned from EntityWrapper.makeEntity(context:) was already parented to another entity. This is not supported and may lead to unexpected behavior. SwiftUI adds entities to internally-managed entity hierarchies.** RealityView { content, attachments in if let loadedModel = try? await Entity(named: "cloud_glb", in: realityKitContentBundle) { content.add(loadedModel) loadedModel.components.set(ManipulationComponent()) } Thanks !
Replies
4
Boosts
0
Views
1.1k
Activity
Aug ’26
Pinch gesture not recognized on MTKView when attaching it to a RealityView using a ViewAttachmentComponent in a immersive space
Hello! We are seeing a problem with a SwiftUI view that wraps an MTKView and that MTKView uses gesture recognizers from UIKit. One of those gestures we are using is UIPinchGestureRecognizer. And that gesture isn’t recognized at all when the SwiftUI view is attached to a RealityView using the ViewAttachmentComponent AND the RealityView is being shown in an ImmersiveSpace. If the SwiftUI view is attached to the RealityView using the init that has an attachment closure then pinching works fine there. So this definitely seems like a bug. Here is some code to help you reproduce the problem. Run this on a Vision Pro device. A simple red square will be rendered and if a single tap or pinch gesture is recognized on the red square, it will print to the console. App Code: import SwiftUI @main struct VisionPinchProblemsApp: App { var body: some Scene { WindowGroup { MenuView() } ImmersiveSpace(id: "RedSquare") { RedSquareView() } } } View code: import MetalKit import RealityKit import SwiftUI import UIKit struct MenuView: View { @Environment(\.openImmersiveSpace) private var openImmersiveSpace @Environment(\.dismissImmersiveSpace) private var dismissImmersiveSpace @State private var showImmersiveSpace = false @State private var immersiveSpaceIsOpen = false var body: some View { Form { Toggle("Show red square", isOn: $showImmersiveSpace) .task(id: showImmersiveSpace) { if showImmersiveSpace { await openImmersiveSpace(id: "RedSquare") immersiveSpaceIsOpen = true } else { if immersiveSpaceIsOpen { await dismissImmersiveSpace() immersiveSpaceIsOpen = false } } } } .onDisappear { // Attempt to close the immersive space on the way out. Task { if immersiveSpaceIsOpen { await dismissImmersiveSpace() } } } } } struct RedSquareView: View { let metalViewAttachmentID = "metalID" var body: some View { // Adds SwiftUI view using attachments closure. // Pinching and single taps are recognized here! // RealityView { content, attachments in // if let metalViewEntity = attachments.entity(for: metalViewAttachmentID) { // metalViewEntity.position = [0, 1, -1.25] // content.add(metalViewEntity) // } // } placeholder: { // ProgressView() // } attachments: { // Attachment(id: metalViewAttachmentID) { // MetalView() // } // } // Add SwiftUI view using ViewAttachmentComponent. // Pinching is not recognized here! // Single tapping is recognized ! // Why doesn't the red square show up in the Vision Pro simulator? RealityView { content in let metalViewEntity = Entity() let metalView = MetalView() .frame(width: 500, height: 500) let component = ViewAttachmentComponent(rootView: metalView) metalViewEntity.components.set(component) metalViewEntity.position = [0, 1, -1.25] content.add(metalViewEntity) } placeholder: { ProgressView() } } } struct MetalView: UIViewRepresentable { var device: MTLDevice? init() { self.device = MTLCreateSystemDefaultDevice() } func makeUIView(context: Context) -> MTKView { let mtkView = MTKView() mtkView.device = device mtkView.clearColor = MTLClearColor(red: 1.0, green: 0.0, blue: 0.0, alpha: 1.0) mtkView.delegate = context.coordinator let pinchGesture = UIPinchGestureRecognizer(target: context.coordinator, action: #selector(context.coordinator.handlePinch(_:))) mtkView.addGestureRecognizer(pinchGesture) let tapGesture = UITapGestureRecognizer(target: context.coordinator, action: #selector(context.coordinator.handleTap(_:))) mtkView.addGestureRecognizer(tapGesture) return mtkView } func updateUIView(_ uiView: MTKView, context: Context) { } func makeCoordinator() -> Coordinator { Coordinator(self) } class Coordinator: NSObject, MTKViewDelegate { var parent: MetalView init(_ parent: MetalView) { self.parent = parent } func mtkView(_ view: MTKView, drawableSizeWillChange size: CGSize) { } func draw(in view: MTKView) { guard let drawable = view.currentDrawable else { return } guard let descriptor = view.currentRenderPassDescriptor else { return } let commandQueue = parent.device?.makeCommandQueue() let commandBuffer = commandQueue?.makeCommandBuffer() let renderEncoder = commandBuffer?.makeRenderCommandEncoder(descriptor: descriptor) renderEncoder?.endEncoding() commandBuffer?.present(drawable) commandBuffer?.commit() } @objc func handlePinch(_ sender: UIPinchGestureRecognizer) { print("Pinch detected") } @objc func handleTap(_ sender: UITapGestureRecognizer) { print("Tap detected") } } }
Replies
1
Boosts
0
Views
1.3k
Activity
Aug ’26
ManipulationComponent causes makeUIView(context:) to get called twice
Here I have some demo code that is rendering a cylinder "platter" using RealityKit and there is a red circle rendered on top of it which uses Metal and SwiftUI. When the platter appears you will see in the console that makeUIView(context:) is called twice while it is documented that it will only be called once when the view appears for the first time. So this seems like a bug. If you remove ManipulationComponent from the platter's components you will see that this problem goes away so it seems like that is the cause of the problem. Any insight here would be appreciated! Thank you. Here is what is printed in the console: Entity returned from EntityWrapper.makeEntity(context:) was already parented to another entity. This is not supported and may lead to unexpected behavior. SwiftUI adds entities to internally-managed entity hierarchies. Make UI View! This should be called once. Make UI View! This should be called once. Here is the app code: import SwiftUI @main struct SomeApp: App { var body: some Scene { WindowGroup { ContentView() } ImmersiveSpace(id: "TableTop") { TableTopPlatterView() } } } Here is the view code: import MetalKit import RealityKit import SwiftUI struct ContentView: View { @Environment(\.openImmersiveSpace) private var openImmersiveSpace @Environment(\.dismissImmersiveSpace) private var dismissImmersiveSpace @State private var showImmersiveSpace = false @State private var immersiveSpaceIsOpen = false var body: some View { Form { Toggle("Show table top", isOn: $showImmersiveSpace) .task(id: showImmersiveSpace) { if showImmersiveSpace { await openImmersiveSpace(id: "TableTop") immersiveSpaceIsOpen = true } else { if immersiveSpaceIsOpen { await dismissImmersiveSpace() immersiveSpaceIsOpen = false } } } } .onDisappear { // Attempt to close the immersive space on the way out. Task { if immersiveSpaceIsOpen { await dismissImmersiveSpace() } } } } } struct TableTopPlatterView: View { private var attachmentID: String { "RedCircle" } var body: some View { RealityView { content, attachments in if let redCircleEntity = attachments.entity(for: attachmentID) { // Lays the red circle in the platter. let rotation = Rotation3D(redCircleEntity.orientation) .rotated(by: .init(angle: .degrees(-90), axis: .x)) redCircleEntity.setOrientation(.init(rotation), relativeTo: nil) redCircleEntity.position.y = 0.026 platterEntity.addChild(redCircleEntity) content.add(platterEntity) } } placeholder: { ProgressView() } attachments: { Attachment(id: attachmentID) { MetalView() .clipShape(.circle) } } } /// The platter entity that the red circle lays on top of. private let platterEntity: ModelEntity = { let anchor = AnchorEntity( .plane( .horizontal, classification: .table, minimumBounds: [0.01, 0.01] ) ) let material = SimpleMaterial( color: .lightGray, roughness: 0.5, isMetallic: false ) let platter = ModelEntity( mesh: .generateCylinder(height: 0.05, radius: 0.475), materials: [material] ) platter.generateCollisionShapes(recursive: false) let components: [any Component] = [ InputTargetComponent(), GroundingShadowComponent(castsShadow: true), ManipulationComponent() // MARK: This is causing makeUIView to get called twice! ] platter.components.set(components) // Placed closer to the user when booted up. platter.position = [0, 1, -1.25] anchor.addChild(platter) return platter }() } // Metal view that renders a red square. struct MetalView: UIViewRepresentable { var device: MTLDevice? init() { self.device = MTLCreateSystemDefaultDevice() } func makeUIView(context: Context) -> MTKView { print("Make UI View! This should be called once.") let mtkView = MTKView() mtkView.device = device mtkView.clearColor = MTLClearColor(red: 1.0, green: 0.0, blue: 0.0, alpha: 1.0) mtkView.delegate = context.coordinator return mtkView } func updateUIView(_ uiView: MTKView, context: Context) { } func makeCoordinator() -> Coordinator { Coordinator(self) } class Coordinator: NSObject, MTKViewDelegate { var metalView: MetalView init(_ metalView: MetalView) { self.metalView = metalView } func mtkView(_ view: MTKView, drawableSizeWillChange size: CGSize) { } func draw(in view: MTKView) { guard let drawable = view.currentDrawable else { return } guard let descriptor = view.currentRenderPassDescriptor else { return } let commandQueue = metalView.device?.makeCommandQueue() let commandBuffer = commandQueue?.makeCommandBuffer() let renderEncoder = commandBuffer?.makeRenderCommandEncoder(descriptor: descriptor) renderEncoder?.endEncoding() commandBuffer?.present(drawable) commandBuffer?.commit() } } }
Replies
3
Boosts
0
Views
2.1k
Activity
Aug ’26
Adding a ManipulationComponent detaches the whole RealityView content hierarchy on visionOS 27
On visionOS 27 (27.0 Seed 4, 24M5326g), adding a ManipulationComponent to a single entity makes RealityKit remove the RealityView's ENTIRE content hierarchy from the scene and re-add it about 30ms later. Not just the manipulated entity — everything. Same app binary on visionOS 26: never happens. FB24092291. Bisected on the same build and device: two entities, neither with manipulation -> no detach add one entity with a ManipulationComponent -> detach, every time same entity, manipulation swapped for my own drag/rotate/ scale gesture handling -> no detach Everything else is identical between the last two cases — same entity, same collision shape, same InputTargetComponent, same ViewAttachmentComponent, same scene, same view hierarchy. The only variable is whether the component is installed. Setup: an immersive space containing a SwiftUI RealityView. One root entity added to the RealityViewContent in the make closure; all app content built as its descendants. The detach is brief but not harmless: every entity in the hierarchy gets SceneEvents.WillRemoveEntity and then DidAddEntity, so anything keyed to those events runs a full teardown and re-add for content that was never meant to go anywhere. In my app that cascade is what makes the scene visibly empty and rebuild. Nothing in my code removes it. Stack captured from a WillRemoveEntity subscription on the root — frame 0 is my callback, everything above it is framework: SwiftUI (AttributeGraph StatefulRule.withObservation ...) -> RealityFoundation -> CoreRE -> entity removal Ruled out before landing on the component: make runs exactly once, the root re-enters a scene with the same ObjectIdentifier (not a re-host), the hosting controller is never deallocated, there is exactly one RealityViewContent.add(_:) in the whole app, and with Self._printChanges() in the view's body the detach happens in an update pass where the body is not re-evaluated at all. Also ruled out: transient overlay UI (suppressed entirely, still detached) and entity count. Workaround if this is biting you: skip ManipulationComponent on 27 and handle drag/rotate/scale yourself. That is what I am doing for now. If you are debugging something similar: SceneEvents.WillRemoveEntity also fires when an entity merely leaves a scene, so a teardown-looking log is not proof anything was destroyed. Check whether make ran twice and whether the scene identity changed before suspecting your own code.
Replies
1
Boosts
2
Views
1.6k
Activity
Aug ’26
How Did Apple Create the Fitness App Awards Animation?
Open your Apple Fitness app, I am facinated by the animation of the Awards screen, it is so smooth. Any idea how this was transiton created from a Grid/CollectionView to a RealityView? I assume the interative badge is a RealityView with a 3D model in there, but how the hell the transition was so smooth from the parent list screen to the detail screen.
Replies
0
Boosts
0
Views
649
Activity
Jul ’26
Why is this framework marketed as AR/VR ?
Its an honest question, while VR and AR are a cool thing, they are a bit far from what the vast majority of gaming is. (wither mobile or none mobile) Im only a few weeks with the system, but it think its insanely powerful as a game dev tool.Data orient based with ECS, fast, seamless, frictionless, well thought, optimized and packed everything needed. Just my humble opinion, Maybe consider naming it Apple Game Engine / Apple Game Framework - it will attract a lot of developers who just want to make games and are tired of the bloated and messy unreal and unity, tired of being chained into a bloated OOP hierarchy. "Reality" within the name of the kit, as a concept scares away a lot of developers, they think its just a tool only for VR, they dont know its actually a complete powerful and optimized framework to create any game you want. It have so many advantages over unreal. Unreal wraps everything in proprietary objects, forcing you learn mountains of made up keywords even if you already know native C++ A lot of developers are attracted to rayLib because of how bloated commercial game engines are, they dont know Reality Kit is actually a better solution then a lot of framework libraries out there. Just my thoughts,Maybe it finds its way to some folks in marketing. Either way thank you for creating this framework. its absolutely amazing to use
Replies
0
Boosts
1
Views
688
Activity
Jul ’26
Is GaussianSplatComponent different than other Components?
I have been analyzing Apple's Gaussian-Splats-on-visionOS demo project, and it appears that the GaussianSplatComponent is treated differently than other Components. In the function makeSplatEntity(), a GaussianSplatComponent is created and added to an Entity's components: let comp = try assembleSplatComponent(from: buffers, isLinear: isLinear) let entity = Entity() entity.components.set(comp) But when I check the entity's components collection before and after the add/set, nothing has changed. The entity still has the same children. Is the GaussianSplatComponent stored in a different location in the Entity? If so, is it a public property I can inspect? (Or is my interpretation of the components system wrong?) Here are the results of print statements before and after the set() BEFORE ADDING COMPONENT Entity number of components: 2 Component: Transform(scale: SIMD3<Float>(1.0, 1.0, 1.0), rotation: simd_quatf(real: 1.0, imag: SIMD3<Float>(0.0, 0.0, 0.0)), translation: SIMD3<Float>(0.0, 0.0, 0.0)) Component: RealityKit.SynchronizationComponent AFTER ADDING COMPONENT Entity number of components: 2 Component: Transform(scale: SIMD3<Float>(1.0, 1.0, 1.0), rotation: simd_quatf(real: 1.0, imag: SIMD3<Float>(0.0, 0.0, 0.0)), translation: SIMD3<Float>(0.0, 0.0, 0.0)) Component: RealityKit.SynchronizationComponent
Replies
1
Boosts
0
Views
809
Activity
Jul ’26
Viewing Gaussian splats while inside the model
I am trying to view immersive environments that are Gaussian Splats created by marble [dot] worldlabs [dot] ai. For example, here is one named "Steampunk City Night View": (Note: the low-res version is 500,000 splats, so I am just testing the first 200,000 splats). But when I try to view it at scale, nothing appears. I tried shrinking it down (scale factor of 0.00595458) and positioning it in front of me, and I could see the content, so the parsing of the PLY file is working. Example: But when I move into the model, it goes clear, also hiding virtual content behind it that is also in the splats' space (like the Mac virtual screen). Example: It is as if when I move inside the Gaussian Splats' bounding box, it sets an opacity material on the splats. This prevents me from using these Gaussian Splats as immersive environments. Is there any way to disable this behavior?
Replies
1
Boosts
0
Views
809
Activity
Jul ’26
Reality Kit 3 large scale practices - what is under the hood
Hello guys I want to use Reality Kit 3 for none AR games. Coming from unreal (7y) I have a few technical questions if i may please: (all questions are assuming my game will handle 50-500 enemies on screen) How does RTK3 handle everything under the hood? its easy to create a struct that hold multiple floats as a data blocks for the system to handle (like hp, dmg, ect), but, what happen when i want generic ASSET data (materials, meshes) ? What if i want to construct a material and hand a cheap id to the processor (system) ? while make sure i packed it nicely to the cpu's cache line . There is not much in the documentation to explain the under the hood architecture of the system for me to make educated decision abut my code. i try to avoid reference bloat in my game, and make sure the system does not coupled new material or new mesh every time it want to perform a generic command on the entity (in 60 fps) For example, in unreal mass, as bloated and over complicated it is - its pretty clear to me what is construction phase, what is the recommended way to generate my data blocks (called fragments there) and how exactly to use them in the system so its all tightly packed At this moment my only assumption is to create a 3 layers custom system to make sure its all running properly: The builder of the pool - it will do the " let shinyMetal = SimpleMaterial(color: ... ect" and save it on some dictionary map the data block - just a simple struct that hold a thin reference to the pool that can be use as a component in the ECS the system - The RTK3 built is system real time . can use the struct as a component and even swap the id in real time is this safe consider how RTK3 build under the hood? or is this even slower ? i have very little to go by. if reality kit engineer can reply and help me here it would be awesome :) Thank you so much for reading guys ! was kinda long. cheers
Replies
1
Boosts
1
Views
945
Activity
Jul ’26
Reality Composer Pro 3 is unable to properly import my .usdz file
Hi, I’m running into an issue when importing a .usdz asset into Reality Composer Pro 3. The exact same asset was working perfectly in Reality Composer Pro 2, and it also works correctly when previewed in Xcode 27.0 beta or loaded directly in RealityKit from code. However, in Reality Composer Pro 3, the asset does not import correctly: The material appears completely broken and looks different from the original asset. The animations are not working. The same .usdz works as expected in Reality Composer Pro 2, including materials and animations. The same .usdz also works as expected in Xcode / RealityKit. I also noticed that the Preview app does not seem to import the asset correctly either. In Preview, the asset has the wrong orientation and it looks like the skeleton and animations may not be recognized properly. Here are a few screenshots showing how the asset appears in each app: Reality Composer Pro 2: asset works correctly, including materials and animations. Xcode 27.0 beta preview: asset works correctly, including materials and animations. Reality Composer Pro 3: material is broken and animations are not working. Preview app: wrong orientation, possible skeleton / animation import issue. I filed a ticket FB23214120 that contains an example Xcode project and RCP2 & RCP3 projects. Is there anything specific that changed in the USDZ import pipeline in Composer Pro 3 that could explain this behavior? Thanks!
Replies
3
Boosts
2
Views
1.3k
Activity
Jun ’26