So if I drag an entity in RealityView I have to disable the PhysicsBodyComponent to make sure nothing fights dragging the entity around. This makes sense. When I finish a drag, this closure gets executed: .gesture( DragGesture() .targetedToAnyEntity() .onChanged { e in // ... } .onEnded { e in let velocity: CGSize = e.gestureValue.velocity } If I now re-add PhysicsBodyComponent to the component I just dragged, and I make it mode: .dynamic it will loose all velocity and drop straight down through gravity. Instead the solution is to apply mode: .kinematic and also apply a PhysicsMotionComponent component to the entity. This should retain velocity after letting go of the object. However, I need to instatiate it with PhysicsMotionComponent(linearVelocity: SIMD3<Float>, angularVelocity: SIMD3<Float>). How can I calculate the linearVelocity and angularVelocity when the e.gestureValue.velocity I get is just a CGSize? Is there another prop of gestureValue I should be looking at?

I am working on a RealityView on iOS 18 that needs to render objects farther away than 1,000 meters. My app is used outside in open areas. I am using RealityView with content.camera = .spatialTracking and I have turned off occlusion, collisions, and plane detection with a simple scene understanding like this. let configuration = SpatialTrackingSession.Configuration( tracking: [.camera], sceneUnderstanding: [], //We don't want occlusions, collisions, etc camera: .back) let session = SpatialTrackingSession() if let unavailable = await session.run(configuration) { print("unavailable \(unavailable)") } Is this possible with spatialTracking with RealityView or with ARView? I have my RealityView working on visionOS inside an ImmersiveSpace. On visionOS I don't have the camera as a passthrough, it is virtual scene and it has wold tracking set up via the WorldTrackingProvider and I can render objects father away than 1000 meters. I would like to do the same thing on iOS. I don't need to have the camera pass through, but I do need to have the world tracking. I see that PerspectiveCameraComponent lets me set the near and far clipping planes, but I don't see how I can use that camera with world tracking.

I'm writing a RealityKit/ARKit app that runs on iOS. Starting with Xcode 16.0 beta 1, at least through Xcode 16.1 beta 2 (16B5014f), in the iOS 18 simulator, my app randomly crashes in about 20% of app sessions the first time it attempts to present an ARView. The crashes seem to occur at multiple points within RealityKit and Metal. Below, I've included screenshots of the call stacks of the crashes, which occur as a result of both EXC_BAD_ACCESS and assertion failures within RealityKit. The app only crashes in the iOS 18 simulator, and does not crash in the iOS 17 simulator or earlier. The app only crashes in the simulator, and does not crash on a device running iOS 18. Before I investigate further, I'd appreciate it if an Apple engineer could give me a sense of if these crashes are most likely the result of known issues within RealityKit and/or the simulator, or if your opinion is that there are probably bugs in my app's code. I've submitted several feedback issues in the past, and I'd love to submit this issue too, but I expect that I would spend many hours attempting to create a repro case in a sample app. Understandably, I'd rather not spend this time if an Apple engineer could tell me this is a known issue, for example. Thank you.

Hello, I am trying to obtain the RealityKit URL. Can someone share please?

Hi, does anyone know if it is an easy way to determine the distance between floor and ceiling in vision Pro?

Can VisionOS take screenshots besides simultaneously pressing buttons

Once GKAccessPoint is active, then enter an arview page, the arview will lose camera feed. OSVersion: iOS 18.0.1, iOS 18.1

I want use SwiftUI views as RealityKit entities to display AR Labels within a RealityKit scene, and the labels could be more complicated than just text and window as they might include images, dynamic texts, animations, WebViews, etc. Vision OS enables this through RealityView attachments, and there is a RealityView support on iOS 18. Tried running RealityView attachments code samples from VisionOS on iOS 18. However, the code below gives errors on iOS 18: import SwiftUI import RealityKit struct PassportRealityView: View { let qrCodeCenter: SIMD3<Float> let assetID: String var body: some View { RealityView { content, attachments in // Setup your AR content, such as markers or 3D models if let qrAnchor = try? await Entity(named: "QRAnchor") { qrAnchor.position = qrCodeCenter content.add(qrAnchor) } } attachments: { Attachment(id: "passportTextAttachment") { Text(assetID) .font(.title3) .foregroundColor(.white) .background(Color.black.opacity(0.7)) .padding(5) .cornerRadius(5) } } .frame(width: 300, height: 400) } } When I remove "attachments" keyword and the block, the errors are kind of gone. That does not help me as I want to attach SwiftUI views to Anchor Entities in RealityKit. As I understand, RealityView attachments are not supported on iOS 18. I wonder if there is any way of showing SwiftUI views as entities on iOS 18 at this point. Or am I forced to use the text meshes and 3d planes to build the UI? I checked out the RealityUI plugin, but it's too simple for my use case of building complex AR labels. Any advice would be appreciated. Thanks!

I decided to use a club to kick a ball and let it roll on the turf in RealityKit, but now I can only let it slide but can not roll. I add collision on the turf(static), club (kinematic) and the ball(dynamic), and set some parameters: radius, mass. Using these parameters calculate linear damping, inertia, besides, use time between frames and the club position to calculate speed. Code like these: let radius: Float = 0.025 let mass: Float = 0.04593 // 质量，单位：kg var inertia = 2/5 * mass * pow(radius, 2) let currentPosition = entity.position(relativeTo: nil) let distance = distance(currentPosition, rgfc.lastPosition) let deltaTime = Float(context.deltaTime) let speed = distance / deltaTime let C_d: Float = 0.47 //阻力系数 let linearDamping = 0.5 * 1.2 * pow(speed, 2) * .pi * pow(radius, 2) * C_d //线性阻尼（1.2表示空气密度） entity.components[PhysicsBodyComponent.self]?.massProperties.inertia = SIMD3<Float>(inertia, inertia, inertia) entity.components[PhysicsBodyComponent.self]?.linearDamping = linearDamping // force let acceleration = speed / deltaTime let forceDirection = normalize(currentPosition - rgfc.lastPosition) let forceMultiplier: Float = 1.0 let appliedForce = forceDirection * mass * acceleration * forceMultiplier entityCollidedWith.addForce(appliedForce, at: rgfc.hitPosition, relativeTo: nil) Also I try to applyImpulse but not addForce, like: let linearImpulse = forceDirection * speed * forceMultiplier * mass No matter how I adjust the friction(static, dynamic) and restitution, using addForce or applyImpulse, the ball can only slide. How can I solve this problem?

I have created a simple scene in reality composer (composer not composer pro). It contains just a cube and text item. I convert this to usdz file and load it into a Arkit swift app. Since ios 18/xcode 16 - the "text" element is not displayed at all. The cube is displayed, anchors correctly and can be moved etc.... The output from usdchecker ➜ Desktop usdchecker GKTUHR1.6.3.usdz -v --arkit Opening GKTUHR1.6.3.usdz Checking layer <GKTUHR1.6.3.usdz>. Checking package <GKTUHR1.6.3.usdz> Checking prim </Root>. Checking prim </Root/Scenes>. Checking prim </Root/Scenes/Scene>. Checking prim </Root/Scenes/Scene/Gravity>. Checking prim </Root/Scenes/Scene/sceneGroundPlane>. Checking prim </Root/Scenes/Scene/sceneGroundPlane/physicsMaterial>. Checking prim </Root/Scenes/Scene/Children>. Checking prim </Root/Scenes/Scene/Children/hello>. Checking prim </Root/Scenes/Scene/Children/hello/Generated>. Checking prim </Root/Scenes/Scene/Children/hello/Generated/Text>. Checking prim </Root/Scenes/Scene/Children/hello/Generated/Text/Material>. Checking prim </Root/Scenes/Scene/Children/hello/Generated/Text/Material/PBRShader>. Checking shader </Root/Scenes/Scene/Children/hello/Generated/Text/Material/PBRShader>. Checking prim </Root/Scenes/Scene/Children/hello/Children>. Checking prim </Root/Scenes/Scene/Children/Box>. Checking prim </Root/Scenes/Scene/Children/Box/Generated>. Checking prim </Root/Scenes/Scene/Children/Box/Generated/Mesh0>. Checking prim </Root/Scenes/Scene/Children/Box/Generated/Mesh0/Mesh0>. Checking prim </Root/Scenes/Scene/Children/Box/Generated/Mesh0/Material>. Checking prim </Root/Scenes/Scene/Children/Box/Generated/Mesh0/Material/PBRShader>. Checking shader </Root/Scenes/Scene/Children/Box/Generated/Mesh0/Material/PBRShader>. Checking prim </Root/Scenes/Scene/Children/Box/Children>. Checking prim </Root/Scenes/Scene/Children/Box/PhysicsMaterial_Box>. Found material bindings but no MaterialBindingAPI applied on the prim </Root/Scenes/Scene/sceneGroundPlane>. (fails 'MaterialBindingAPIAppliedChecker') Found material bindings but no MaterialBindingAPI applied on the prim </Root/Scenes/Scene/Children/hello/Generated/Text>. (fails 'MaterialBindingAPIAppliedChecker') Found material bindings but no MaterialBindingAPI applied on the prim </Root/Scenes/Scene/Children/Box>. (fails 'MaterialBindingAPIAppliedChecker') Found material bindings but no MaterialBindingAPI applied on the prim </Root/Scenes/Scene/Children/Box/Generated/Mesh0>. (fails 'MaterialBindingAPIAppliedChecker') Failed!

I would like to receive some guidance and discussion on the ideas implemented with RealityKit.

I'm trying to position an Entity with inverse kinematics while dragging the handle only, but use forward kinematics (pose jointTransforms) otherwise. The System, Components, Gestures and Rig all seem to work individually. My approach is to add the IKComponent when dragging starts on the handle and removing the IKComponent it is released. The switch into IK works, but when removing the IKComponent the App crashes * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x8) * frame #0: 0x00000001aa5bb188 CoreRE`(anonymous namespace)::IKComponentSolverWrapper::getSolver() + 60 frame #1: 0x00000001aa5bafb0 CoreRE`re::internal::ikParametersNodeCallback(re::Slice<re::StringID>, re::Slice<re::RigDataValue>, re::Slice<re::StringID>, re::MutableSlice<re::RigDataValue>, void*) + 48 frame #2: 0x00000001aa52d090 CoreRE`re::(anonymous namespace)::resolveEvaluationContextCallback(re::EvaluationContext&, void*) + 152 frame #3: 0x00000001aa68c024 CoreRE`re::(anonymous namespace)::$_76::__invoke(re::Slice<unsigned long>, re::(anonymous namespace)::RegisterTable&) + 1080 frame #4: 0x00000001aa678c94 CoreRE`re::EvaluationModelSingleThread::evaluate(re::EvaluationContextSlices&) + 1188 frame #5: 0x00000001aa866984 CoreRE`re::SkeletalPoseRuntimeData::executeEvaluationTree() + 136 frame #6: 0x00000001aadf37ec CoreRE`re::ecs2::SkeletalPoseComponent::calculateSkeletalPoseBufferWithRig(re::ecs2::MeshComponent*, re::ecs2::RigComponent*, re::ecs2::SkeletalPoseBufferComponent*) + 492 frame #7: 0x00000001aadf4a84 CoreRE`re::ecs2::SkeletalPoseComponentStateImpl::processPreparingComponents(re::ecs2::System::UpdateContext const&, re::ecs2::BasicComponentStateSceneData<re::ecs2::SkeletalPoseComponent>*, re::ecs2::ComponentBuckets<re::ecs2::SkeletalPoseComponent>::BucketIteration, void*) + 268 frame #8: 0x00000001aadf54b0 CoreRE`re::ecs2::SkeletalPoseSystem::update(re::ecs2::System::UpdateContext) const + 732 frame #9: 0x00000001aaed3e54 CoreRE`re::internal::Callable<re::ecs2::ECSManager::configurePhaseECSSystems(re::Scheduler::ScheduleDescriptor&, re::ecs2::ECSSystemGroup, unsigned long)::$_1, void (float)>::operator()(float&&) const + 168 frame #10: 0x00000001ab40eda4 CoreRE`re::Scheduler::executePhase(unsigned long) + 440 frame #11: 0x00000001aa6a3b74 CoreRE`re::Engine::executePhase(re::FramePhase) + 144 frame #12: 0x000000023173de9c RealitySystemSupport`RCPSharedSimulationExecuteUpdate + 64 frame #13: 0x00000002276c9820 MRUIKit`__65-[MRUISharedSimulation _doJoinWithConnectionConfiguration:error:]_block_invoke.35 + 168 frame #14: 0x00000002276c8530 MRUIKit`__addCAPreFenceHandler_block_invoke + 32 frame #15: 0x000000018af22058 QuartzCore`CA::Transaction::run_commit_handlers(CATransactionPhase) + 112 frame #16: 0x000000018aef2ad4 QuartzCore`CA::Context::commit_transaction(CA::Transaction*, double, double*) + 592 frame #17: 0x000000018af21898 QuartzCore`CA::Transaction::commit() + 652 frame #18: 0x000000018af22dac QuartzCore`CA::Transaction::flush_as_runloop_observer(bool) + 68 frame #19: 0x0000000185a26820 UIKitCore`_UIApplicationFlushCATransaction + 48 frame #20: 0x0000000184f97af0 UIKitCore`_UIUpdateSequenceRun + 76 frame #21: 0x0000000185954290 UIKitCore`schedulerStepScheduledMainSection + 168 frame #22: 0x00000001859536d8 UIKitCore`runloopSourceCallback + 80 frame #23: 0x00000001804157fc CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 24 frame #24: 0x0000000180415744 CoreFoundation`__CFRunLoopDoSource0 + 172 frame #25: 0x0000000180414eb0 CoreFoundation`__CFRunLoopDoSources0 + 232 frame #26: 0x000000018040f454 CoreFoundation`__CFRunLoopRun + 788 frame #27: 0x000000018040ecd4 CoreFoundation`CFRunLoopRunSpecific + 552 frame #28: 0x0000000190104b70 GraphicsServices`GSEventRunModal + 160 frame #29: 0x0000000185a27e30 UIKitCore`-[UIApplication _run] + 796 frame #30: 0x0000000185a2c058 UIKitCore`UIApplicationMain + 124 frame #31: 0x00000001d29558b4 SwiftUI`closure #1 (Swift.UnsafeMutablePointer<Swift.Optional<Swift.UnsafeMutablePointer<Swift.Int8>>>) -> Swift.Never in SwiftUI.KitRendererCommon(Swift.AnyObject.Type) -> Swift.Never + 164 frame #32: 0x00000001d29555dc SwiftUI`SwiftUI.runApp<τ_0_0 where τ_0_0: SwiftUI.App>(τ_0_0) -> Swift.Never + 84 frame #33: 0x00000001d265ecdc SwiftUI`static SwiftUI.App.main() -> () + 164 frame #34: 0x000000010303f1c4 Playground.debug.dylib`static PlaygroundApp.$main() at <compiler-generated>:0 frame #35: 0x000000010303f290 Playground.debug.dylib`main at PlaygroundApp.swift:7:8 frame #36: 0x0000000102f6d410 dyld_sim`start_sim + 20 frame #37: 0x000000010312e274 dyld`start + 2840 Is there a workaround or another way to switch between IK and FK?

Hey, I am having issues getting my Material X shaders to work properly in Reality Composer Pro that I've authored in Houdini. The shader is very simple. It starts with a tiled image node that is written to the diffuse color of the preview surface node. This node is called mtxltileimage2. When I create a tiled image node in RCP and configure it to have the same parameter values I get the texture to show up correctly. This node is called TiledImage. One difference I can identify is that the second node has a grey icon whereas the first node has a blue icon. Could this be related to this issue? Here is the USD viewer output for the two variants of the tiled image node. Any pointers, misconceptions and help would be greatly appreciated. My goal is to be able and author these shaders in Houdini and import them into RCP. Trying to figure out the right pipeline for this workflow.

https://developer.apple.com/documentation/arkit/arkit_in_ios/specifying_a_lighting_environment_in_ar_quick_look How can I disable it? or at least use a custom texture that's just black? I don't see the purpose of having the real-time environment probe that captures IBL, but always add this fake studio IBL that you can't remove...

Starting with iOS 18.0 beta 1, I've noticed that RealityKit frequently crashes in the simulator when an app launches and presents an ARView. I was able to create a small sample app with repro steps that demonstrates the issue, and I've submitted feedback: FB16144085 I've included a crash log with the feedback. If possible, I'd appreciate it if an Apple engineer could investigate and suggest a workaround. It's awkward to be restricted to the iOS 17 simulator, which does not exhibit this behavior. Please let me know if there's anything I can do to help. Thank you.

If I have one portal on the ceiling and one on the floor, can a tall Entity cross multiple portals at once? Will the opposing portal directions cause it to fail? No matter what I try for the crossingMode and clippingMode of the PortalComponent I can only get it to fully work for one portal at a time. I have tried flipping the normals for the crossingMode and clippingMode planes. I have also tried creating a ceiling portal plane with inverted normals. It seems like whatever Entity is passing through a portal has one portal it wants to deal with at a time and that's it. My other option is to create portals using occlusion but I prefer the simplest way.

Hi, I wanted to do something quite simple: Put a box on a wall or on the floor. My box: let myBox = ModelEntity( mesh: .generateBox(size: SIMD3<Float>(0.1, 0.1, 0.01)), materials: [SimpleMaterial(color: .systemRed, isMetallic: false)], collisionShape: .generateBox(size: SIMD3<Float>(0.1, 0.1, 0.01)), mass: 0.0) For that I used Plane Detection to identify the walls and floor in the room. Then with SpatialTapGesture I was able to retrieve the position where the user is looking and tap. let position = value.convert(value.location3D, from: .local, to: .scene) And then positioned my box myBox.setPosition(position, relativeTo: nil) When I then tested it I realized that the box was not parallel to the wall but had a slightly inclined angle. I also realized if I tried to put my box on the wall to my left the box was placed perpendicular to this wall and not placed on it. After various searches and several attempts I ended up playing with transform.matrix to identify if the plane is wall or a floor, if it was in front of me or on the side and set up a rotation on the box to "place" it on the wall or a floor. let surfaceTransform = surface.transform.matrix let surfaceNormal = normalize(surfaceTransform.columns.2.xyz) let baseRotation = simd_quatf(angle: .pi, axis: SIMD3<Float>(0, 1, 0)) var finalRotation: simd_quatf if acos(abs(dot(surfaceNormal, SIMD3<Float>(0, 1, 0)))) < 0.3 { logger.info("Surface: ceiling/floor") finalRotation = simd_quatf(angle: surfaceNormal.y > 0 ? 0 : .pi, axis: SIMD3<Float>(1, 0, 0)) } else if abs(surfaceNormal.x) > abs(surfaceNormal.z) { logger.info("Surface: left/right") finalRotation = simd_quatf(angle: surfaceNormal.x > 0 ? .pi/2 : -.pi/2, axis: SIMD3<Float>(0, 1, 0)) } else { logger.info("Surface: front/back") finalRotation = baseRotation } Playing with matrices is not really my thing so I don't know if I'm doing it right. Could you tell me if my tests for the orientation of the walls are correct? During my tests I don't always correctly identify whether the wall is in front or on the side. Is this generally the right way to do it? Is there an easier way to do this? Regards Tof

Hey I wanted to make an app that tracks changes in the room and room lightning and I was wondering if its possible to use VirtualEnvironmentProbeComponent to obtain the EnvironmentResource image and store it? If so are there any example of similar operation I could use? Thank you!

I have a 3D model with morphing animation that works correctly in Blender. I exported this model as a USDZ file and tried to display it in an Xcode-developed visionOS app, but the morphing animation does not play. What I Have Tried: Morphing animation works correctly in Blender. After exporting to USDZ, the morphing animation does not play in the Xcode app. Linear motion animations (such as object movement) work fine. Behavior in Reality Converter: GLB files do not display. USDZ files load, but morphing animations do not play. What I Want to Know: Is there a way to play morphing animations in an Xcode-developed app? Does RealityKit support morphing animations? Can morphing animations be played in an Xcode-developed app? If RealityKit does not support morphing animations, what alternative methods can be used to play them? I am looking for a way to use the existing animations without recreating them. Additional Information: I have both the Blender file (where animations work) and the USDZ file (where animations do not play). I am developing a visionOS app using Xcode. Any advice or solutions would be greatly appreciated. Thank you in advance!

The farther away the center of a large entity is, the less accurate the positioning is? For example I am changing only the y-axis position of an entity that is tens of meters long, but i notice x and z drifting slowly the farther away the center of the entity is. I would not expect the x and z to move. It might be compounding rounding errors somewhere, or maybe the RealityKit engine is deciding not to be super precise about distant objects? Otherwise I just have a bug somewhere.