RealityKit

RSS for tag

Simulate and render 3D content for use in your augmented reality apps using RealityKit.

RealityKit Documentation

Posts under RealityKit subtopic

Post

Replies

Boosts

Views

Created

Custom EntityAction - different behaviour VisionOS 2.6 vs 26
I implemented an EntityAction to change the baseColor tint - and had it working on VisionOS 2.x. import RealityKit import UIKit typealias Float4 = SIMD4<Float> extension UIColor { var float4: Float4 { if cgColor.numberOfComponents == 4, let c = cgColor.components { Float4(Float(c[0]), Float(c[1]), Float(c[2]), Float(c[3])) } else { Float4() } } } struct ColourAction: EntityAction { // MARK: - PUBLIC PROPERTIES let startColour: Float4 let targetColour: Float4 // MARK: - PUBLIC COMPUTED PROPERTIES var animatedValueType: (any AnimatableData.Type)? { Float4.self } // MARK: - INITIATION init(startColour: UIColor, targetColour: UIColor) { self.startColour = startColour.float4 self.targetColour = targetColour.float4 } // MARK: - PUBLIC STATIC FUNCTIONS @MainActor static func registerEntityAction() { ColourAction.subscribe(to: .updated) { event in guard let animationState = event.animationState else { return } let interpolatedColour = event.action.startColour.mixedWith(event.action.targetColour, by: Float(animationState.normalizedTime)) animationState.storeAnimatedValue(interpolatedColour) } } } extension Entity { // MARK: - PUBLIC FUNCTIONS func changeColourTo(_ targetColour: UIColor, duration: Double) { guard let modelComponent = components[ModelComponent.self], let material = modelComponent.materials.first as? PhysicallyBasedMaterial else { return } let colourAction = ColourAction(startColour: material.baseColor.tint, targetColour: targetColour) if let colourAnimation = try? AnimationResource.makeActionAnimation(for: colourAction, duration: duration, bindTarget: .material(0).baseColorTint) { playAnimation(colourAnimation) } } } This doesn't work in VisionOS 26. My current fix is to directly set the material base colour - but this feels like the wrong approach: @MainActor static func registerEntityAction() { ColourAction.subscribe(to: .updated) { event in guard let animationState = event.animationState, let entity = event.targetEntity, let modelComponent = entity.components[ModelComponent.self], var material = modelComponent.materials.first as? PhysicallyBasedMaterial else { return } let interpolatedColour = event.action.startColour.mixedWith(event.action.targetColour, by: Float(animationState.normalizedTime)) material.baseColor.tint = UIColor(interpolatedColour) entity.components[ModelComponent.self]?.materials[0] = material animationState.storeAnimatedValue(interpolatedColour) } } So before I raise this as a bug, was I doing anything wrong in the former version and got lucky? Is there a better approach?
0
0
47
6d
RealityKit captureHighResolutionFrame from session is broken on iOS26?
A bit of background on what our app is doing: We have a RealityKit ARView session running. During this period we place objects in RealityKit. At some point user can "take photo" and we use session.captureHighResolutionFrame to capture a frame. We then use captured frame and frame.camera.projectPoint to project my objects back to 2D Issue we found is that on devices that have iOS26, first photo user takes and the first frame received from session.captureHighResolutionFrame gives incorrect CGPoint for frame.camera.projectPoint. If user takes the second photo with the same camera phostion, second frame received from session.captureHighResolutionFrame gives correct CGPoint for frame.camera.projectPoint I notices some difference between first and subsequent frames that i believe is corresponding with the issue. Yaw value of camera (frame.camera.eulerAngles.y) on first frame is not correct ( inconsistent with any subsequent frame) I also created a small example app and i followed Building an Immersive Experience with RealityKit example to create it. The issue exists in this app for iOS26, while iOS18.* has consistent values between first and subsequent captured frames. Note: The yaw value seems to differ more if we start session in portrait but take photo in landscape. Example result for 3 captured frames: Frame captured with yaw: 1.4855177402496338 Frame captured with yaw: -0.08803760260343552 Frame captured with yaw: -0.08179682493209839 Example code: class CustomARView: ARView, ARSessionDelegate { required init(frame: CGRect) { super.init(frame: frame) } required init?(coder decoder: NSCoder) { fatalError("init(coder:) has not been implemented")} func setup() { let singleTap = UITapGestureRecognizer(target: self, action: #selector(handleTap)) addGestureRecognizer(singleTap) } @objc func handleTap(_ gestureRecognizer: UIGestureRecognizer) { Task { do { let frame = try await session.captureHighResolutionFrame() print("Frame captured with yaw: \(Double(frame.camera.eulerAngles.y))") } catch { } } } } struct CustomARViewUIViewRepresentable: UIViewRepresentable { func makeUIView(context: Context) -> some UIView { let arView = CustomARView(frame: .zero) arView.setup() return arView } func updateUIView(_ uiView: UIViewType, context: Context) { } } struct ContentView: View { var body: some View { CustomARViewUIViewRepresentable() .frame(maxWidth: .infinity, maxHeight: .infinity) .ignoresSafeArea() } }
3
1
357
3d