RealityKit captureHighResolutionFrame from session is broken on iOS26?

A bit of background on what our app is doing:

  • We have a RealityKit ARView session running.
  • During this period we place objects in RealityKit.
  • At some point user can "take photo" and we use session.captureHighResolutionFrame to capture a frame.
  • We then use captured frame and frame.camera.projectPoint to project my objects back to 2D

Issue we found is that on devices that have iOS26, first photo user takes and the first frame received from session.captureHighResolutionFrame gives incorrect CGPoint for frame.camera.projectPoint. If user takes the second photo with the same camera phostion, second frame received from session.captureHighResolutionFrame gives correct CGPoint for frame.camera.projectPoint

I notices some difference between first and subsequent frames that i believe is corresponding with the issue. Yaw value of camera (frame.camera.eulerAngles.y) on first frame is not correct ( inconsistent with any subsequent frame)

I also created a small example app and i followed Building an Immersive Experience with RealityKit example to create it. The issue exists in this app for iOS26, while iOS18.* has consistent values between first and subsequent captured frames.

Note: The yaw value seems to differ more if we start session in portrait but take photo in landscape.

Example result for 3 captured frames:

Frame captured with yaw: 1.4855177402496338
Frame captured with yaw: -0.08803760260343552
Frame captured with yaw: -0.08179682493209839

Example code:

class CustomARView: ARView, ARSessionDelegate {
    required init(frame: CGRect) { super.init(frame: frame) }
    required init?(coder decoder: NSCoder) { fatalError("init(coder:) has not been implemented")}
    
    func setup() {
        let singleTap = UITapGestureRecognizer(target: self, action: #selector(handleTap))
        addGestureRecognizer(singleTap)
    }
    @objc
    func handleTap(_ gestureRecognizer: UIGestureRecognizer) {
        Task {
            do {
                let frame = try await session.captureHighResolutionFrame()
                print("Frame captured with yaw: \(Double(frame.camera.eulerAngles.y))")
            } catch { }
        }
    }
}

struct CustomARViewUIViewRepresentable: UIViewRepresentable {
    func makeUIView(context: Context) -> some UIView {
        let arView = CustomARView(frame: .zero)
        arView.setup()
        return arView
    }
    func updateUIView(_ uiView: UIViewType, context: Context) { }
}

struct ContentView: View {
    var body: some View {
        CustomARViewUIViewRepresentable()
            .frame(maxWidth: .infinity, maxHeight: .infinity)
            .ignoresSafeArea()
    }
}
Answered by DTS Engineer in 859727022

Hello @NemanjaScandit,

Are you able to reproduce this issue on the iOS 26.1 beta?

--Greg

Accepted Answer

Hello @NemanjaScandit,

Are you able to reproduce this issue on the iOS 26.1 beta?

--Greg

RealityKit captureHighResolutionFrame from session is broken on iOS26?
 
 
Q