RealityKit captureHighResolutionFrame from session is broken on iOS26?

A bit of background on what our app is doing:

  • We have a RealityKit ARView session running.
  • During this period we place objects in RealityKit.
  • At some point user can "take photo" and we use session.captureHighResolutionFrame to capture a frame.
  • We then use captured frame and frame.camera.projectPoint to project my objects back to 2D

Issue we found is that on devices that have iOS26, first photo user takes and the first frame received from session.captureHighResolutionFrame gives incorrect CGPoint for frame.camera.projectPoint. If user takes the second photo with the same camera phostion, second frame received from session.captureHighResolutionFrame gives correct CGPoint for frame.camera.projectPoint

I notices some difference between first and subsequent frames that i believe is corresponding with the issue. Yaw value of camera (frame.camera.eulerAngles.y) on first frame is not correct ( inconsistent with any subsequent frame)

I also created a small example app and i followed Building an Immersive Experience with RealityKit example to create it. The issue exists in this app for iOS26, while iOS18.* has consistent values between first and subsequent captured frames.

Note: The yaw value seems to differ more if we start session in portrait but take photo in landscape.

Example result for 3 captured frames:

Frame captured with yaw: 1.4855177402496338
Frame captured with yaw: -0.08803760260343552
Frame captured with yaw: -0.08179682493209839

Example code:

class CustomARView: ARView, ARSessionDelegate {
    required init(frame: CGRect) { super.init(frame: frame) }
    required init?(coder decoder: NSCoder) { fatalError("init(coder:) has not been implemented")}
    
    func setup() {
        let singleTap = UITapGestureRecognizer(target: self, action: #selector(handleTap))
        addGestureRecognizer(singleTap)
    }
    @objc
    func handleTap(_ gestureRecognizer: UIGestureRecognizer) {
        Task {
            do {
                let frame = try await session.captureHighResolutionFrame()
                print("Frame captured with yaw: \(Double(frame.camera.eulerAngles.y))")
            } catch { }
        }
    }
}

struct CustomARViewUIViewRepresentable: UIViewRepresentable {
    func makeUIView(context: Context) -> some UIView {
        let arView = CustomARView(frame: .zero)
        arView.setup()
        return arView
    }
    func updateUIView(_ uiView: UIViewType, context: Context) { }
}

struct ContentView: View {
    var body: some View {
        CustomARViewUIViewRepresentable()
            .frame(maxWidth: .infinity, maxHeight: .infinity)
            .ignoresSafeArea()
    }
}
Answered by DTS Engineer in 859727022

Hello @NemanjaScandit,

Are you able to reproduce this issue on the iOS 26.1 beta?

--Greg

Accepted Answer

Hello @NemanjaScandit,

Are you able to reproduce this issue on the iOS 26.1 beta?

--Greg

Hey @DTS Engineer Greg,

Yes, the issue is fixed on iOS 26.1 beta, thanks for that!

What would your recommendation be on fixing this for devices still on iOS 26.0? Our current fix it to repeat the capture few times until the angle difference between session.currentFrame() and session.captureHighResolutionFrame() is low enough that we assume its correct. If you are aware of underling issue, can you recommend a more stable way to fix this?

Nemanja

Hey @NemanjaScandit,

I am aware of the issue. On iOS 26.0, it should only occur for the first capture, so a workaround (for iOS 26.0) would be to ignore the transform associated with that first capture.

--Greg

RealityKit captureHighResolutionFrame from session is broken on iOS26?
 
 
Q