Building Dasher (eye-gaze text-entry system for people with motor impairments — ALS, cerebral palsy, etc.) for visionOS. See https://github.com/dasher-project/Dasher-Apple The app needs continuous (x, y) gaze coordinates to steer a zooming alphabet interface. Vision Pro has incredible eye tracking hardware built in — this should be the perfect platform — but I can't get gaze coordinates through any API I've tried. What I've tested (all deliver zero events in a windowed app):
- UIHoverGestureRecognizer — Apple's docs say it "detects when the user is looking at a view" on visionOS. Added it to a UIView inside UIViewRepresentable. Looked at the view for 2+ minutes on real hardware. Console.app shows zero began/changed/ended callbacks.
- .onContinuousHover (SwiftUI) — zero .active phases delivered, whether attached to the canvas view or the ZStack root.
- UIPointerInteraction with a delegate — regionFor callback never fires.
The system clearly is tracking gaze — I can see beginScrollingWithRegion events in Console.app with the correct window rect, and the visual hover highlight appears on buttons. But the coordinates are routed to scroll handling, not exposed as a usable pointer position.
What does work: pinch-and-drag follows the hand position (where the pinch is in 3D space), not gaze. That's fine for general apps but doesn't help users who need eye-only control.
What I'm wondering:
- Has anyone successfully received continuous gaze coordinates in a windowed (non-immersive) visionOS app? If so, which API?
- Is UIHoverGestureRecognizer supposed to work on UIView inside UIViewRepresentable, or does it require a pure UIKit view hierarchy?
- Is there an ARKit provider for eye tracking that I'm missing? I see HandTrackingProvider but no EyeTrackingProvider in the capability list.
- Could the issue be that my SwiftUI overlay (toolbar VStack) is intercepting hover events before they reach the canvas underneath? If so, how do you make a SwiftUI overlay pass-through for hover?
Dasher is a well-established accessibility tool that lets people type using only their eyes. Its the fastest text entry method for eyegaze and head controlled systems. On every other platform (iOS with external trackers, Windows with Tobii, etc.) it reads continuous gaze coordinates and zooms toward wherever the user is looking. Vision Pro should be the best hardware for this — the tracking is already there, we just can't read it. If there's no public API for this, I'd love to know so I can scrap my feedback assistant entry!. If there IS a way and I've missed it, I'd be grateful for a pointer. Thanks!