We followed this documentation https://developer.apple.com/documentation/compositorservices/drawing_fully_immersive_content_using_metal to display a fully immersive map using our metal rendering engine, which worked great. But this part of the article: https://developer.apple.com/documentation/compositorservices/drawing_fully_immersive_content_using_metal#4193614 mentions how to use the onSpatialEvent callback to receive gesture events. We are receiving the gesture events but the location property of the event (https://developer.apple.com/documentation/swiftui/spatialeventcollection/event/location) is always coming back as (x: 0, y:0) which is not helpful. We are unable to get a single valid location of any gesture, therefore, we are unable to hook up these gestures. We tried this on a simulator and a Vision Pro device.
Fully immersive content using Metal is not getting the correct gesture locations
In the case of CompositorServices there is no location for the spatialEvent. But you get the pose3D of the pinch:
https://developer.apple.com/documentation/swiftui/spatialeventcollection/event/inputdevicepose-swift.struct/pose3d
As well as the gaze ray when the pinch began: https://developer.apple.com/documentation/swiftui/spatialeventcollection/event/selectionray
Interesting, I'll try that out. Thank you.