Hi,
I am in the process of implementing SharePlay into our app. The shared experience opens an Immersive Space and we set systemCoordinator.configuration.supportsGroupImmersiveSpace = true Now visionOS establishes a shared coordinate space for the immersive space. From the docs:
To achieve consistent positioning of RealityKit entities across multiple devices in an immersive space during a SharePlay session
There are cases where we want to position content in front of the user (independent of the shared session, and for each user individually). Normally to do that we use the transform retrieved via worldTrackingProvider.queryDeviceAnchor.originFromAnchorTransform
to position content in front of the user (plus some Z Offset and smooth interpolation).
This works fine in non-SharePlay instances and the device transform is where I would expect it to be but during the FaceTime call deviceAnchor.originFromAnchorTransform seems to use the shared origin of the immersive space and then I end up with a transform that might be offset.
Here is a video of the issue in action: https://streamable.com/205r2p
The blue rect is place using AnchorEntity(.head, trackingMode: .continuous). This works regardless of the call and the entity is always placed based on the head position. The green rect is adjusted on every frame using the transform I get from worldTrackingProvider.queryDeviceAnchor. As you can see it's offset.
Is there any way I can query query this transform locally for the user during a FaceTime call? Also I would like to know if it's possible to disable this automatic entity transform syncing behavior? Setting entity.synchronization = nil results in the entity not showing up at all. https://developer.apple.com/documentation/realitykit/synchronizationcomponent
Is SynchronizationComponent only relevant for the legacy MultiPeerConnectivity approach?
Thank you!
Okay I just wasted over 24h solving this, so hopefully this will save someone some pain: On visionOS 26 if you have an ImmersiveSpace and a regular WindowGroup in you app and want to use SharePlay with the ImmersiveSpace, you should set groupActivityAssociation on the view within the ImmersiveSpace:
ImmersiveSpace(id: "immersiveSpaceID", for: AppScene.self) { $scene in
let view = makeMyView()
if #available(visionOS 26.0, *) {
view
.groupActivityAssociation(.primary("immersiveSpaceID")) // if you don't set this you might be in for a bad time
} else {
view
}
}
If you don't do this, and query the device anchor during your SharePlay session, you will get a transform that is offset – supposedly based on the shared displacement. 💀 Once groupActivityAssociation is set, everything works just as expected. Feels like a bug to me, but at least it seems to be undocumented behavior.
https://developer.apple.com/documentation/SwiftUI/View/groupActivityAssociation(_:)
In my particular case I have a window that I hide but not dismiss while the ImmersiveSpace is active, and apparently by default the window will get the primary association.