Currently VisionOS has WorldAnchors which are a great, privacy-preserving method to affix and localize entities into physical spaces. These are great for apps that work in Mixed immersive mode as well as augmented layer or virtual overlays onto physical spaces. However, they have some fundamental limitations when compared to iOS’s ARWorldMap which limit their capabilities as truly persistent WorldAnchors, which the platform would greatly benefit from.
The problem - Impermanence:
WorldAnchors are on-device only, stored in data at a system level and expose their ID for collection and usa within apps. If the app which created them is removed, or if the device is rese, those saved World Anchors are removed and lost forever. Their ID will no longer point to an anchor in physical space and it is as if the anchor has been deleted or never existed. This means that anchors effectively only temporarily exist on a single device while that specific device is in an undisturbed state. A second device using that same app with interest in utilizing that anchor is not available.
With no way to export this Anchor, save it, re-load it either inside an app bundle, from a server, on-site data storage, etc. the anchor is 'trapped' on the device which creates it. ARWorldMap can be saved to a file, and acquired in many ways which circumvents this problem, and allows it to exist as a single source of truth for that particular physical environment that any iOS or iPadOS device can localize to as long as they have the ARWorldMap.
Imperfect Workaround #1 - SharedAnchors:
After setting a WorldAnchor on Device #1, start a local S harePlay session with Device #1 & #2 in the same physical space. Then, use a 'Shared Anchor' with transform offset data matching the WorldAnchor. On Device #2, save that offset as a new WorldAnchor. This new WorldAnchor will have the same transform effectively matching the original, though it will have a new ID and is entirely separate.
Problem with Workaround #1:
This requires two VisionOS devices to be in the same physical location simultaneously, have an active SharePlay session between the two devices, then conduct a sharing operation. This means that if Device #1 does not happen to be available in the physical space at the same time as Device #2 for any reason, Device #2 will never have access to this anchor. In cases such as a public or industrial space, it is not realistically viable to always have a person with Device #1 available at all times. There are many situations where Device #2 would want to enter the space and observe the anchor while they are alone, and would never have access to this anchor.
Workaround #2 - iOS / iPadOS Middleman:
Once a visionOS WorldAnchor is created on Device #1, that same user can manually co-locate an iOS or iPadOS device, with some shared session (for example, using ImageTrigger on the iOS/iPadOS device's display which the visionOS device reads, then determines an offset for). Then, create an ARWorldMap on that co-located iOS/iPadOS device, and save it, then serve that ARWorldMap. Then to localize visionOS Device #2, enter the space and first localize an iOS / iPadOS device with the ARWorldMap. Then, manually co-locate visionOS Device #2 with the iOS/iPadOS device and access the offset. Then save this offset as WorldAnchor on Device #2, at this point the iOS/iPadOS device is no longer needed.
Problems With Workaround #2:
At a minimum, this requires 3 devices, or more realistically 4 if Device #1 and #2 are not in the space at the same time. This is not only a very poor user experience to have to operate an iOS/iPadOS device simultaneously while wearing visionOS device, but it is also very complicated and a several step process that is not intuitive to most users.
The accuracy of this process is extremely poor, as tracking an image from the screen of an iOS/iPadOS device will never be as precise as the internal tracking system on VisionOS. It will always have SOME margin for drift and error, which can result in the anchor being very far offset from the intended anchor position. This is antithetical to the purpose of WorldAnchors, as the delta can be so inconsistent that Device #2 will observe content attached to the anchor at an incorrect location, which could lead to unintended user behaviour when interacting with this content potentially moving attached content to look correct for them, which would negatively impact all other devices viewing the content.
Desired Solution:
An optimal upgrade to visionOS WorldAnchors would;
Use the same underlying tracking & relocalization system that is currently in use
Allow for exporting of the WorldAnchor's data in an encrypted, privacy preserving way (not simply a point cloud) which could be saved and shared as a file similar to ARWorldMap, then used for relocalization on ANY device with access to that file and application.
Be interoperable across platforms where iOS, iPadOS, visionOS and any other spatial-capable platform can use that single file, and localize themselves in the physical space. This would unify ARKit WorldAnchors for all platforms, ensuring that the same physical space can be localized to by all devices, anchors can be created by all devices, and the content would exist in the same position on all devices. Allowing an iOS or iPadOS device to create a WorldAnchor that is then identical on visionOS, or visa-versa.
These features unlock true persistence in anchoring content to real-world spaces, a critical component of a Spatial Computing platform that maximizes the unique capabilities and benefits of the medium. This creates an upwards path for users to start on iOS / iPadOS today, and upgrade to visionOS in the future.
Accessibility:
Not all users are able to use visionOS for various reasons, including physical, regional and financial circumstances among many many others. There is no reason why someone who is unable to use visionOS for any one of these reasons should be LOCKED-OUT from Spatial Computing applications. Spatial computing is the most human computing medium ever created, and applications need to allow all humans to engage with Spatial Computing experiences regardless of their level of access to visionOS devices.
We as developers want to build everlasting Spatial Computing applications that accentuate the medium, maximize the benefits of Spatial Computing, include / invite all humans regardless of their Accessibility level, and establish virtual content that can outlive the individuals who create it. Please, take this request into serious consideration, as the feature as described would contribute to the Apple Ecosystem being the single greatest Spatial Computing platform of all time (and space), enabling permanent layering of physical spaces, preserving privacy for sensitive data, and maximizing accessibility across the spectrum of humans and devices.
Thank you.
Topic:
Spatial Computing
SubTopic:
ARKit
3
3
95