<!--
{
  "availability" : [
    "visionOS: 27.0.0 -",
    "Xcode: 27.0.0 -"
  ],
  "documentType" : "article",
  "framework" : "visionOS",
  "identifier" : "/documentation/visionOS/working-with-generic-spatial-accessories",
  "metadataVersion" : "0.1.0",
  "role" : "sampleCode",
  "title" : "Working with generic spatial accessories"
}
-->

# Working with generic spatial accessories

Let people place digital replicas of a generic spatial accessory by tracking the accessory with ARKit.

## Overview

Specialized apps become more immersive when they respond to people using purpose-built devices, like medical instruments, steering rigs, or industrial tooling.
<doc://com.apple.documentation/documentation/ARKit> recognizes these devices as *generic spatial accessories*, but they’re distinct from spatial controllers and styli, which have their own dedicated APIs.

Manufacturers create generic spatial accessories by following the [Accessory Design Guidelines](https://developer.apple.com/accessories/Accessory-Design-Guidelines.pdf), paying close attention to the Spatial Accessories section.
ARKit provides precise, low-latency tracking of these accessories in Apple Vision Pro across varied lighting conditions, and continues tracking orientation even when an accessory moves outside the field of view or becomes visually obscured.
<doc://com.apple.documentation/documentation/GameController> also provides input and haptic feedback for these accessories.

In an immersive space, the sample lets people place digital replicas of the accessory.
In a volume, the sample displays a digital replica that mirrors the accessory’s orientation in real time.
Accessory manufacturers can also use the sample to test the tracking, input, and haptic behavior of their accessories.

> Note: Simulator doesn’t support accessory tracking. Run the sample on a physical device.

## Configure the sample code project

The sample works with any generic spatial accessory.
Before building the sample, configure the Xcode project to access the `.referenceaccessory` file for the accessory you want to track.
For step-by-step instructions, see [Setting up access to a reference accessory file](/documentation/visionOS/setting-up-access-to-a-reference-accessory-file).

> Important: If you run the sample before configuring access to a reference accessory file, the sample displays “Unable to resolve the reference accessory.”

## Add the accessory tracking capability

To help protect people’s privacy, visionOS limits app access to spatial accessory data and other sensor data on Apple Vision Pro.
Add the Accessory Tracking capability to your app’s target and provide a usage description that explains how your app uses spatial accessory data.
People see that description when the system prompts for access to accessory-tracking data.
For more information on app capabilities, see <doc://com.apple.documentation/documentation/Xcode/adding-capabilities-to-your-app>.

## Obtain authorization to track accessories

To read an accessory’s transform, your app needs Accessory Tracking authorization.
The sample monitors the authorization status so it can change the availability of features that require the transform.

At startup, the sample uses an <doc://com.apple.documentation/documentation/ARKit/ARKitSession> to query the current authorization status:

```swift
// AccessoryModel.swift

authorizationStatus = await arkitSession.queryAuthorization(for: [.accessoryTracking])[.accessoryTracking] ?? .notDetermined
```

The sample then observes authorization changes on the same session so the app can respond when people grant or revoke permission:

```swift
// AccessoryModel.swift

for await event in arkitSession.events {
    switch event {
    case .authorizationChanged(.accessoryTracking, let status):
        authorizationStatus = status
    default:
        break
    }
}
```

> Note: Your app doesn’t need authorization to anchor content to an accessory, respond to its input, or play haptics through the accessory.

## Discover connected accessories

Before your app can anchor content to an accessory, track it, or respond to its input, your app needs a reference to the <doc://com.apple.documentation/documentation/GameController/GCSpatialAccessory> instance for the connected accessory.

The sample uses a custom `AccessoryModel` to manage this reference, tracking the most recently connected accessory throughout the app’s life cycle.

At startup, `AccessoryModel` checks <doc://com.apple.documentation/documentation/GameController/GCSpatialAccessory/spatialAccessories> for an already-connected accessory:

```swift
// AccessoryModel.swift

if let accessory = GCSpatialAccessory.spatialAccessories.first {
    self.accessoryDevice = accessory
}
```

To handle accessories that connect later, `AccessoryModel` listens for connect notifications:

```swift
// AccessoryModel.swift

private func observeAccessoryConnectNotifications() async {
    for await notification in NotificationCenter.default.notifications(named: .GCSpatialAccessoryDidConnect) {
        if let accessory = notification.object as? GCSpatialAccessory {
            self.accessoryDevice = accessory
        }
    }
}
```

To release the reference when the active accessory disconnects, `AccessoryModel` listens for disconnect notifications:

```swift
// AccessoryModel.swift

private func observeAccessoryDisconnectNotifications() async {
    for await notification in NotificationCenter.default.notifications(named: .GCSpatialAccessoryDidDisconnect) {
        if let accessory = notification.object as? GCSpatialAccessory,
           accessory == self.accessoryDevice {
            self.accessoryDevice = nil
        }
    }
}
```

## Load a 3D model of the accessory

When a person opens the immersive space, the app displays a semi-transparent digital replica of the accessory.
The accessory manufacturer can include a `.usdz` model of the accessory in the `.referenceaccessory` file.
When an accessory connects, `AccessoryModel` retrieves the model through an <doc://com.apple.documentation/documentation/RealityKit/AnchoringComponent/AccessoryAnchoringSource> and loads it as an <doc://com.apple.documentation/documentation/RealityKit/Entity>.
If the `.referenceaccessory` file doesn’t include a `.usdz`, or loading fails, `AccessoryModel` falls back to a placeholder entity:

```swift
// AccessoryModel.swift

private func loadReferenceEntity(from anchoringSource: AnchoringComponent.AccessoryAnchoringSource) async {
    if let usdzURL = anchoringSource.underlyingAccessory?.usdzFile {
        do {
            referenceEntity = try await Entity(contentsOf: usdzURL)
        } catch {
            logger.warning("Failed to load USDZ file, using missing reference entity fallback: \(error)")
            referenceEntity = Entity.createMissingReferenceEntity()
        }
    } else {
        referenceEntity = Entity.createMissingReferenceEntity()
    }
}
```

## Anchor entities to named accessory locations

The app uses an <doc://com.apple.documentation/documentation/RealityKit/AnchorEntity> to keep the replica locked to the physical accessory’s origin as a person moves the accessory:

```swift
// ImmersiveView.swift

private func createReferenceAnchorEntity(
    for source: AnchoringComponent.AccessoryAnchoringSource,
    trackingMode: AnchoringComponent.TrackingMode
) -> Entity? {
    guard let referenceEntity = appModel.accessoryModel.referenceEntity else {
        return nil
    }

    let anchorEntity = AnchorEntity(
        .accessory(from: source, location: .origin),
        trackingMode: trackingMode,
        physicsSimulation: .none
    )
    anchorEntity.addChild(referenceEntity.clone(recursive: true))
    anchorEntity.components.set(OpacityComponent(opacity: 0.5))

    return anchorEntity
}
```

All generic accessories support anchoring entities to their origin, and some support anchoring to other named locations the manufacturer defines.
The sample queries the <doc://com.apple.documentation/documentation/RealityKit/AnchoringComponent/AccessoryAnchoringSource/accessoryLocations> property on `AccessoryAnchoringSource` to discover an accessory’s supported locations, then presents a toggle for each in `AccessorySettingsForm`.
When a person turns on a location toggle, the sample anchors a white sphere at that location.

The form also presents a picker for the tracking mode.
The tracking mode controls the trade-off between latency and accuracy.
Use <doc://com.apple.documentation/documentation/RealityKit/AnchoringComponent/TrackingMode-swift.struct/continuous> for higher accuracy with increased latency, or <doc://com.apple.documentation/documentation/RealityKit/AnchoringComponent/TrackingMode-swift.struct/predicted> for lower latency with less accuracy.

## Read the accessory’s transform using RealityKit

When a person taps the “Place digital replica” button in `ContentView`, `ImmersiveView` reads the reference anchor entity’s transform, clones the reference entity at that position, and logs the underlying <doc://com.apple.documentation/documentation/ARKit/AccessoryAnchor> to demonstrate how to access it:

```swift
// ImmersiveView.swift

private func placeDigitalReplica() {
    guard let referenceAnchorEntity = referenceRoot.children.first else { return }

    guard let referenceEntity = appModel.accessoryModel.referenceEntity else {
        // ...
        return
    }

    let replica = referenceEntity.clone(recursive: true)
    replica.transform.matrix = referenceAnchorEntity.transformMatrix(relativeTo: nil)

    if let arkitComponent = referenceAnchorEntity.components[ARKitAnchorComponent.self],
       let underlyingAnchor = arkitComponent.anchor as? AccessoryAnchor {
        // Access the anchor associated with an anchor entity.
        logger.debug("Underlying anchor: \(underlyingAnchor)")
    }

    digitalReplicaRoot.addChild(replica)
}
```

Reading an anchor entity’s transform requires a running <doc://com.apple.documentation/documentation/RealityKit/SpatialTrackingSession> configured to track accessories.

Entities parented to an `AnchorEntity` always render at the accessory’s latest pose.
When placing a digital replica, the sample instead clones the reference entity and applies the anchor entity’s transform to the clone.
This pattern introduces a one-frame lag, because by the time RealityKit renders the clone, the accessory has moved to a new pose.
This approach is simple and works well when the one-frame lag is acceptable.
For content that needs continuous low-latency tracking, see the [Track an accessory using ARKit](/documentation/visionOS/working-with-generic-spatial-accessories#Track-an-accessory-using-ARKit) section.

## Track an accessory using ARKit

Use <doc://com.apple.documentation/documentation/ARKit/AccessoryTrackingProvider> when your app needs to choose how and when it reads the accessory’s transform, or when it needs more control over rendering than RealityKit provides.
For example, when your app:

- displays content in a volume, whose fixed bounds can prevent an anchored entity from following the accessory past the edges.
- renders outside of RealityKit.
- predicts the accessory’s position at a future timestamp.

The `AccessoryModel` creates an `AccessoryTrackingProvider` for the connected <doc://com.apple.documentation/documentation/ARKit/Accessory> and runs it on an <doc://com.apple.documentation/documentation/ARKit/ARKitSession>:

```swift
// AccessoryModel.swift

let accessory = try await Accessory(device: accessoryDevice)
let provider = AccessoryTrackingProvider(accessories: [accessory])
try await arkitSession.run([provider])
```

The connected accessory may change during the app’s life cycle.
When it does, the sample updates the running provider rather than stopping and restarting it, using <doc://com.apple.documentation/documentation/ARKit/AccessoryTrackingProvider/updateAccessories(_:)>:

```swift
// AccessoryModel.swift

let accessories = await buildAccessoriesForTrackingProvider()
try await accessoryTrackingProvider.updateAccessories(accessories)
```

With the provider running, the app can respond when a person moves the accessory.
When a person taps the “Show the Volume” button in `ContentView`, the app displays `VolumeView` containing a digital replica that rotates in real time to mirror the physical accessory’s orientation:

```swift
// VolumeView.swift

RealityView { content in
    guard let sourceEntity = appModel.accessoryModel.referenceEntity else { return }
    let clone = sourceEntity.clone(recursive: true)

    content.add(clone)
    // ...
    scaleToFit(clone)

    subscription = content.subscribe(to: SceneEvents.Update.self) { _ in
        if let accessoryAnchor = appModel.accessoryModel.queryLatestAccessoryAnchor(),
           let rotation = accessoryAnchor.coordinateSpace(correction: .rendered).ancestorFromSpaceTransformFloat().rotation {
            clone.transform.rotation = unsafe rotation.quaternion
        } else {
            clone.transform.rotation = simd_quatf()
        }
    }
}
```

Typically, apps iterate the provider’s <doc://com.apple.documentation/documentation/ARKit/AccessoryTrackingProvider/anchorUpdates> to react to accessory anchor updates, for example:

```swift
// Reference snippet: This isn't part of the project's sample code.

for await update in provider.anchorUpdates {
    switch update.event {
    case .added, .updated:
        let anchor = update.anchor
        // Use the anchor.
    case .removed:
        // Clean up the anchor.
    }
}
```

However, the sample takes a different approach. Rather than relying on <doc://com.apple.documentation/documentation/ARKit/AccessoryTrackingProvider/anchorUpdates>, this project subscribes to <doc://com.apple.documentation/documentation/RealityKit/SceneEvents/Update> from a render loop in RealityKit.
This bypasses an update cycle in <doc://com.apple.documentation/documentation/SwiftUI> which otherwise invalidates views at the provider’s high frame rate.

To keep the replica’s rotation in sync with the accessory, `VolumeView` reads the accessory’s pose each frame by calling `queryLatestAccessoryAnchor`.
In predicted mode, `queryLatestAccessoryAnchor` uses <doc://com.apple.documentation/documentation/ARKit/AccessoryTrackingProvider/predictAnchor(for:at:)> to estimate the accessory’s pose a few frames into the future:

```swift
// AccessoryModel.swift

func queryLatestAccessoryAnchor() -> AccessoryAnchor? {
    guard let accessoryTrackingProvider,
          accessoryTrackingProvider.state == .running,
          let latestAnchor = accessoryTrackingProvider.latestAnchors.first else {
        return nil
    }

    if trackingMode == .predicted {
        return accessoryTrackingProvider.predictAnchor(for: latestAnchor, at: CACurrentMediaTime() + renderLatencyCompensation)
    }

    return latestAnchor
}
```

The `VolumeView` passes <doc://com.apple.documentation/documentation/ARKit/ARKitCoordinateSpace/Correction/rendered> to `coordinateSpace` because it applies the rotation to a rendered entity.
A non-rendering use case, like a measuring app, would pass <doc://com.apple.documentation/documentation/ARKit/ARKitCoordinateSpace/Correction/none> to read the raw rotation.

The `VolumeView` only reads the rotation from the accessory’s transform.
To obtain the accessory’s full transform, wrap `ancestorFromSpaceTransformFloat()` in a <doc://com.apple.documentation/documentation/RealityKit/Transform>:

```swift
// Reference snippet: This isn't part of the project's sample code.

let worldTransform = Transform(
    projectiveTransform: accessoryAnchor
        .coordinateSpace(correction: .rendered)
        .ancestorFromSpaceTransformFloat()
)
```

For more information on using `AccessoryTrackingProvider`, see <doc://com.apple.documentation/documentation/visionOS/drawing-in-the-air-and-on-surfaces-with-a-spatial-stylus>.

For information on tracking in a volume, see <doc://com.apple.documentation/documentation/ARKit/tracking-accessories-in-volumetric-windows>.
For information on using an accessory’s transform to drive interactive content, see <doc://com.apple.documentation/documentation/ARKit/tracking-a-handheld-accessory-as-a-virtual-sculpting-tool>.

## Respond to accessory input

Some accessories have buttons.
The accessory’s `input` property provides the familiar <doc://com.apple.documentation/documentation/GameController> interface.
If the connected accessory has buttons, a person can press any button to initiate the same functionality as the “Place digital replica” button in `ContentView`.

When an accessory connects, `AccessoryModel` sets an <doc://com.apple.documentation/documentation/GameController/GCDevicePhysicalInput/elementValueDidChangeHandler> on the accessory’s input to call `initiateDigitalReplicaPlacement` on every button press:

```swift
// AccessoryModel.swift

private func handleAccessoryDeviceChange() {
    // ...

    accessoryDevice.input?.elementValueDidChangeHandler = { [weak self] (_, element) in
        guard let self else { return }
        if let button = element as? GCButtonElement,
           button.pressedInput.isPressed {
            logger.info("Button pressed: \(element.localizedName ?? "Unnamed element")")
            initiateDigitalReplicaPlacement()
        }
    }

    // ...
}
```

For more information on callback and polling approaches to input handling, see <doc://com.apple.documentation/documentation/GameController/handling-input-events>.

## Play haptic feedback

Some accessories support haptic feedback.
The accessory’s `haptics` property provides access to <doc://com.apple.documentation/documentation/CoreHaptics>.
If the connected accessory supports haptics, a person can tap the “Play haptics” button in `ContentView` to provide feedback on the accessory.

When `HapticModel` initializes, it creates a <doc://com.apple.documentation/documentation/CoreHaptics/CHHapticEngine> on the accessory’s default locality and starts it:

```swift
// HapticModel.swift

init(accessory: GCSpatialAccessory) async {
    guard let engine = accessory.haptics?.createEngine(withLocality: .default) else {
        logger.info("The accessory doesn't support haptics.")
        return
    }

    do {
        try await engine.start()
        hapticEngine = engine
        logger.info("The haptic engine started successfully.")
    } catch {
        logger.error("Failed to start the haptic engine: \(error)")
        hapticEngine = nil
    }
}
```

With the engine running, the sample plays haptic patterns using <doc://com.apple.documentation/documentation/CoreHaptics/CHHapticPattern> whenever a person taps “Play haptics”.

For more information on input and haptics with accessories, see <doc://com.apple.documentation/documentation/GameController/discovering-and-tracking-spatial-game-controllers-and-styli>.

## Topics

### Working with reference accessory files

  <doc://com.apple.documentation/documentation/ARKit/preparing-spatial-accessories-for-tracking-in-your-visionos-app>

[Setting up access to a reference accessory file](/documentation/visionOS/setting-up-access-to-a-reference-accessory-file)

Track a generic spatial accessory by helping ARKit find the device’s reference accessory file.


---

Copyright &copy; 2026 Apple Inc. All rights reserved. | [Terms of Use](https://www.apple.com/legal/internet-services/terms/site.html) | [Privacy Policy](https://www.apple.com/privacy/privacy-policy)