WorldAnchors added and removed immediately

I can add a WorldAnchor to a WorldTrackingProvider. The next time I start my app, the WorldAnchor is added back, and then is immediately removed:

dataProviderStateChanged(dataProviders: [WorldTrackingProvider(0x0000000300bc8370, state: running)], newState: running, error: nil)
AnchorUpdate(event: added, timestamp: 43025.248134708, anchor: WorldAnchor(id: C0A1AE95-F156-45F5-9030-895CAABF16E9, isTracked: true, originFromAnchorTransform: <translation=(0.048458 0.000108 -0.317565) rotation=(0.00° 15.44° -0.00°)>))
AnchorUpdate(event: removed, timestamp: 43025.348131208, anchor: WorldAnchor(id: C0A1AE95-F156-45F5-9030-895CAABF16E9, isTracked: false, originFromAnchorTransform: <translation=(0.000000 0.000000 0.000000) rotation=(-0.00° 0.00° 0.00°)>))

It always leaves me with zero anchors in .allAnchors...the ARKitSession is still active at this point

Answered by jnorris441 in 823027022

Actually I did sometimes have the issue even when stopping SpatialTrackingSession.

Using a SpatialTrackingSession in the same project with ARKitSession + WorldTrackingProvider does not work properly. I switched to only using PlaneDetectionProvider for the floor and ceiling heights instead.

From the comments for WorldTrackingProvider anchorUpdates:

World anchors persist across device restarts until they are explicitly removed.

Removed by the user? Or removed by ARKit under some unknown condition?

Hi @jnorris441

This behavior is unexpected. I ran a test and could not reproduce the issue. Do you have a large project? It's possible a framework or some other rogue code is calling removeAnchor. Anchors can be removed by the system. Usually when you go far away or add too many anchors. That sounds unlikely in this case.

I recommend trying to reproduce this with a focused project. I typed this up to get you started. It will wait 5 seconds for a world anchor. If one isn't added within 5 seconds it creates one at the origin. When a world anchor is added it positions a sphere to match the world anchor's transform. In this case it positions the sphere at the origin so look near your feet the first time you run it and where you were standing the next time you run it.

import SwiftUI
import RealityKit
import ARKit

struct ImmersiveView: View {
    @State var worldTrackingProvider = WorldTrackingProvider()
    @State var arKitSession = ARKitSession()
    @State var firstWorldAnchorTransform: simd_float4x4?

    var body: some View {
        RealityView { content in
            
        }
        update: { content in
            if let firstWorldAnchorTransform {
                let entity = ModelEntity(mesh: .generateSphere(radius: 0.2), materials: [SimpleMaterial(color: .red, isMetallic: false)])
                
                entity.setTransformMatrix(firstWorldAnchorTransform, relativeTo: nil)
                content.add(entity)
            }
            else {
                print("Waiting for world anchor.")
            }
        }
        .task {
            do {
                try await arKitSession.run([worldTrackingProvider])
            }
            catch {
                print("Error starting session", error)
            }
                            
            Task {
                await processWorldAnchorUpdates()
            }
            
            // For demo purposes only. Don't do this in prod :)
            // Wait 5 seconds for a world anchor. If one isn't added create one.
            // Add anchor should only be called the first time you run this.
            // Delete the app and reinstall it to clear the anchor.
            Task {
                try? await Task.sleep(for: .seconds(5))
                
                if firstWorldAnchorTransform == nil {
                    let anchor = WorldAnchor(originFromAnchorTransform: matrix_identity_float4x4)
                    
                    do {
                        try await worldTrackingProvider.addAnchor(anchor)
                    }
                    catch {
                        print("Error adding anchor", error)
                    }
                }
            }
        }
    }
    
    func processWorldAnchorUpdates() async {
        print("Tracking the world")
        
        for await update in worldTrackingProvider.anchorUpdates {
            await processWorldAnchorUpdate(update: update)
        }
    }
    
    func processWorldAnchorUpdate(update:AnchorUpdate<WorldAnchor>) async {
        print("world anchor updated", update.event,  update.anchor.id)
        
        if(update.event == .added && firstWorldAnchorTransform == nil) {
            firstWorldAnchorTransform = update.anchor.originFromAnchorTransform
        }
    }
}

Let me know how it turns out.

Thank you. Your code works exactly as you would expect. The other full example app from Apple for room tracking also works properly.

.removeAnchor() is not used in my project at all.

It must be something I am doing. I am also running a SpatialTrackingSession to detect the floor and ceiling anchors. I will try to simplify my code.

Hello @Vision Pro Engineer

If I add a SpatialTrackingSession to your example I can replicate the problem. The world anchor will be added/updated, then immediately removed without calling removeAnchor().


import SwiftUI
import RealityKit
import ARKit

struct TestAnchor: View {
    @State var spatialTrackingSession = SpatialTrackingSession()
    @State var worldTrackingProvider = WorldTrackingProvider()
    @State var arKitSession = ARKitSession()
    @State var firstWorldAnchorTransform: simd_float4x4?

    var body: some View {
        RealityView { content in
            
        }
        update: { content in
            if let firstWorldAnchorTransform {
                let entity = ModelEntity(mesh: .generateSphere(radius: 0.2), materials: [SimpleMaterial(color: .red, isMetallic: false)])
                
                entity.setTransformMatrix(firstWorldAnchorTransform, relativeTo: nil)
                content.add(entity)
            }
            else {
                print("Waiting for world anchor.")
            }
        }
        .task {
            do {
                try await arKitSession.run([worldTrackingProvider])
            }
            catch {
                print("Error starting session", error)
            }
            
            let configuration = SpatialTrackingSession.Configuration(tracking: [.plane])
            if let unavailableCapabilities = await spatialTrackingSession.run(configuration) {
                if unavailableCapabilities.anchor.contains(.plane) {
                    print("plane tracking is unavailable.")
                }
            }
            
            Task {
                await processWorldAnchorUpdates()
            }
            
            
            // For demo purposes only. Don't do this in prod :)
            // Wait 5 seconds for a world anchor. If one isn't added create one.
            // Add anchor should only be called the first time you run this.
            // Delete the app and reinstall it to clear the anchor.
            Task {
                try? await Task.sleep(for: .seconds(5))
                
                if firstWorldAnchorTransform == nil {
                    let anchor = WorldAnchor(originFromAnchorTransform: matrix_identity_float4x4)
                    
                    do {
                        try await worldTrackingProvider.addAnchor(anchor)
                    }
                    catch {
                        print("Error adding anchor", error)
                    }
                }
            }
        }
    }
    
    func processWorldAnchorUpdates() async {
        print("Tracking the world")
        
        for await update in worldTrackingProvider.anchorUpdates {
            print(update)
            await processWorldAnchorUpdate(update: update)
        }
    }
    
    func processWorldAnchorUpdate(update:AnchorUpdate<WorldAnchor>) async {
        print("world anchor updated", update.event,  update.anchor.id)
        
        if(update.event == .added && firstWorldAnchorTransform == nil) {
            firstWorldAnchorTransform = update.anchor.originFromAnchorTransform
        }
    }
}

If I stop the SpatialTrackingSession before starting the ARKitSession, I do not have the issue.

Accepted Answer

Actually I did sometimes have the issue even when stopping SpatialTrackingSession.

Using a SpatialTrackingSession in the same project with ARKitSession + WorldTrackingProvider does not work properly. I switched to only using PlaneDetectionProvider for the floor and ceiling heights instead.

Hi @jnorris441

Thanks so much for recreating the issue and providing a workaround! This sounds like a bug so I want our teams to investigate it further. I'd appreciate it if you open a bug report, and post the FB number here once you do. The specific info you include your bug report might help our investigation, and filing the bug report allows you to get notified when it is resolved.

Bug Reporting: How and Why? explains how you can open a bug report.

In addition, the code I gave you yesterday is a bit error prone. I've cleaned it up and added comments to highlight the changes and included your changes.

struct ImmersiveView: View {
    @State var spatialTrackingSession = SpatialTrackingSession()
    @State var worldTrackingProvider = WorldTrackingProvider()
    @State var arKitSession = ARKitSession()
    @State var firstWorldAnchor: WorldAnchor?
    @State var entity: ModelEntity
    
    init() {
        // Create the entity once
        entity = ModelEntity(mesh: .generateSphere(radius: 0.2), materials: [SimpleMaterial(color: .red, isMetallic: false)])
        
    }
    var body: some View {
        RealityView { content in
            // Add the entity once
            content.add(entity)
            
            guard let firstWorldAnchor else { return }
            
            entity.setTransformMatrix(firstWorldAnchor.originFromAnchorTransform, relativeTo: nil)
        }
        update: { content in
            if let firstWorldAnchor {
                entity.setTransformMatrix(firstWorldAnchor.originFromAnchorTransform, relativeTo: nil)
            }
            else {
                print("Waiting for world anchor.")
            }
        }
        .task {
            do {
                try await arKitSession.run([worldTrackingProvider])
            }
            catch {
                print("Error starting session", error)
            }
            
            // Uncomment to recreate the issue
//            let configuration = SpatialTrackingSession.Configuration(tracking: [.plane])
//            if let unavailableCapabilities = await spatialTrackingSession.run(configuration) {
//                if unavailableCapabilities.anchor.contains(.plane) {
//                    print("plane tracking is unavailable.")
//                }
//            }
            
            Task {
                await processWorldAnchorUpdates()
            }
            
            
            // For demo purposes only. Don't do this in prod :)
            // Wait 5 seconds for a world anchor. If one isn't added create one.
            // Add anchor should only be called the first time you run this.
            // Delete the app and reinstall it to clear the anchor.
            Task {
                try? await Task.sleep(for: .seconds(5))
                
                if firstWorldAnchor == nil {
                    let anchor = WorldAnchor(originFromAnchorTransform: matrix_identity_float4x4)
                    
                    do {
                        try await worldTrackingProvider.addAnchor(anchor)
                    }
                    catch {
                        print("Error adding anchor", error)
                    }
                }
            }
        }
    }
    
    func processWorldAnchorUpdates() async {
        print("Tracking the world")
        
        for await update in worldTrackingProvider.anchorUpdates {
            await processWorldAnchorUpdate(update: update)
        }
    }
    
    func processWorldAnchorUpdate(update:AnchorUpdate<WorldAnchor>) async {
        print("world anchor", update.event,  update.anchor.id)
        
        // Ignore removed for this demo.
        guard update.event != .removed else { return }
        
        // Respond to add and update
        if firstWorldAnchor == nil || firstWorldAnchor?.id == update.anchor.id {
            firstWorldAnchor = update.anchor
        }
    }
}

@Vision Pro Engineer I submitted FB16424173 thank you

WorldAnchors added and removed immediately
 
 
Q