How do I use RoomAnchor?

What I want to do:

I want to turn only the walls of a room into RealityKit Entities that I can collide with, or turn into occlusion surfaces.

This requires adding and maintaining RealityKit entities that with mesh information from the RoomAnchor. It also requires creating a "collision shape" from the mesh information.

What I've explored:

A RoomAnchor can provide me MeshAnchor.Geometry's that match only the "wall" portions of a Room.

I can use this mesh information to create RealityKit entities and add them to my immersive view.

But those Mesh's don't come with UUIDs, so I'm not sure how I could know which entities meshes need to to be updated as the RoomAnchor is updated.

As such I just keep adding duplicate wall entities.


A RoomAnchor also provides me with the UUIDs of its plane anchors, but no way to connect those to the provided meshes that I've discovered so far.


Here is how I add the green walls from the RoomAnchor wall meshes.

Note: I don't like that I need to wrap this in a task to satisfy the async nature of making a shape from a mesh. could be stuck with it, though.

Warning: this code will keep adding walls, even if there are duplicates and will likely cause performance issues :D.

func updateRoom(_ anchor: RoomAnchor) async throws {
        print("ROOM ID: \(anchor.id)")
        
        anchor.geometries(of: .wall).forEach { mesh in
            Task {
                let newEntity = Entity()
                newEntity.components.set(InputTargetComponent())
                realityViewContent?.addEntity(newEntity)
                newEntity.components.set(PlacementUtilities.PlacementSurfaceComponent())
                
                collisionEntities[anchor.id]?.components.set(OpacityComponent(opacity: 0.2))
                collisionEntities[anchor.id]?.transform = Transform(matrix: anchor.originFromAnchorTransform)
                
                // Generate a mesh for the plane
                do {
                    let contents = MeshResource.Contents(planeGeometry: mesh)
                    let meshResource = try MeshResource.generate(from: contents)
                    // Make this plane occlude virtual objects behind it.
//                     entity.components.set(ModelComponent(mesh: meshResource, materials: [OcclusionMaterial()]))
                    collisionEntities[anchor.id]?.components.set(ModelComponent(mesh: meshResource, materials: [SimpleMaterial.init(color: .green, roughness: 1.0, isMetallic: false)]))
                } catch {
                    print("Failed to create a mesh resource for a plane anchor: \(error).")
                    return
                }
                
                // Generate a collision shape for the plane (for object placement and physics).
                var shape: ShapeResource? = nil
                do {
                    let vertices = anchor.geometry.vertices.asSIMD3(ofType: Float.self)
                    shape = try await ShapeResource.generateStaticMesh(positions: vertices,
                                                                       faceIndices: anchor.geometry.faces.asUInt16Array())
                } catch {
                    print("Failed to create a static mesh for a plane anchor: \(error).")
                    return
                }
                
                if let shape {
                    let collisionGroup = PlaneAnchor.verticalCollisionGroup
                    
                    collisionEntities[anchor.id]?.components.set(CollisionComponent(shapes: [shape], isStatic: true,
                                                                                    filter: CollisionFilter(group: collisionGroup, mask: .all)))
                    // The plane needs to be a static physics body so that objects come to rest on the plane.
                    let physicsMaterial = PhysicsMaterialResource.generate()
                    let physics = PhysicsBodyComponent(shapes: [shape], mass: 0.0, material: physicsMaterial, mode: .static)
                    collisionEntities[anchor.id]?.components.set(physics)
                }
                
                collisionEntities[anchor.id]?.components.set(InputTargetComponent())
            }
        }
    }
Answered by Vision Pro Engineer in 790374022

You are on a good track! In case you haven't seen it yet, we just published a developer sample application for room tracking which demonstrates how to work with wall geometries. In a nutshell: The API is not designed for tracking walls over time (hence no wall UUIDs). The sample demonstrates how to a) visualise the wall that user is currently looking at and b) allows users to then lock a wall that they are happy with. After that, the entities are no longer updated.

Accepted Answer

You are on a good track! In case you haven't seen it yet, we just published a developer sample application for room tracking which demonstrates how to work with wall geometries. In a nutshell: The API is not designed for tracking walls over time (hence no wall UUIDs). The sample demonstrates how to a) visualise the wall that user is currently looking at and b) allows users to then lock a wall that they are happy with. After that, the entities are no longer updated.

How do I use RoomAnchor?
 
 
Q