Interacting with 3D objects in VisionOS

Hi!

Here's a bunch of steps on how to do this. You'll need to create the tap gesture, install the gesture on the RealityView, and then set the collision and input target components.

Create a tap gesture:

 // Build a tap gesture
    private var tapGesture: some Gesture {
        return SpatialTapGesture()
            .targetedToAnyEntity()
            .onEnded {
                let previous = entityModel.currentEntity
                // Change the current entity to the tapped one
                entityModel.currentEntity = .init(rawValue: $0.entity.name) ?? .none
                // If it's the same entity tapped, deselect it
                if entityModel.currentEntity == previous {
                    entityModel.currentEntity = .none
                }
            }
    }

Here, I have a data model that holds the current entity (as an enum) which is the entity that is tapped. I am deselecting the entity on tap and selecting a new one.

--

Install the tap gesture on the RealityView

RealityView {
...
}
.gesture(tapGesture) //if you have multiple gestures, use .simultaneousGesture instead

--

Example

Here's how to set the collision and input target components, but this quickly just became an entire example on how to load and set the components of an entity :)

  if let loadedEntity = try? await Entity(named: someTitle, in: realityKitContentBundle) {
//load an entity from your Reality Composer pro scene ^

                    // Find the model component by looking for the name of the model entity in Reality Composer Pro (has the little box to the left of it in the asset navigator)
                    let modelComponentEntity = loadedEntity.findEntity(named: modelEntityName) as! ModelEntity
                    guard var modelComponent = modelComponentEntity.components[ModelComponent.self]
                    else { return }
                    
                    // if you need to set the material, do it here
                    if let material = someMaterial {
                        modelComponent.materials = [material]
                    }
                    
                    // If you need to, generate mesh in a specific shape and set the collisionComponent.
                    let shape = try? await ShapeResource.generate(from: modelComponent.mesh)
                    modelComponentEntity.components.set(CollisionComponent(shapes: [shape!]))
                    
                    // Set the model component
                    modelComponentEntity.components.set(modelComponent)
                    
                    //Configure the entity for tap interaction - this allows for both direct and indirect input
modelComponentEntity.components.set(InputTargetComponent(allowedInputTypes: .all))
//if you want a hover effect, do it here
        modelComponentEntity.components.set(HoverEffectComponent())

// set the entity 
                    entity = modelComponentEntity
                    
                    // Add each entity to the root
                   root.addChild(entity)
                    
                    //Set the position
                    entity.position = [0, 0, 1]
                }

Tips:

• Make sure the entity has a Collision Component and an InputTargetComponent
• Be sure to use .targetedToAnyEntity()
• Action to complete should be in the .onEnded

There's more gestures in this sample: https://developer.apple.com/documentation/realitykit/transforming-realitykit-entities-with-gestures

It shows you how to use other gestures with 3D objects in visionOS.

Hope this helps,

Sydney