Independent gestures for multiple entities in RealityView

Question

monono OP

Created 4d

Replies 1

Boosts 0

Participants 2

Hi,

I'm developing an app for the Apple Vision Pro.

Inside the app the user should be able to load objects from the web into the scene and then be able to move them around (dragging and rotating) via gestures.

My question: I'm working with RealityKit and use RealityView.. I have no issues loading in one object and making it interactive by adding gestures to the entire RealityView via the .gestures() function. Also I succeeded in loading multiple objects into the scene. My problem is that I can't figure out how to add my gestures to multiple objects independently. I can't use Reality composer since I'm loading the objects dynamically into the scene. Using .gestures() doesn't work for multiple objects since every gesture needs to be targeted to a specific entity but I have multiple entities.

I also tried defining a GestureComponent and adding it to every newly loaded entity but that doesn't seem to work as nothing happens, even though my gesture is targeted to every entity having my GestureComponent.

I only found solutions that are not usable on Visionos / RealityView like installGestures.

Also I tried following this guide: https://developer.apple.com/documentation/realitykit/transforming-realitykit-entities-with-gestures But I feel like there are things missing and contradictory, like a extension of RealityView is mentioned but not shown and the guide states that we don't need to store the translation values for every entity and therefore creates a EntityGestureState.swift file to store these values but then it's not used and a GestureStateComponent is used instead that was never mentioned and contradicts what was just said about not storing values per entity because a new instance of it is created for every entity. Nevermind, following the guide didn't lead me to a working solution.

Boost

Answer 1

Vision Pro Engineer OP

Apple

4d

Hi @monono

I'm fairly confident I can help you make dynamic entities intractable. Here's a snippet that dynamically creates intractable entities. To run it:

Download the sample code project Transforming RealityKit entities using gestures.
Copy the Extensions and Components folders from ./Packages/RealityKitContent/Sources/RealityKitContent/ into your project.
Important: add code to your app initializer to register the component.

init() {
        GestureComponent.registerComponent()
}

Add the snippet to a volume. Note the same logic works in a RealityView in an immersive space.

struct ContentView: View {
    let root = Entity()
    var body: some View {
        VStack {
            RealityView { content in
                content.add(root)
            }
            .installGestures()
            
            Button("Add entity") {
                addNewEntity()
            }
        }
        .padding()
    }
    
    func addNewEntity() {
        let size:Float = 0.08
        let mesh = MeshResource.generateBox(size: size)
        let entity = ModelEntity(mesh: mesh, materials: [SimpleMaterial(color: .red, isMetallic: true)])
        
        // Tell RealityKit the entity is intractable.
        entity.components.set(InputTargetComponent())
        
        // Important, make sure this is big enough to surround your entity.
        // For complex shapes you can use generateStaticMesh(from:).
        // https://developer.apple.com/documentation/realitykit/shaperesource/generatestaticmesh(from:)-6fgkt
        let shape = ShapeResource.generateBox(width: size, height: size, depth: size)
        let collisionComponent = CollisionComponent(
            shapes: [shape]
        )
        // Tell RealityKit where to hit test.
        entity.components.set(collisionComponent)
                
        // Install the gesture component
        var gestureComponent = GestureComponent()
        gestureComponent.canDrag = true
        gestureComponent.canScale = false
        gestureComponent.canRotate = true
        entity.components.set(gestureComponent)
        
        entity.position.x = Float(root.children.count) * (size * 1.1)

        root.addChild(entity)
    }
}

Run the app. Click "Add entity" to add as many entities as you want to the scene. Note each entity is created dynamically and is intractable.

I want to help you achieve your goal. If this answers your question please accept it. Otherwise, please followup. If you can include code that will help me better help you.

1