glassMaterialEffect not working in immersive view with a skybox

Question

Created May ’24

Replies 1

Boosts 0

Participants 2

I seem to be running into an issue where the .glassBackgroundEffect modifier doesn't seem to render correctly. The issue is occurring when attached to a view shown in a RealityKit immersive view with a Skybox texture. The glass effect is applied but doesn't let any of the colour of the skybox behind it though.

I have created a sample project which is just the immersive space template with the addition of a skybox texture and an attachment with the glassBackgroundEffect modifier. The RealityView itself is

struct ImmersiveView: View {
    var body: some View {
        RealityView { content, attachments in
            // Add the initial RealityKit content
            if let immersiveContentEntity = try? await Entity(named: "Immersive", in: realityKitContentBundle) {
                content.add(immersiveContentEntity)

                let attachment = attachments.entity(for: "foo")!
                let leftSphere = immersiveContentEntity.findEntity(named: "Sphere_Left")!
                attachment.position = [0, 0.2, 0]
                leftSphere.addChild(attachment)


                // Add an ImageBasedLight for the immersive content
                guard let resource = try? await EnvironmentResource(named: "ImageBasedLight") else { return }
                let iblComponent = ImageBasedLightComponent(source: .single(resource), intensityExponent: 0.25)
                immersiveContentEntity.components.set(iblComponent)
                immersiveContentEntity.components.set(ImageBasedLightReceiverComponent(imageBasedLight: immersiveContentEntity))

                // Put skybox here.  See example in World project available at
                var skyboxMaterial = UnlitMaterial()
                let skyboxTexture = try! await TextureResource(named: "pano")
                skyboxMaterial.color = .init(texture: .init(skyboxTexture))
                let skyboxEntity = Entity()
                skyboxEntity.components.set(ModelComponent(mesh: .generateSphere(radius: 1000), materials: [skyboxMaterial]))
                skyboxEntity.scale *= .init(x: -1, y: 1, z: 1)
                content.add(skyboxEntity)
            }
        } update: { _, _ in
        } attachments: {
            Attachment(id: "foo") {
                Text("Hello")
                    .font(.extraLargeTitle)
                    .padding(48)
                    .glassBackgroundEffect()
            }
        }
    }
}

The effect is shown bellow

RocketSim_Screenshot_Apple_Vision_Pro_-1.0_2024-05-23_14.21.04.png

I've tried this both in the simulator and in a physical device and get the same behaviour.

Not sure if this is an issue with RealityKit or if I'm just holding it wrong, would greatly appreciate any help. Thanks.

Boost

Answer 1

Vision Pro Engineer OP

Apple

May ’24

Accepted Answer

Recommended

This is a known limitation of the modifier. Please consider filing an enhancement request using Feedback Assistant.

Here are some alternate approaches that may work for your use case:

Use the background modifier when creating the attachment view instead of the glassBackgroundEffect modifier. The drawback of this is you lose the blurring and other visual treatments that come with the glass background.

// change the modifier values to suit your app
.background(
  Color.white
    .cornerRadius(15)
    .opacity(0.5)
)

Control the appearance of the background by creating a plane using MeshResource.generatePlane, with a material of your choosing and append the attachments to the plane. Here’s a snippet that uses a SimpleMaterial to do that.

// note you can use PhysicalMetrics to convert this to pixels https://developer.apple.com/documentation/swiftui/environmentvalues/physicalmetrics

let controlsSize = CGSize(width: 1.0, height: 1.0)

if let attachment = attachments.entity(for: attachmentId) {
    let backgroundMesh = MeshResource.generatePlane(width: Float(controlsSize.width), height: Float(controlsSize.height), cornerRadius: 0.1)

    // Material of your choosing
    let backgroundMaterial = SimpleMaterial(color: .blue.withAlphaComponent(0.5), isMetallic: true)
    let backgroundEntity = ModelEntity(mesh: backgroundMesh, materials: [backgroundMaterial])

    backgroundEntity.addChild(attachment)
    backgroundEntity.position = [0, 1.0, -1.0]

    content.add(backgroundEntity)
}

Present the controls in a separate 2D window (by defining and opening a new window group). The drawback of this is windows can not be positioned.

1