EnvironmentBlendingComponent(.occluded(by: .surroundings)) culling entities entirely?

Hi there!

While building my visionOS app, I’ve encountered some strange behaviour with EnvironmentBlendingComponent. Entities using .occluded(by: .surroundings) are culled entirely whenever a Mac Virtual Display or Apple Immersive Environment is behind them, regardless of the entities’ actual depth, and even when the environment is set to coexist.

This feels like it could be a bug, although I’m fairly new to visionOS development, so I may be missing something!

In a .mixed immersive space with plain passthrough, the component behaves as I would expect: an entity is occluded accurately when a real-world object is placed in front of it.

However, when either of the following is visible behind the entity:

  • a Mac Virtual Display window; or
  • an Apple Immersive Environment enabled using the Digital Crown,

the entity carrying EnvironmentBlendingComponent disappears completely. This happens even when the entity is physically closer to the viewer than the virtual surface. An otherwise identical entity without the component remains visible.

Moving the virtual surface out of the line of sight, disabling the Immersive Environment, or removing the component immediately makes the entity visible again.

Is it intended that .occluded(by: .surroundings) treats Mac Virtual Display and Apple Immersive Environments as occluders? The documentation describes this mode as depth-based occlusion against static “real-world objects”, so it isn’t clear whether these system-rendered virtual surfaces should participate.

Even if they are intended to participate, the observed behaviour appears depth-independent: the entire entity is hidden when the virtual surface is behind it, rather than only when the surface is closer to the viewer. That seems inconsistent with the documented depth-based behaviour.

I can reproduce this on an M5 Apple Vision Pro running visionOS 26.5 using the following minimal example:

import SwiftUI
import RealityKit
import UIKit

@main
struct OcclusionReproApp: App {
    var body: some SwiftUI.Scene {
        WindowGroup {
            OcclusionReproLauncher()
        }

        ImmersiveSpace(id: "immersive") {
            RealityView { content in
                // Control: opaque, no component — proves the bare setup renders.
                let control = ModelEntity(
                    mesh: .generateSphere(radius: 0.12),
                    materials: [SimpleMaterial(color: .green, isMetallic: false)]
                )
                control.position = [-0.25, 1.2, -1.3]   // ~1.3 m ahead, eye height

                // Under test: identical opaque sphere plus the occlusion component.
                let occluded = ModelEntity(
                    mesh: .generateSphere(radius: 0.12),
                    materials: [SimpleMaterial(color: .red, isMetallic: false)]
                )
                occluded.position = [0.25, 1.2, -1.3]
                occluded.components.set(
                    EnvironmentBlendingComponent(preferredBlendingMode: .occluded(by: .surroundings))
                )

                content.add(control)
                content.add(occluded)
            }
        }
        .immersionStyle(selection: .constant(.mixed), in: .mixed)
        // Let the app's mixed space coexist with an Apple Immersive Environment,
        // as visionOS suppresses Environments while an immersive space is open otherwise.
        // The same bug can be reproduced with just a Mac Virtual Display, but
        // this provides a second way to reproduce the same behaviour:
        .immersiveEnvironmentBehavior(.coexist)
    }
}

struct OcclusionReproLauncher: View {
    @Environment(\.openImmersiveSpace) private var openImmersiveSpace

    var body: some View {
        Button("Open immersive space") {
            Task { await openImmersiveSpace(id: "immersive") }
        }
        .padding()
    }
}

To reproduce (on a Vision Pro, as the Simulator won't provide passthrough):

  1. Open the immersive space in plain passthrough. Both spheres should be visible.
  2. Place a real object in front of the red sphere. It should be occluded correctly according to depth.
  3. Open a Mac Virtual Display or enable an Apple Immersive Environment, with its visible surface behind the red sphere.
  4. The red sphere disappears entirely, despite being in front of that surface. The green control sphere remains visible.
  5. Move the virtual surface out of view or disable it, and the red sphere reappears.

Please let me know if this is expected behaviour, or if I'm doing something wrong - thanks!

Jack

Hi @jackwh,

Thank you for the detailed question, what you're seeing is in fact the expected behavior.

This is briefly covered in the EnvironmentBlendingComponent documentation, though I can try to provide some additional context.

Regarding Mac Virtual Display:

An entity with occlusion enabled behaves like a real-world passthrough object. It always renders behind other virtual contents that don’t have occlusion enabled.

The key here is that entities with EnvironmentBlendingComponent(.occluded(by: .surroundings)) are occluded just like real-world objects. One way to think about this is that just like how all the real-world objects in your environment are occluded by Mac Virtual Display regardless of their "depth" (for example, the monitor on your desk, the coffee cup on your table, etc.), virtual entities with occlusion behave the exact same way.

Regarding system environments:

For progressive space, it works only for entities outside of the portal world. When the portal mask expands and overlaps with those entities by turning the Digital Crown, it fades those entities out gradually.

While the use of term "progressive space" is admittedly a bit confusing here, this is actually referring to exactly what you're seeing with immersiveEnvironmentBehavior(.coexist) where entities with occlusion become invisible when they overlap with the system environment made visible by turning the Digital Crown.

Thanks for such a comprehensive post—I really appreciate the detail (and the feedback ticket)!

EnvironmentBlendingComponent(.occluded(by: .surroundings)) culling entities entirely?
 
 
Q