Setting immerstionStyle while in immersive space breaks all entities.

I have my immersive space set up like:

ImmersiveSpace(id: "Theater") {
    ImmersiveTeleopView()
        .environment(appModel)
        .onAppear() {
            appModel.immersiveSpaceState = .open
        }
        .onDisappear {
            appModel.immersiveSpaceState = .closed
        }
}
.immersionStyle(selection: .constant(appModel.immersionStyle.style), in: .mixed, .full)

Which allows me to set the immersive style while in the space (from a Picker on a SwiftUI window). The scene responds correctly but a lot of the functionality of my immersive space is gone after the change in style; in that I am no longer able to enable/disable entities (which I also have a toggles for in the SwiftUI window). I have to exit and reenter the immersive space to regain the ability to change the enabled state of my entities.

My appModel.immersionStyle is inspired by the Compositor-Services demo (although I am using a RealityView) listed in https://developer.apple.com/documentation/CompositorServices/interacting-with-virtual-content-blended-with-passthrough and looks like this:

public enum IStyle: String, CaseIterable, Identifiable {
    case mixedStyle, fullStyle
    public var id: Self { self }
    var style: ImmersionStyle {
        switch self {
        case .mixedStyle:
            return .mixed
        case .fullStyle:
            return .full
        }
    }
}

/// Maintains app-wide state
@MainActor
@Observable
class AppModel {

    // Immersion Style
    public var immersionStyle: IStyle = .mixedStyle