VideoMaterial Black Screen on Vision Pro Device (Works in Simulator)

VideoMaterial Black Screen on Vision Pro Device (Works in Simulator)

App Overview

App Name: Extn Browser Bundle ID: ai.extn.browser Purpose: A visionOS web browser that plays 360°/180° VR videos in an immersive sphere environment

Development Environment & SDK Versions

App is released in the TestFlight.

Frameworks Used

• SwiftUI - UI framework
• RealityKit - 3D rendering, MeshResource, ModelEntity, VideoMaterial
• AVFoundation - AVPlayer, AVAudioSession
• WebKit - WKWebView for browser functionality
• Network - NWListener for local proxy server

Sphere Video Mechanism

The app creates an immersive 360° video experience using the following approach:

// 1. Create sphere mesh (10 meter radius for immersive viewing)
let mesh = MeshResource.generateSphere(radius: 10.0)

// 2. Create initial transparent material
var material = UnlitMaterial()
material.color = .init(tint: .clear)

// 3. Create entity and invert sphere (negative X scale)
let sphere = ModelEntity(mesh: mesh, materials: [material])
sphere.scale = SIMD3<Float>(-1, 1, 1)  // Inverts normals for inside-out viewing
sphere.position = SIMD3<Float>(0, 1.5, 0)  // Eye level

// 4. Create AVPlayer with video URL
let player = AVPlayer(url: videoURL)

// 5. Configure audio session for visionOS
let audioSession = AVAudioSession.sharedInstance()
try audioSession.setCategory(.playback, mode: .moviePlayback, options: [.mixWithOthers])
try audioSession.setActive(true)

// 6. Create VideoMaterial and apply to sphere
let videoMaterial = VideoMaterial(avPlayer: player)
if var modelComponent = sphere.components[ModelComponent.self] {
    modelComponent.materials = [videoMaterial]
    sphere.components.set(modelComponent)
}

// 7. Start playback
player.play()

ImmersiveSpace Configuration

// browserApp.swift
ImmersiveSpace(id: appModel.immersiveSpaceID) {
    ImmersiveView()
        .environment(appModel)
}
.immersionStyle(selection: .constant(.mixed), in: .mixed)

Entitlements

<!-- browser.entitlements -->
<key>com.apple.security.app-sandbox</key>
<true/>
<key>com.apple.security.network.client</key>
<true/>
<key>com.apple.security.network.server</key>
<true/>

Info.plist Network Configuration

<key>NSAppTransportSecurity</key>
<dict>
    <key>NSAllowsArbitraryLoads</key>
    <true/>
</dict>

The Issue

Behavior in Simulator: Video plays correctly on the inverted sphere surface - 360° video is visible and wraps around the user as expected.

Behavior on Physical Vision Pro: The sphere displays a black screen. No video content is visible, though the sphere entity itself is present.

Important: Not a DRM/Licensing Issue

This issue is NOT related to Digital Rights Management (DRM) or FairPlay. I have tested with:

• Unlicensed raw MP4 video files (no DRM protection)
• Self-hosted video content with no copy protection
• Direct MP4 URLs from CDN without any licensing requirements

The same black screen behavior occurs with all unprotected video sources, ruling out DRM as the cause.

(Plain H.264 MP4, no DRM)

Screen Recording: Working in Simulator

The following screen recording demonstrates playing a 360° YouTube video in the immersive sphere on the visionOS Simulator:

https://cdn.commenda.kr/screen-001.mov

This confirms that the VideoMaterial and sphere rendering work correctly in the simulator, but the same setup shows a black screen on the physical Vision Pro device.

Observations

1. AVPlayer status reports .readyToPlay - The video appears to load successfully
2. VideoMaterial is created without errors - No exceptions thrown
3. Sphere entity renders - The geometry is visible (black surface)
4. Audio session is configured - No errors during audio session setup
5. Network requests succeed - The video URL is accessible from the device
6. Same result with local/unprotected content - DRM is not a factor

Console Logs (Device)

The logging shows:

• Sphere created and added to scene
• AVPlayer created with correct URL
• VideoMaterial created and applied
• Player status transitions to .readyToPlay
• player.play() called successfully
• Rate shows 1.0 (playing)

Despite all success indicators, the rendered output is black.

Questions for Apple

1. Are there known differences in VideoMaterial behavior between the visionOS Simulator and physical Vision Pro hardware?

2. Does VideoMaterial(avPlayer:) require specific video codec/format requirements that differ on device? (The test video is a standard H.264 MP4)

3. Is there a required Metal capability or GPU feature for VideoMaterial that may not be available in certain contexts on device?

4. Does the immersion style (.mixed) affect VideoMaterial rendering on hardware?

5. Are there additional entitlements required for video texture rendering in RealityKit on physical hardware?

Attempted Solutions

• Configured AVAudioSession with .playback category
• Added delay before player.play() to ensure material is applied
• Verified sphere scale inversion (-1, 1, 1)
• Tested multiple video URLs (including raw, unlicensed MP4 files)
• Confirmed network connectivity on device
• Ruled out DRM/FairPlay issues by testing unprotected content

Environment Details

• Device: Apple Vision Pro
• visionOS Version: 26.2
• Xcode Version: 26.2
• macOS Version: Darwin 25.2.0