Pink Screen with VideoMaterial in ARKit

Hello @mr_app,

Is it a pink-striped material? That is what RealityKit renders if there is a model with no material set. Is it possible that you are removing the videoMaterial you have set in the code you have shown?

Best regards,

Greg

Same problem, video material pink on ios 18

are there any solutions?

I have the same issue: the video material is pink on iOS 18. On iOS 17, it is working fine.

When I track my AR asset, it starts playing the video. I can hear the audio, but I have this pink video. On iOS 17, it is working fine, but on iOS 18, I have this bug. Any suggestion on how to fix it?

That is my code, I still have the pink material for iOS 18. While on iOS 17 is working fine. Any advice?

// Create Video Screen
private func createVideoScreen(width: Float, height: Float, url: URL, uuid: UUID) -> ModelEntity {
    let screenMesh = MeshResource.generatePlane(width: width, height: height)
    let videoItem = createVideoItem(with: url)
    let videoMaterial = createVideoMaterial(with: videoItem, uuid: uuid)
    let videoScreenModel = ModelEntity(mesh: screenMesh, materials: [videoMaterial])
    return videoScreenModel
}

// Create Video Item
private func createVideoItem(with url: URL) -> AVPlayerItem {
    let asset = AVURLAsset(url: url)
    let videoItem = AVPlayerItem(asset: asset)
    return videoItem
}

// Create Video Material
private func createVideoMaterial(with videoItem: AVPlayerItem, uuid: UUID) -> VideoMaterial {
    let player = AVPlayer()
    player.actionAtItemEnd = .none
    let videoMaterial = VideoMaterial(avPlayer: player)
    player.replaceCurrentItem(with: videoItem)
    player.play()
    
    NotificationCenter.default.addObserver(forName: .AVPlayerItemDidPlayToEndTime, object: player.currentItem, queue: .main) { _ in
        player.seek(to: .zero)
        player.play()
    }
    videoPlayers[uuid] = player
    return videoMaterial
}

// Place Video Screen
private func placeVideoScreen(videoScreen: ModelEntity, imageAnchor: ARImageAnchor, uuid: UUID) {
    guard let arView = arView else { return }
    
    let imageAnchorEntity = AnchorEntity(anchor: imageAnchor)
    let rotationAngle = simd_quatf(angle: GLKMathDegreesToRadians(-90), axis: SIMD3(x: 1, y: 0, z: 0))
    videoScreen.setOrientation(rotationAngle, relativeTo: imageAnchorEntity)
    imageAnchorEntity.addChild(videoScreen)
    
    arView.scene.addAnchor(imageAnchorEntity)
    activeAnchors[uuid] = imageAnchorEntity
}

Same issue, iOS 18.1.1; solid pink, not striped

Same issue here! Solid pink screen!!! Works on iOS 17 but not on iOS 18+ (tested also on the latest iOS 18.2.1)

Trying to reproduce with this simple snippet:

struct ContentView : View {
    
    private let player = AVPlayer()

    var body: some View {
        RealityView { content in
            
            let model = Entity()
            let mesh = MeshResource.generatePlane(width: 1, height: 1)
            let material = VideoMaterial(avPlayer: player)
            model.components.set(ModelComponent(mesh: mesh, materials: [material]))
            model.position = [0, 1, -2]
            
            content.add(model)

            content.camera = .spatialTracking
        }
        .edgesIgnoringSafeArea(.all)
        .onAppear() {
            Task {
                let videoURL = URL(string: "http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4")
                
                let playerItem = AVPlayerItem(url: videoURL!)
//                    let videoAssetTrack = try await playerItem.asset.loadTracks(withMediaType: .video)
//                    try await videoAssetTrack.first!.load(.naturalSize)
                
                // Tell the player to load and play
                player.replaceCurrentItem(with: playerItem)
                player.play()
            }
        }
    }
}

If you argue that i'm using http instead of https, i tryed both (also with "App Transport Security Settings" -> "Allow Arbitrary Loads": yes into info.plist)

Got a lot of warnings (i suppose that some shader or some material, as VideoMaterial, are missing):

Error creating the CFMessagePort needed to communicate with PPT.
Could not locate file 'default-binaryarchive.metallib' in bundle.
Video texture allocator is not initialized.
[VideoLightSpillGenerator] [VideoLightSpillMPSCallsPrewarm] Failed to create input texture
Registering library (/System/Library/PrivateFrameworks/CoreRE.framework/default.metallib) that already exists in shader manager. Library will be overwritten.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/suFeatheringCreateMergedOcclusionMask.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arKitPassthrough.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arSegmentationComposite.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute0.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute1.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute2.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute3.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute4.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute5.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute6.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute7.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute8.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.

...

Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute11.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute12.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute13.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute14.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.
Could not resolve material name 'engine:BuiltinRenderGraphResources/AR/arInPlacePostProcessCombinedPermute15.rematerial' in bundle at '/private/var/containers/Bundle/Application/374F0025-A5DF-4C20-8A0A-711DCC837C0F/ProvaAR.app'. Loading via asset path.

and when i play the video i got lot of these (not an encoding issue because the same video works if i use AVPlayer with plain VideoPlayer in swiftUI):

Failed to sample image from video receiver (error: -15579)

Please HEEEEEELP!!!!