Capture Video from my own app using enterprise APIs in visionOS

Question

Sandip_Aura OP

Created 2w

Replies 1

Boosts 0

Participants 3

Hello,

I want to capture video from Vision Pro in the Vision OS app. I am referring to the (https://developer.apple.com/videos/play/wwdc2024/10139/) Apple video and their code. step like below

import ARKit
com.apple.developer.arkit.main-camera-access.allow = true in info.plist
Do below code

 func loadCameraFeed() async {
        // Main Camera Feed Access Example
        let formats = CameraVideoFormat.supportedVideoFormats(for: .main, cameraPositions:[.left])
        let cameraFrameProvider = CameraFrameProvider()
        var arKitSession = ARKitSession()
        var pixelBuffer: CVPixelBuffer?
        await arKitSession.queryAuthorization(for: [.cameraAccess])
        do {
            try await arKitSession.run([cameraFrameProvider])
        } catch {
            return
        }
        guard let cameraFrameUpdates =
            cameraFrameProvider.cameraFrameUpdates(for: formats[0]) else {
            return
        }
        print(cameraFrameUpdates)
        for await cameraFrame in cameraFrameUpdates {
            
            print(cameraFrame)
            
            guard let mainCameraSample = cameraFrame.sample(for: .left) else {
                continue
            }
            pixelBuffer = mainCameraSample.pixelBuffer
        }
    }

I want to convert "pixelBuffer" into video streaming and show it in a frame like iOS.

Please guide me on how to achieve my next step. I am blank after this code.

Boost

Answer 1

Vision Pro Engineer OP

Apple

2w

Hey @Sandip_Aura,

If you haven't already, please make sure that you have requested the associated entitlements. Enterprise APIs are eligible for business use only, and you can only distribute apps that you develop with the Enterprise APIs privately as proprietary in-house apps or custom apps using Apple Business Manager.

Once you have the entitlement, use the init(cvPixelBuffer:) initializer create a CIImage. Once you have an CIImage, use the createCGImage(_:from:) method on a CIContext instance to create a CGImage. From there you can create a UIImage and then pass it to your SwiftUI Image view.

let ciImage = CIImage(cvPixelBuffer: pixelBuffer)
let context = CIContext(options: nil)
if let cgImage = context.createCGImage(ciImage, from: ciImage.extent) {
    Image(uiImage: UIImage(cgImage: cgImage))
}

Let me know if you have additional questions,

Michael

0