Spatial streaming from iPhone

Hi, I am trying to stream spatial video in realtime from my iPhone 16.

I am able to record spatial video as a file output using:

let videoDeviceOutput = AVCaptureMovieFileOutput()

However, when I try to grab the raw sample buffer, it doesn't include any spatial information:

let captureOutput = AVCaptureVideoDataOutput()

//when init camera
session.addOutput(captureOutput)
captureOutput.setSampleBufferDelegate(self, queue: sessionQueue)

//finally
func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
       //use sample buffer (but no spatial data available here)
}

Is this how it's supposed to work or maybe I am missing something?

this video: https://developer.apple.com/videos/play/wwdc2023/10071 gives us a clue towards setting up spatial streaming and I've got the backend all ready for 3D HLS streaming. Now I am only stuck at how to send the video stream to my server.