AVCaptureDataOutputSynchronizer Frame Rate and Size not configurable?

Im having three problems with AVCaptureDataOutputSynchronizer that i can't get to solve:

I want to capture video frames (as still photos) along with depth data. When pressing a start/stop button, recording is set to true or false respectively.

1.) Is this the right way to start/stop the capture? (I don't want to session.startRunning()because the preview would stop as well. Could this cause frames that return after pressing stop to be dropped?

2.) In the delegate, i copy the buffers of the depth and video frames to put these in a list (otherwise the internal pool of buffers is exhausted and frames are dropped after ~ frame 13). But still, frames are dropped due to AVCaptureOutputDataDroppedReasonOutOfBuffers. Also, more than ~ 5fps is never achieved. Is copying taking too much time? Should this be done with an extra queue?

3.) To set the frame size, i use session.sessionPreset = AVCaptureSession.Preset.vga640x480 but this is not working for the video frames, they are 1920 × 1440. What am i doing wrong?

This is the delegate i have at the moment:

func dataOutputSynchronizer(_ synchronizer: AVCaptureDataOutputSynchronizer, didOutput synchronizedDataCollection: AVCaptureSynchronizedDataCollection) {
    if (recording) {
        if let syncedDepthData: AVCaptureSynchronizedDepthData = synchronizedDataCollection.synchronizedData(for: self.depthOutput) as? AVCaptureSynchronizedDepthData {
            if !syncedDepthData.depthDataWasDropped {
                if let syncedVideoData: AVCaptureSynchronizedSampleBufferData = synchronizedDataCollection.synchronizedData(for: self.videoDataOutput) as? AVCaptureSynchronizedSampleBufferData {
                    if !syncedVideoData.sampleBufferWasDropped {
                        
                        //processing depth and video data
                        
                    } else {
                        print("Video frame dropped, reason:", syncedVideoData.droppedReason.rawValue)
                    }
                }
            } else {
                print("Depth data dropped, reason:", syncedDepthData.droppedReason.rawValue)
            }
            
        }
    }
}

The app should simultaneously capture video frames and depth data for 3D processing, that's why the Dataobjects are send over to a Socke.IO server after pressing stop.

If it is of any help, here is the code i use for processing:

frameCount += 1
                            
//let timestamp = CMTimeGetSeconds(syncedDepthData.timestamp) / 1000
let timestamp = syncedVideoData.timestamp

// intrinsic matrix
let depth_converted = syncedDepthData.depthData
let intrinsic_matrix = depth_converted.cameraCalibrationData!.intrinsicMatrix
let intrinsic = [intrinsic_matrix[0][0], intrinsic_matrix[1][1], intrinsic_matrix[2][0], intrinsic_matrix[2][1]]
let instrinsic_matrix = intrinsic.withUnsafeBufferPointer {Data(buffer: $0)}

let depthSampleBuffer = syncedDepthData.depthData.converting(toDepthDataType: kCVPixelFormatType_DepthFloat32).depthDataMap
let videoSampleBuffer = syncedVideoData.sampleBuffer

guard let imagePixelBuffer = CMSampleBufferGetImageBuffer(videoSampleBuffer) else { fatalError() }

let frameData = imagePixelBuffer.copy()
let depthData = depthSampleBuffer.copy()

guard let image_data = getDataFromCMSampleBuffer(sampleBuffer: toCMSampleBuffer(pixelBuffer: frameData)) else {fatalError()}
//let image_data = imagePixelBuffer.GetData()
let depth_data = depthData.GetData()                                    // for socket

depthData.normalize()

forSocketToSend.0.append(depth_data)
forSocketToSend.1.append(image_data)
forSocketToSend.2.append(instrinsic_matrix)
forSocketToSend.3.append(timestamp)