Swift Array Out of Bounds Crash in VTFrameProcessor when using VTLowLatencyFrameInterpolationParameters

Question

Created 14h

Replies 0

Boosts 0

Participants 1

Hi everyone,

Our team is encountering a reproducible crash when using VTLowLatencyFrameInterpolation on iOS 26.3 while processing a live LL-HLS input stream.

🤖 Environment

Device: iPhone 16
OS: iOS 26.3
Xcode: Xcode 26.3
Framework: VideoToolbox

💥 Crash Details

The application crashes with the following fatal error: Fatal error: Swift/ContiguousArrayBuffer.swift:184: Array index out of range

The stack trace highlights the following:

VTLowLatencyFrameInterpolationImplementation processWithParameters:frameOutputHandler:
Called from VTFrameProcessor.process(parameters:)

Here is the simplified implementation block where the crash occurs. (Note: PrismSampleBuffer and PrismLLFIError are our internal custom wrapper types).

// Create `VTFrameProcessorFrame` for the source (previous) frame.
let sourcePTS = sourceSampleBuffer.presentationTimeStamp
var sourceFrame: VTFrameProcessorFrame?
if let pixelBuffer = sourceSampleBuffer.imageBuffer { 
    sourceFrame = VTFrameProcessorFrame(buffer: pixelBuffer, presentationTimeStamp: sourcePTS) 
}

// Validate the source VTFrameProcessorFrame.
guard let sourceFrame else { throw PrismLLFIError.missingImageBuffer }

// Create `VTFrameProcessorFrame` for the next frame.
let nextPTS = nextSampleBuffer.presentationTimeStamp
var nextFrame: VTFrameProcessorFrame?
if let pixelBuffer = nextSampleBuffer.imageBuffer { 
    nextFrame = VTFrameProcessorFrame(buffer: pixelBuffer, presentationTimeStamp: nextPTS) 
}

// Validate the next VTFrameProcessorFrame.
guard let nextFrame else { throw PrismLLFIError.missingImageBuffer }

// Calculate interpolation intervals and allocate destination frame buffers.
let intervals = interpolationIntervals()
let destinationFrames = try framesBetween(firstPTS: sourcePTS, lastPTS: nextPTS, interpolationIntervals: intervals)

let interpolationPhase: [Float] = intervals.map { Float($0) }

// Create VTLowLatencyFrameInterpolationParameters.
// This sets up the configuration required for temporal frame interpolation between the previous and current source frames.
guard let parameters = VTLowLatencyFrameInterpolationParameters(
    sourceFrame: nextFrame, 
    previousFrame: sourceFrame, 
    interpolationPhase: interpolationPhase, 
    destinationFrames: destinationFrames
) else { 
    throw PrismLLFIError.failedToCreateParameters 
}

try await send(sourceSampleBuffer)

// Process the frames.
// Using progressive callback here to get the next processed frame as soon as it's ready,
// preventing the system from waiting for the entire batch to finish.
for try await readOnlyFrame in self.frameProcessor.process(parameters: parameters) {
    
    // Create an interpolated sample buffer based on the output frame.
    let newSampleBuffer: PrismSampleBuffer = try readOnlyFrame.frame.withUnsafeBuffer { pixelBuffer in
        try PrismLowLatencyFrameInterpolation.createSampleBuffer(from: pixelBuffer, readOnlyFrame.timeStamp)
    }

    // Pass the newly generated frame to the output stream.
    try await send(newSampleBuffer)
}

🙋 Questions

Are there any known limitations or bugs regarding VTLowLatencyFrameInterpolation when handling live 60fps streams?
Are there any undocumented constraints we should be aware of regarding source/previous frame timing, pixel buffer attributes, or how destinationFrames and interpolationPhase arrays must be allocated?
Is a "warm-up" sequence recommended after startSession() before making the first process(parameters:) call?

Boost