Hi all,
I'm working on a custom Metal-based video pipeline using AVCaptureVideoDataOutput, and I've run into an unexpected issue related to exposure.
Setup: I'm capturing video frames using kCVPixelFormatType_420YpCbCr8BiPlanarFullRange.
In my Metal shader, I:
- Convert YCbCr (full range Rec.709) to linear Rec.709 RGB.
- Apply Rec.709 → sRGB gamma encoding.
- Output to .bgra8Unorm_srgb via MTKView.
Everything renders correctly in terms of colorspace math, but the image appears significantly brighter (~+3 stops EV) compared to AVCaptureVideoPreviewLayer and the native iOS Camera app under the same camera exposure settings.
What I’ve verified: The color transforms are correct: YCbCr709 to RGB, then linear to sRGB.
I'm not applying any tone mapping or aggressive look LUTs yet.
Camera exposure is locked using: device.setExposureModeCustom(duration: ..., iso: ...)
The same EV (e.g., ISO 50, 1/125s, f/5.6) on my iPhone appears visually 3 stops brighter than on my digital cameras (Sony/Canon etc).
To match the look of the preview layer or camera app, I have to simulate a ~–3 EV shift in my custom pipeline.
Questions: Is AVCaptureVideoPreviewLayer applying extra tone mapping, digital gain, or contrast shaping (like OOTF etc)?
Does the camera ISP expose "hotter" (i.e., with more light) internally for the preview layer than what we get in video frame buffers?
Is there a standard way to compensate for this ISP behavior in custom pipelines using AVCaptureVideoDataOutput?
Can this be accounted for using metadata (e.g., exposure bias, gain, gamma curve)?