Radiometric interpretation of Apple ProRAW and Bayer RAW access via AVFoundation

Question

Created 2w

Replies 1

Boosts 0

Participants 2

I am working on a computational photography research project involving multi-exposure HDR reconstruction using Bayer RAW and Apple ProRAW captures. I would like to clarify the radiometric interpretation of Apple ProRAW and the availability of Bayer RAW capture through AVFoundation.

My questions are:

On current iPhone Pro devices, is it possible for third-party apps to capture and export true Bayer-pattern RAW DNG files through AVFoundation, rather than Apple ProRAW linear DNG files? If so, which availableRawPhotoPixelFormatTypes correspond to Bayer RAW, and what device or format restrictions apply? Apple ProRAW appears to be demosaiced and computationally processed, and may include multi-frame fusion. Is the decoded ProRAW image intended to be radiometrically linear and scene-referred? For a bracketed ProRAW sequence captured with fixed ISO, white balance, lens, and focus, but different exposure times, can one assume that the decoded linear pixel values Y_i(p) satisfy an exposure-proportional model in non-saturated regions, such as Y_i(p) ≈ t_i R(p), across brackets?

This question is about radiometric consistency for algorithmic use, not about visual editing or tone mapping.

Thank you for your help.

Answered by DTS Engineer in 887990022

Thanks for the precise framing — answering each in turn.

Q1 — Bayer RAW capture through AVFoundation:

Yes, third-party apps can capture Bayer RAW through AVFoundation. Both Bayer RAW and Apple ProRAW formats appear in AVCapturePhotoOutput.availableRawPhotoPixelFormatTypes. The recommended way to discriminate them is through the static query methods:

AVCapturePhotoOutput.isBayerRAWPixelFormat(_:)
AVCapturePhotoOutput.isAppleProRAWPixelFormat(_:)

Apple ProRAW is opt-in: confirm photoOutput.isAppleProRAWSupported, then set photoOutput.isAppleProRAWEnabled = true. With ProRAW disabled, availableRawPhotoPixelFormatTypes exposes only Bayer formats; enabling ProRAW adds the ProRAW format to the same array.

Device and format restrictions:

Apple ProRAW: iPhone 12 Pro / Pro Max and later Pro models (iOS 14.3+).
Bayer RAW: broadly available on devices that support RAW capture. Some virtual / fusion capture devices (for example, multi-cam) may not expose Bayer RAW because there is no single sensor providing the Bayer mosaic. The reliable way to check is to query availableRawPhotoPixelFormatTypes on the photo output after it's added to the configured session.

Reference: Capturing photos in RAW and Apple ProRAW formats.

Q2 — ProRAW radiometric interpretation:

The documentation describes Apple ProRAW as providing "the benefits of RAW capture" while applying "many of the multi-image fusion techniques previously unavailable to RAW workflows." The output is a DNG that preserves the editing latitude of RAW, but multi-image fusion is part of the processing pipeline.

I'm not going to speculate beyond what's publicly documented about the specifics of that fusion — which algorithms run, when they run, how they vary by scene content, or how they interact with bracketing. Those are internal details that aren't part of the public API contract.

Q3 — Exposure-proportional model across bracketed ProRAW:

For your specific research use case — Y_i(p) ≈ t_i × R(p) across brackets in non-saturated regions — ProRAW is not the appropriate format. The documented multi-image fusion in ProRAW means a decoded frame combines data from multiple source frames, and there is no public contract that the proportional relationship is preserved across brackets. I won't speculate on how fusion varies bracket-to-bracket; what I can say is that the proportional model isn't something you can rely on with ProRAW.

Standard Bayer RAW is the right tool for radiometrically rigorous HDR reconstruction. The article calls out the tradeoff explicitly: standard RAW capture "bypasses the advanced processing that the image-capture pipeline provides." For your use case, bypassing that processing is the goal — Bayer RAW gives you the predictable per-frame, exposure-proportional behavior your model needs (modulo the usual sensor effects you'll want to characterize anyway: black level, PRNU, and non-linearity near saturation).

For deeper engagement:

If you have specific questions about radiometric guarantees beyond what's in public documentation — for example, exact characterization of non-linearity at high or low ISO, or sensor-specific calibration — that would benefit from a Code-Level Support request. I can engage directly and route specifics to engineering as needed.

Answer 1

DTS Engineer OP

Apple

2w

Recommended

Thanks for the precise framing — answering each in turn.

Q1 — Bayer RAW capture through AVFoundation:

Yes, third-party apps can capture Bayer RAW through AVFoundation. Both Bayer RAW and Apple ProRAW formats appear in AVCapturePhotoOutput.availableRawPhotoPixelFormatTypes. The recommended way to discriminate them is through the static query methods:

AVCapturePhotoOutput.isBayerRAWPixelFormat(_:)
AVCapturePhotoOutput.isAppleProRAWPixelFormat(_:)

Apple ProRAW is opt-in: confirm photoOutput.isAppleProRAWSupported, then set photoOutput.isAppleProRAWEnabled = true. With ProRAW disabled, availableRawPhotoPixelFormatTypes exposes only Bayer formats; enabling ProRAW adds the ProRAW format to the same array.

Device and format restrictions:

Apple ProRAW: iPhone 12 Pro / Pro Max and later Pro models (iOS 14.3+).
Bayer RAW: broadly available on devices that support RAW capture. Some virtual / fusion capture devices (for example, multi-cam) may not expose Bayer RAW because there is no single sensor providing the Bayer mosaic. The reliable way to check is to query availableRawPhotoPixelFormatTypes on the photo output after it's added to the configured session.

Reference: Capturing photos in RAW and Apple ProRAW formats.

Q2 — ProRAW radiometric interpretation:

The documentation describes Apple ProRAW as providing "the benefits of RAW capture" while applying "many of the multi-image fusion techniques previously unavailable to RAW workflows." The output is a DNG that preserves the editing latitude of RAW, but multi-image fusion is part of the processing pipeline.

I'm not going to speculate beyond what's publicly documented about the specifics of that fusion — which algorithms run, when they run, how they vary by scene content, or how they interact with bracketing. Those are internal details that aren't part of the public API contract.

Q3 — Exposure-proportional model across bracketed ProRAW:

For your specific research use case — Y_i(p) ≈ t_i × R(p) across brackets in non-saturated regions — ProRAW is not the appropriate format. The documented multi-image fusion in ProRAW means a decoded frame combines data from multiple source frames, and there is no public contract that the proportional relationship is preserved across brackets. I won't speculate on how fusion varies bracket-to-bracket; what I can say is that the proportional model isn't something you can rely on with ProRAW.

Standard Bayer RAW is the right tool for radiometrically rigorous HDR reconstruction. The article calls out the tradeoff explicitly: standard RAW capture "bypasses the advanced processing that the image-capture pipeline provides." For your use case, bypassing that processing is the goal — Bayer RAW gives you the predictable per-frame, exposure-proportional behavior your model needs (modulo the usual sensor effects you'll want to characterize anyway: black level, PRNU, and non-linearity near saturation).

For deeper engagement:

If you have specific questions about radiometric guarantees beyond what's in public documentation — for example, exact characterization of non-linearity at high or low ISO, or sensor-specific calibration — that would benefit from a Code-Level Support request. I can engage directly and route specifics to engineering as needed.