If you have two video segments, one HDR and one SDR, next to one another in a composition, the SDR one appears dark, since its max luminance will be lower than the max luminance of the HDR clip. iMovie handles this well by (reverse) tone mapping the SDR content to make it look HDR in an HDR composition. This is what I want to achieve.
I've looked into algorithms to do this, and the best that I can find is the conversion from RGB to YCbCr described in Table 4 of BT.2020, followed by conversion method A (Section 4.2, table 4) of BT.2446-1. I have these implemented in a Core Image kernel, available at this repo. The issue that I'm seeing is that the colors are still much too hot, and while there are frames that appear close to properly tone mapped, it doesn't come close to the accuracy of iMovie's approach.
If someone is an expert in colorimetry, etc., I'd really appreciate a breakdown of what I'm doing wrong here. To be specific:
- Assuming non-constant luminance for the YCbCr conversion
- Using a Metal Core Image kernel for the actual tone mapping
- The video composition uses Core Image filters directly
- The video composition is using BT.2020 colorimetry and the PQ transfer function.
Below is a comparison of two screenshots, one using an unaltered HDR asset, and the other using the same asset transcoded to SDR with QuickTime, and reverse tone mapped to HDR for playback. Is there something I'm missing?
For the full story on how HDR is supported on our platform and processing pipelines we recommend I suggest the following WWDC videos (presented in reverse chronological order):
Use HDR for dynamic image experiences in your app https://developer.apple.com/videos/play/wwdc2024/10177/
Support HDR images in your app https://developer.apple.com/videos/play/wwdc2023/10181/
Display EDR content with Core Image, Metal, and SwiftUI https://developer.apple.com/videos/play/wwdc2022/10114
Explore EDR on iOS https://developer.apple.com/videos/play/wwdc2022/10113
Explore HDR rendering with EDR https://developer.apple.com/videos/play/wwdc2021/10161/