Exporting Metal Texture in BGRA10_XR to CGImage (while maintaining P3)

Question

Created Jun ’20

Replies 1

Boosts 0

Participants 2

I've been struggling to export an MTLTexture that uses BGRA10_XR (for P3) to a CGImage/UIImage that matches the same colors displayed in the MTLTexture (or on screen).

Code Block Swiftfunc getImage() -> UIImage? {
	let bufferWidth:  Int = Int(size.width)
	let bufferHeight: Int = Int(size.height)
	let bytesPerRow:	Int = Int(ceilf(self.size.width * 4/32)) * Int(32)
  let bufferSize:  Int = bytesPerRow * bufferHeight;
	let colorspace:  	 	  CGColorSpace
	let bitmapInfo:   	  CGBitmapInfo
	let bitsPerComponent: Int
	let bitsPerPixel:   	Int
	if self.isBGRA10_XR {
		bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.premultipliedLast.rawValue).union(.byteOrder16Little)
		bitsPerComponent = 16
		bitsPerPixel   	 = 64
		colorspace    	 = CGColorSpace(name: CGColorSpace.displayP3)!
	} else {
		bitmapInfo = CGBitmapInfo(rawValue: (CGBitmapInfo.byteOrder32Little.rawValue | CGImageAlphaInfo.first.rawValue))
		bitsPerComponent = 8
		bitsPerPixel  	 = 32
		colorspace   		 = CGColorSpaceCreateDeviceRGB()
	}
	let data = malloc(bufferSize)
	memcpy(data, self.mtlTextureBuffer.contents(), bufferSize)
provider = CGDataProvider(dataInfo: nil, data: data!, size: bufferSize, releaseData: { info, data, size in
      free(UnsafeMutableRawPointer(mutating: data))
    })!
     
    guard let image = CGImage(width: bufferWidth, height: bufferHeight,
                 bitsPerComponent: bitsPerComponent,
                 bitsPerPixel:   	 bitsPerPixel,
                 bytesPerRow:   	 bytesPerRow,
                 space:     			 colorspace,
                 bitmapInfo:   		 bitmapInfo,
                 provider:     		 provider,
                 decode: nil, shouldInterpolate: false,
                 intent: .relativeColorimetric)
    else { return nil }
     
	let scale = useScale ? UIScreen.main.scale : 1
	let img = UIImage(cgImage: image, scale: scale, orientation: .up)
	return img
}

Boost

Answer 1

Graphics and Games Engineer OP

Apple

Jun ’20

CGImage doesn't natively recognize the XR10 formats. So you will need to convert from the XR10 format to one that CGImage will recognize, like a 32 bit per component float format, before providing your data to it. Alternatively, you can avoid using an XR10 format and use a Metal format like RGBA16Float which both supports P3 wide colors and CGImage will recognize

Note that the RGBA16Float generally is not as fast as the XR10 formats, although it uses the same amount of space and bandwidth as BGRA10XR (but more space and bandwidth than BGR10XR since Alpha uses an additional 32-bits per pixel).

The following function would convert a single 10 bit XR10 component to Float32:

Code Block func XR10ToFloat32(xr10Val: Int16) -> Float {
    return Float(xr10Val - 384) / 510.0;
}

You would need to do some bit shifting and masking to extract the 10bit from each channel of the 64 or 32 bit XR10 pixel.

Another important concept to understand about the XR10 formats is that while they can express the full P3 gamut (range of colors), they express it in terms of sRGB colorspace. sRGB has a more limited range so it works by expressing values within the sRGB gamut with values between 0.0-1.0 and expressing wider color values (such as those in the P3 gamut) with values outside that range. (There are some legacy reasons for expressing colors relative to sRGB and not as P3 directly).

If you choose to render to an XR10_sRGB format and use the XR10ToFloat32 equation above, you would need to use the following to setup your arguments to the CGImage creation function:

Code Block if self.isBGRA10_XR {
	bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.premultipliedLast.rawValue).union(.byteOrder32Little)
		bitmapInfo |= .floatComponents
		bitsPerComponent = 32
		bitsPerPixel  	 = 128
		colorspace   	 = CGColorSpace(name: CGColorSpace.extendedSRGB)!
} ...

If instead you were to render to RGBAFloat16 instead of using an XR10 format, you can provide the data directly to the CGImage and use the following to setup your arguments:

Code Block if self.isBGRA10_XR {
	bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.premultipliedLast.rawValue).union(.byteOrder16Little)
		bitmapInfo |= .floatComponents
		bitsPerComponent = 16
		bitsPerPixel  	 = 64
		colorspace   	 = CGColorSpace(name: CGColorSpace.extendedLinearSRGB)!
} ...

Note that the colorspace has changed from .extendedSRGB to .extendedLinearSRGB in this case. This is because the RGBA16Float format is a linear format. The non sRGB XR10 formats are also linear and would also require specifying Linear version of the colorspace as well.

0