Hello,
In one of my apps, I'm trying to modify the pixel buffer from a ProRAW capture to then write the modified DNG.
This is what I try to do:
After capturing a ProRAW photo, I work in the delegate function func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, error: Error?) { ... }
In here I can access the photo.pixelBuffer and get its base address:
guard let buffer = photo.pixelBuffer else { return }
CVPixelBufferLockBaseAddress(buffer, [])
let pixelFormat = CVPixelBufferGetPixelFormatType(buffer)
// I check that the pixel format corresponds with ProRAW . This is successful, the code enters the if block
if (pixelFormat == kCVPixelFormatType_64RGBALE) {
guard let pointer = CVPixelBufferGetBaseAddress(buffer) else { return }
// We have 16bits per component, 4 components
let count = CVPixelBufferGetWidth(buffer) * CVPixelBufferGetHeight(buffer) * 4
let mutable = pointer.bindMemory(to: UInt16.self, capacity: count)
// As a test, I want to replace all pixels with 65000 to get a white image
let finalBufferArray : [Float] = Array.init(repeating: 65000, count: count) vDSP_vfixu16(finalBufferArray, 1, mutable, 1, vDSP_Length(finalBufferArray.count))
// I create an vImage Pixel buffer. Note that I'm referencing the photo.pixelBuffer to be sure that I modified the underlying pixelBuffer of the AVCapturePhoto object
let imageBuffer = vImage.PixelBuffer<vImage.Interleaved16Ux4>(referencing: photo.pixelBuffer!, planeIndex: 0)
// Inspect the CGImage
let cgImageFormat = vImage_CGImageFormat(bitsPerComponent: 16, bitsPerPixel: 64, colorSpace: CGColorSpace(name: CGColorSpace.displayP3)!, bitmapInfo: CGBitmapInfo(rawValue: CGImageAlphaInfo.last.rawValue | CGBitmapInfo.byteOrder16Little.rawValue))!
let cgImage = imageBuffer.makeCGImage(cgImageFormat: cgImageFormat)!
// I send the CGImage to the main view controller. This is successful, I can see a white image when rendering the CGImage into a UIImage. This lets me think that I successfully modified the photo.pixelBuffer
firingFrameDelegate?.didSendCGImage(image: cgImage)
}
// Now I try to write data. Unfortunately, this does not work. The photo.fileDataRepresentation() writes the data corresponding to the original, unmodified pixelBuffer
`if let photoData = photo.fileDataRepresentation() {
// Sending the data to the view controller and rendering it in the UIImage displays the original photo, not the modified pixelBuffer
firingFrameDelegate?.didSendData(data: photoData)
thisPhotoData = photoData
}`
CVPixelBufferUnlockBaseAddress(buffer, [])
The same happens if I try to write the data to disk. The DNG file displays the original photo and not the data corresponding to the modified photo.pixelBuffer.
Do you know why this code should not work? Do you have any ideas on how I can modify the ProRAW pixel buffer so that I can write the modified buffer into a DNG file?
My goal is to write a modified file, so, I'm not sure I can use CoreImage of vImage to output a ProRAW file.