I am processing CVPixelBuffers received from camera using both Metal and CoreImage, and comparing the performance. The only processing that is done is taking a source pixel buffer and applying crop & affine transforms, and saving the result to another pixel buffer. What I do notice is CPU usage is as high a 50% when using CoreImage and only 20% when using Metal. The profiler shows most of the time spent is in CIContext render: let cropRect = AVMakeRect(aspectRatio: CGSize(width: dstWidth, height: dstHeight), insideRect: srcImage.extent) var dstImage = srcImage.cropped(to: cropRect) let translationTransform = CGAffineTransform(translationX: -cropRect.minX, y: -cropRect.minY) var transform = CGAffineTransform.identity transform = transform.concatenating(CGAffineTransform(translationX: -(dstImage.extent.origin.x + dstImage.extent.width/2), y: -(dstImage.extent.origin.y + dstImage.extent.height/2))) transform = transform.concatenating(translationTransform) transform = transform.concatenating(CGAffineTransform(translationX: (dstImage.extent.origin.x + dstImage.extent.width/2), y: (dstImage.extent.origin.y + dstImage.extent.height/2))) dstImage = dstImage.transformed(by: translationTransform) let scale = max(dstWidth/(dstImage.extent.width), CGFloat(dstHeight/dstImage.extent.height)) let scalingTransform = CGAffineTransform(scaleX: scale, y: scale) transform = CGAffineTransform.identity transform = transform.concatenating(scalingTransform) dstImage = dstImage.transformed(by: transform) if flipVertical { dstImage = dstImage.transformed(by: CGAffineTransform(scaleX: 1, y: -1)) dstImage = dstImage.transformed(by: CGAffineTransform(translationX: 0, y: dstImage.extent.size.height)) } if flipHorizontal { dstImage = dstImage.transformed(by: CGAffineTransform(scaleX: -1, y: 1)) dstImage = dstImage.transformed(by: CGAffineTransform(translationX: dstImage.extent.size.width, y: 0)) } var dstBounds = CGRect.zero dstBounds.size = dstImage.extent.size _ciContext.render(dstImage, to: dstPixelBuffer!, bounds: dstImage.extent, colorSpace: srcImage.colorSpace ) Here is how CIContext was created: _ciContext = CIContext(mtlDevice: MTLCreateSystemDefaultDevice()!, options: [CIContextOption.cacheIntermediates: false]) I want to know if I am doing anything wrong and what could be done to lower CPU usage in CoreImage?

macOS 14 Sonoma cifilter issue (Designed for iPad) I just updated my M1 Mac to MacOS 14 Sonoma. I ran the iOS/iPadOS app built in Xcode 15.0 on a Mac whose OS was updated using Testflight. (My app is Designed for iPad and is set to run on macOS.) When UIImage is processed using cifilter, the generated image may be grayed out. The same code does not occur on iOS and iPadOS. In other words, it does not occur on iPhone and iPad iOS/iPadOS 17. Has anyone experienced a similar issue? And I would like to know the solution ASAP. I am distributing the app as Designed for iPad. Every year, when Mac OS is updated, an issue arises. So, I am considering not allowing Vision Pro to run the current iOS/iPadOS app.

Hello All, I am trying to compress PNG image by applying PNG Filters like(Sub, Up, Average, Paeth), I am applying filers using property kCGImagePropertyPNGCompressionFilter but there is no change seen in resultant images after trying any of the filter. What is the issue here can someone help me with this. Do I have compress image data after applying filter? If yes how to do that? Here is my source code CGImageDestinationRef outImageDestRef = NULL; long keyCounter = kzero; CFStringRef dstImageFormatStrRef = NULL; CFMutableDataRef destDataRef = CFDataCreateMutable(kCFAllocatorDefault,0); Handle srcHndl = //source image handle; ImageTypes srcImageType = //'JPEG', 'PNGf, etct; CGImageRef inImageRef = CreateCGImageFromHandle(srcHndl,srcImageType); if(inImageRef) { CFTypeRef keys[4] = {nil}; CFTypeRef values[4] = {nil}; dstImageFormatStrRef = CFSTR("public.png"); long png_filter = IMAGEIO_PNG_FILTER_SUB; //IMAGEIO_PNG_FILTER_SUB, IMAGEIO_PNG_FILTER_UP, IMAGEIO_PNG_FILTER_AVG, IMAGEIO_PNG_FILTER_PAETH .. it is one of this at a time keys[keyCounter] = kCGImagePropertyPNGCompressionFilter; values[keyCounter] = CFNumberCreate(NULL,kCFNumberLongType,&png_filter); keyCounter++; outImageDestRef = CGImageDestinationCreateWithData(destDataRef, dstImageFormatStrRef, 1, NULL); if(outImageDestRef) { // keys[keyCounter] = kCGImagePropertyDPIWidth; // values[keyCounter] = CFNumberCreate(NULL,kCFNumberLongType,&Resolution); // keyCounter++; // // keys[keyCounter] = kCGImagePropertyDPIHeight; // values[keyCounter] = CFNumberCreate(NULL,kCFNumberLongType,&Resolution); // keyCounter++; CFDictionaryRef options = CFDictionaryCreate(NULL,keys,values,keyCounter,&kCFTypeDictionaryKeyCallBacks,&kCFTypeDictionaryValueCallBacks); CGImageDestinationAddImage(outImageDestRef,inImageRef, options); CFRelease(options); status = CGImageDestinationFinalize(outImageDestRef); if(status == true) { UInt8 *destImagePtr = CFDataGetMutableBytePtr(destDataRef); destSize = CFDataGetLength(destDataRef); //using destImagePtr after this ... } CFRelease(outImageDestRef); } for(long cnt = kzero; cnt < keyCounter; cnt++) if(values[cnt]) CFRelease(values[cnt]); if(inImageRef) CGImageRelease(inImageRef); }

Hello! After recent talk on the WWDC2023 about HDR support and finding this documentation page on Applying Apple HDR effect on photos, I became very interested in the HDR Gain Map format. From documentation page it is clear how we can restore original HDR from SDR and Gain Map representation, but my question is - how from HDR we can convert back to the SDR + Gain Map representation? As I understand right know, conversion from HDR to SDR + Gain Map includes two steps: Tone mapping of HDR for getting correct SDR When we have both HDR and SDR, from equation in the documentation page we can calculate Gain Map Am I correct? If so, what tone mapping algorithm for HDR -> SDR conversion is used right know? Can't find any information about this in the internet:( Would be very grateful for your response!

HEIF Decompression Crash on iOS 17.

I'm trying to create a sky mask on pictures taken from my iPhone. I've seen in the documentation that CoreImage support semantic segmentation for Sky among other type for person (skin, hair etc...) For now, I didn't found the proper workflow to use it. First, I watched https://developer.apple.com/videos/play/wwdc2019/225/ I understood that images must be captured with the segmentation with this kind of code: photoSettings.enabledSemanticSegmentationMatteTypes = self.photoOutput.availableSemanticSegmentationMatteTypes photoSettings.embedsSemanticSegmentationMattesInPhoto = true I capture the image on my iPhone, save it as HEIC format then later, I try to load the matte like that : let skyMatte = CIImage(contentsOf: imageURL, options: [.auxiliarySemanticSegmentationSkyMatte: true]) Unfortunately, self.photoOutput.availableSemanticSegmentationMatteTypes always give me a list of types for person only and never a types Sky. Anyway, the AVSemanticSegmentationMatte.MatteType is just [Hair, Skin, Teeth, Glasses] ... No Sky !!! So, How am I supposed to use semanticSegmentationSkyMatteImage ?!? Is there any simple workaround ?

Hello, I am a student and I am doing a search for my thesis on create ML and shape recognition and image processing, so for this subject I want to find the details of the steps used in create ML for this, such as the techniques used for pre-processing, and the methods of extracting characteristics, and the filters applied, ect...

When initializing a CIColor with a dynamic UIColor (like the system colors that resolve differently based on light/dark mode) on macOS 14 (Mac Catalyst), the resulting CIColor is invalid/uninitialized. For instance: po CIColor(color: UIColor.systemGray2) → <uninitialized> po CIColor(color: UIColor.systemGray2.resolvedColor(with: .current)) → <CIColor 0x60000339afd0 (0.388235 0.388235 0.4 1) ExtendedSRGB> But also, not all colors work even when resolved: po CIColor(color: UIColor.systemGray.resolvedColor(with: .current)) → <uninitialized> I think this is caused by the color space of the resulting UIColor: po UIColor.systemGray.resolvedColor(with: .current) → kCGColorSpaceModelRGB 0.596078 0.596078 0.615686 1 po UIColor.systemGray2.resolvedColor(with: .current) → UIExtendedSRGBColorSpace 0.388235 0.388235 0.4 1 This worked correctly before in macOS 13.