Sample Code

Standardizing Arbitrary Image Formats for Processing

Convert assets with disparate color spaces and bit depths to a standard working format for applying vImage operations.

Download

Overview

To apply a vImage operation to an image, you must know its bit depth (the number of bits required to represent each pixel) and its color space (the number and organization of the color channels the image contains). For example, to apply an affine transform to a 32-bit image, you call vImageAffineWarp_ARGBFFFF(_:_:_:_:_:_:). To apply the same transform to an 8-bit image, you call vImageAffineWarp_ARGB8888(_:_:_:_:_:_:).

vImage’s vImageConvert_AnyToAny(_:_:_:_:_:) function helps solve this issue by enabling you to dynamically create converters based on the properties of a source image. By converting all source assets to a standardized working format, you need only one operation.

This sample uses a vImage tent filter to apply a blur to UIImage objects of arbitrary formats. Because a UIImage object can contain image data in many formats and color spaces, this implementation converts all input to 8-bit ARGB format before processing.

The example below shows an original 16-bit CMYK image on the left, and the same image converted to 8-bit ARGB and blurred on the right.

Photos showing the original image and a blurred version of the same image.

This sample walks you through the steps for applying a blur to an image of any format:

  1. Implementing the blurring function.

  2. Creating the source and destination image formats.

  3. Creating the source and destination buffers.

  4. Performing the conversion.

  5. Applying the blur operation.

  6. Returning the blurred result.

  7. Using the blurring function.

Implement the Blurring Function

The code to apply a blur to a UIImage instance with an arbitrary image format is implemented in blurImage(_:blurWidth:blurHeight:). This function accepts three parameters: the image to blur, and the width and height (in pixels) of the blur:

func blurImage(_ sourceImage: UIImage,
               blurWidth: UInt32,
               blurHeight: UInt32) -> UIImage? {

Create the Source and Destination Image Formats

To learn how to create a vImage_CGImageFormat structure from properties derived from the source image, see Creating a Core Graphics Image Format. The properties of the source format are derived from source image. The properties of the destination format are hard coded to match the convolution operation used later in the function:

guard
    let cgImage = sourceImage.cgImage,
    var sourceImageFormat = vImage_CGImageFormat(cgImage: cgImage),
    var rgbDestinationImageFormat = vImage_CGImageFormat(
        bitsPerComponent: 8,
        bitsPerPixel: 32,
        colorSpace: CGColorSpaceCreateDeviceRGB(),
        bitmapInfo: CGBitmapInfo(rawValue: CGImageAlphaInfo.first.rawValue),
        renderingIntent: .defaultIntent) else {
            print("Unable to initialize cgImage or colorSpace.")
            return nil
}

Create the Source and Destination Buffers

With the source and destination image formats defined, you create and initialize a buffer containing the source image, and a buffer that will contain the 8-bit, ARGB conversion of the source image. Be sure to free the memory allocated to these buffers when you’re finished working with them by using the free() function. Because blurImage(_:blurWidth:blurHeight:) may exit early, defer both of these free calls to ensure that they’re always called.

guard
    let sourceBuffer = try? vImage_Buffer(cgImage: cgImage),
    var rgbDestinationBuffer = try? vImage_Buffer(width: Int(sourceBuffer.width),
                                                  height: Int(sourceBuffer.height),
                                                  bitsPerPixel: rgbDestinationImageFormat.bitsPerPixel) else {
                                                    fatalError("Error initializing source and destination buffers.")
}

defer {
    sourceBuffer.free()
    rgbDestinationBuffer.free()
}

Perform the Conversion

Use the source and destination formats to create a converter using the make(sourceFormat:destinationFormat:flags:) function. The converter’s convert(source:destination:flags:) function performs the conversion.

do {
    let toRgbConverter = try vImageConverter.make(sourceFormat: sourceImageFormat,
                                                  destinationFormat: rgbDestinationImageFormat)
    
    try toRgbConverter.convert(source: sourceBuffer,
                               destination: &rgbDestinationBuffer)
} catch {
    fatalError(error.localizedDescription)
}

On return, your destination buffer contains the image in 8-bit ARGB color space.

Apply the Blur Operation

Create a blurred version of the converted image using vImage’s tent filter. This filter calculates a weighted average of pixels within a surrounding grid, known as a kernel, with a size specified by the blurWidth and blurHeight parameters. The tent filter uses a fast algorithm that’s suited for real-time applications.

Create a buffer to receive the tent filter’s result using the same technique that you used for the other buffers. Be sure to free the buffer’s memory when the you’re finished using it.

The width and height values passed to vImageTentConvolve_ARGB8888(_:_:_:_:_:_:_:_:_:) must be odd so that the center of the kernel aligns with each pixel. To guarantee that the kernel sizes passed to the convolve function are odd, calculate oddWidth and oddHeight to add one to any even values passed into the function.

guard var blurResultBuffer = try? vImage_Buffer(width: Int(sourceBuffer.width),
                                                height: Int(sourceBuffer.height),
                                                bitsPerPixel: rgbDestinationImageFormat.bitsPerPixel) else {
                                                    fatalError("Error creating blur result buffer.")
}

defer {
    blurResultBuffer.free()
}

let oddWidth = blurWidth % 2 == 0 ? blurWidth + 1 : blurWidth
let oddHeight = blurHeight % 2 == 0 ? blurHeight + 1 : blurHeight

let error = vImageTentConvolve_ARGB8888(&rgbDestinationBuffer,
                                        &blurResultBuffer,
                                        nil,
                                        0, 0,
                                        oddHeight, oddWidth,
                                        nil,
                                        vImage_Flags(kvImageEdgeExtend))

guard error == kvImageNoError else {
    print("Error in vImageTentConvolve_ARGB8888.")
    return nil
}

Return the Blurred Result

After vImageTentConvolve_ARGB8888(_:_:_:_:_:_:_:_:_:) has completed, blurResultBuffer contains a blurred version of the original image. Your function creates a CGImage representation from the blurred result and returns a UIImage instance from that.

if let cgImage = try? blurResultBuffer.createCGImage(format: rgbDestinationImageFormat) {
    return UIImage(cgImage: cgImage)
} else {
    return nil
}

Use the Blurring Function

This code shows an example usage of the blurImage(_:blurWidth:blurHeight:) function:

let flowers = #imageLiteral(resourceName: "Flowers_2.jpg")

imageView.image = blurImage(flowers,
                            blurWidth: 48,
                            blurHeight: 48)

See Also

vImage Operations

Adjusting the Brightness and Contrast of an Image

Use a gamma function to apply a linear or exponential curve.

Adjusting Saturation and Applying Tone Mapping

Convert an RGB image to discrete luminance and chrominance channels, and apply color and contrast treatments.

Blurring an Image

Filter an image by convolving it with custom and high-speed kernels.

Adding a Bokeh Effect

Simulate a bokeh effect by applying dilation.

Converting Color Images to Grayscale

Convert a color image to grayscale using matrix multiplication.

Specifying Histograms with vImage

Calculate the histogram of one image and apply it to a second image.

Reducing Artifacts in Resampled Images

Avoid ringing effects introduced by the default Lanczos algorithm when scaling an image by using a custom resampling filter.

Finding the Sharpest Image in a Sequence of Captured Images

Share image data between vDSP and vImage to compute the sharpest image from a bracketed photo sequence.

vImage Operations

Apply image manipulation operations to vImage buffers.