Sample Code

Halftone Descreening with 2D Fast Fourier Transform

Reduce or remove periodic artifacts from images.

Download

Overview

Accelerate’s vDSP module provides functions to perform 2D fast Fourier transforms (FFTs) on matrices of data, such as images. You can exploit the amplitude peaks in the frequency domain of periodic patterns, such as halftone screens, to reduce or remove such artifacts from images. The example below shows an image with halftone artifacts (left) and the same image with the halftone artifacts reduced (right):

Photographs showing before and after images.

Using a halftone screen sample that you’ve created programmatically or taken from source material, you’ll follow these steps for halftone descreening of an image:

  1. Converting the image data to a split complex vector

  2. Preparing the FFT setup

  3. Performing forward 2D FFTs on image data

  4. Zeroing the peaks in the halftone sample magnitude

  5. Descreening the source image

  6. Performing inverse 2D FFT on the source image frequency domain data

  7. Generating an image from a split complex vector

The sample works on a single-channel, monochrome image.

Convert Image Data to a Split Complex Vector

Create a split complex vector—suitable for use with vDSP’s 2D FFT—by copying odd pixels to the real parts and even pixels to the imaginary parts of an array of complex numbers. Use the following code to convert the 8-bit unsigned integer image data to floating-point data that the 2D FFT routine works with:

let pixelCount = Int(image.size.width * image.size.height)

let pixelData = cgImage.dataProvider?.data

let pixelsArray = Array(UnsafeBufferPointer(start: CFDataGetBytePtr(pixelData),
                                            count: pixelCount))

let floatPixels = vDSP.integerToFloatingPoint(pixelsArray,
                                              floatingPointType: Float.self)

With the floating-point values populated, create an array of complex numbers, interleavedPixels, that you pass to convert(interleavedComplexVector:toSplitComplexVector:), which copies the values into a split complex vector:

let interleavedPixels = stride(from: 1, to: floatPixels.count, by: 2).map {
    return DSPComplex(real: floatPixels[$0.advanced(by: -1)],
                      imag: floatPixels[$0])
}

vDSP.convert(interleavedComplexVector: interleavedPixels,
             toSplitComplexVector: &splitComplexOut)

Prepare the 2D FFT Setup

Create a setup object that contains all the information required to perform the forward and inverse 2D FFT operations. Creating this setup object can be expensive, so do it only once—for example, when your app is starting—and reuse it.

The following code creates a setup object suitable for performing forward and inverse 2D FFTs on a 1024 x 1024 pixel image:

static let fftSetUp = vDSP.FFT2D(width: 1024,
                                 height: 1024,
                                 ofType: DSPSplitComplex.self)

Perform Forward 2D FFTs on Image Data

Use the transform(input:output:direction:) function to perform a forward 2D FFT on the image data, creating the frequency domain representation of the image. Pass transform(input:output:direction:) a split complex structure as the destination, with the same length as the source structure.

The following example shows the code required to perform the FFT on the source image data that populates sourceImage_floatPixels_frequency. Repeat this step for the halftone sample image, populating halftoneSample_floatPixels_frequency.

let width = Int(size.width)
let height = Int(size.height)
let pixelCount = width * height
let n = pixelCount / 2

var sourceImage_floatPixelsReal_spatial = [Float](repeating: 0,
                                                  count: n)
var sourceImage_floatPixelsImag_frequency = [Float](repeating: 0,
                                                    count: n)
var sourceImage_floatPixels_frequency = DSPSplitComplex(
    realp: &sourceImage_floatPixelsReal_spatial,
    imagp: &sourceImage_floatPixelsImag_frequency)

fftSetUp?.transform(input: sourceImageSplitComplex,
                    output: &sourceImage_floatPixels_frequency,
                    direction: .forward)

Zero the Peaks in the Halftone Sample Magnitude

You can reduce the halftone screen artifacts by manipulating the magnitude of the frequency domain data for the halftone sample. Zero all the samples above a specified threshold in the magnitudes and clamp the data to 0…1. Then multiply the frequency domain data of the source image by the manipulated magnitudes.

The squareMagnitudes(_:result:) function computes the magnitude of the complex values representing the halftone sample:

var halftoneSampleAmplitude = [Float](repeating: 0,
                                      count: n)

vDSP.squareMagnitudes(halftoneSample_floatPixels_frequency,
                      result: &halftoneSampleAmplitude)

Use the threshold(_:to:with:) function to set all magnitude values that are over the threshold to -1, and all magnitude values that are less than or equal to the threshold to 1:

let outputConstant: Float = -1

vDSP.threshold(halftoneSampleAmplitude,
               to: threshold,
               with: .signedConstant(outputConstant),
               result: &halftoneSampleAmplitude)

You can now clip the magnitude data between 0 and 1. After clip(_:to:result:) returns, all the originally high-magnitude values in halftoneSampleAmplitude are set to 0, and all the originally low-magnitude values are set to 1:

vDSP.clip(halftoneSampleAmplitude,
          to: 0 ... 1,
          result: &halftoneSampleAmplitude)

Descreen the Source Image

Multiply the source image frequency domain data by the values in halftoneSampleAmplitude to remove or reduce the halftone screen:

vDSP.multiply(sourceImage_floatPixels_frequency,
              by: halftoneSampleAmplitude,
              result: &sourceImage_floatPixels_frequency)

Perform Inverse 2D FFT on Source Image Frequency Domain Data

You can now perform an inverse 2D FFT to generate a spatial domain version of the image. Use the same fftSetup pointer as you used for the forward 2D FFT, but specify the inverse direction:

var floatPixelsReal_spatial = [Float](repeating: 0,
                                      count: n)
var floatPixelsImag_spatial = [Float](repeating: 0,
                                      count: n)
var floatPixels_spatial = DSPSplitComplex(realp: &floatPixelsReal_spatial,
                                          imagp: &floatPixelsImag_spatial)

fftSetUp?.transform(input: sourceImage_floatPixels_frequency,
                    output: &floatPixels_spatial,
                    direction: .inverse)

Generate an Image from a Split Complex Vector

The last step is to create a displayable image from the spatial domain representation of the treated source image. The final image is generated from 8-bit, unsigned integers. Because single-precision values can exceed the range of an 8-bit, unsigned integer, clamp the values before converting them:

var low: Float = 0
var high: Float = 255

vDSP_vclip(pixelSource.realp,
           stride,
           &low,
           &high,
           pixelSource.realp,
           stride,
           n)

vDSP_vclip(pixelSource.imagp,
           stride,
           &low,
           &high,
           pixelSource.imagp,
           stride, n)

Use convertElements(of:to:) to convert the floating-point values back to 8-bit unsigned integers suitable for generating an image:

var uIntPixels_OUT = [UInt8](repeating: 0,
                             count: pixelCount)

let floatPixels = [Float](fromSplitComplex: pixelSource,
                          scale: 1,
                          count: pixelCount)

vDSP.convertElements(of: floatPixels,
                     to: &uIntPixels_OUT,
                     rounding: .towardZero)

Now that uIntPixels_OUT is populated with the pixel values, generate a CGImage instance from those values:

let buffer = vImage_Buffer(data: &uIntPixels_OUT,
                           height: vImagePixelCount(height),
                           width: vImagePixelCount(width),
                           rowBytes: width)

if
    let format = vImage_CGImageFormat(bitsPerComponent: 8,
                                      bitsPerPixel: 8,
                                      colorSpace: CGColorSpaceCreateDeviceGray(),
                                      bitmapInfo: bitmapInfo),
    let cgImage = try? buffer.createCGImage(format: format) {

    return UIImage(cgImage: cgImage)
} else {
    print("Unable to create CGImage")
    return nil
}

See Also