The Sonogram View Demo Project

For an overview of how audio unit projects are laid out, refer to Audio Unit Programming Guide. That document also describes the role of the Core Audio SDK in building audio units.

The CASonogramView Class

The CASonogramView class, written in Objective-C, draws the sonogram for the audio unit. You can see its implementation in the corresponding source and interface files in the project’s CocoaUI directory.

This class is a subclass of the SonogramViewDemoView class (in the same folder). It uses an NSTimer object for updating the view content. A CASonogramView object draws into a CIImage object. For more on the CIImage class, which is part of Core Image’s Quartz Core framework, see CIImage Class Reference.

A CASonogramView object makes use of a CARingBuffer object for storing processed frames of audio data. The implementation files for the CARingBuffer class are in the Core Audio SDK’s PublicUtility directory.

The CASpectralProcessor Class

The CASpectralProcessor class, written in C++, supports audio processing by converting audio data from the time domain into the frequency domain, or vice versa. It is part of the Core Audio SDK, in the PublicUtility directory.

You can use this class in three ways:

• Roundtrip processing, using the Process method. This takes time domain data, converts it to the frequency domain, lets you process it, and then converts it back to the time domain. This is the method to use for fast Fourier transform (FFT) work.

• Time-to-frequency domain processing, using the ProcessForwards method. This method lets you retrieve audio data and work with it in the frequency domain. The Sonogram View Demo audio unit uses this method.

• Frequency-to-time domain processing, using the ProcessBackwards method. This performs the inverse of the ProcessForwards method, converting frequency domain data to the time domain.

The CASpectralProcessor class’s initialization method is:

Listing 1  CASpectralProcessor constructor

CASpectralProcessor (

    UInt32 inFFTSize,        // The size of the FFT you want to perform

    UInt32 inHopSize,        // The overlap between frames

    UInt32 inNumChannels,    // The number of audio channels to process

    UInt32 inMaxFrames       // the maximum number of frames received per slice

);

Because the Sonogram View Demo audio unit does not perform digital signal processing (DSP) on its incoming audio data—it simply visualizes it—it uses the CASpectralProcessor class’s ProcessForwards method:

Listing 2  CASpectralProcessor::ProcessForwards

bool ProcessForwards (

    UInt32 inNumFrames,

    AudioBufferList* inInput

);

If you want to use a CASpectralProcessor object to process audio, you define a DSP function and then give the object a pointer to your function, using the SetSpectralFunction method:

Listing 3  CASpectralProcessor::SetSpectralFunction

void SetSpectralFunction (

    SpectralFunction  inFunction,

    void              *inUserData

);

The SonogramViewDemo and SonogramViewDemoView classes

The SonogramViewDemo class is the main class for the audio unit. It does its work in its Render method:

Listing 4  SonogramViewDemo::Render

ComponentResult Render (

    AudioUnitRenderActionFlags  &ioActionFlags,

    const AudioTimeStamp        &inTimeStamp,

    UInt32                      inFramesToProcess

);

This method places its output data into a CARing object, which is then queried by the audio unit’s view.

The SonogramViewDemoView class is the main class for the audio unit’s view. It periodically requests the latest data to display, using its updateSpectrum method:

The view then sends the frequency domain audio data to the CASonogramView object, which buffers the data in RGBA (Red Green Blue + Alpha) format. The CASonogramView object goes on to draw the spectrogram by overriding the NSView class’s drawRect method:

To perform the drawing, the CASonogramView object first instantiates a CIImage object. It then fetches the buffered RGBA data and sends it to the CIImage object. The CIImage object, in turn, aligns and filters the data using Core Image filters.