Audio Units is Apple's audio plug-in technology that allows you to
create audio effects and virtual instruments for Apple applications such
as GarageBand, Soundtrack, Logic and Final
Cut Pro, as well as a multitude of popular third-party applications and
systems.
This is possible because of the way that Core Audio integrates a range of audio functionality
directly into the Mac OS X operating system in ways never before possible,
enabling unprecedented performance and ease of use in a virtual studio.
Core Audio lays a new foundation for the next generation of world-class
audio and music applications. Through Audio Units, Core Audio enables
developers to offer audio plug-ins in a more centralized manner, making
it simpler for users to manage the audio capabilities of their setup.
You can use the techniques discussed in this article to build your own Audio
Units using Xcode. By starting with a relatively simple example, this
article will give you a basic understanding of Audio Unit construction,
yet provide you with a solid base for creating your own audio effects.
First we'll introduce the basics of Audio Units. Then we show how
easily you can build an Audio Unit using Xcode. The sample Audio Unit
will be a Volume Unit, which applies a user-selected gain
(amplification) to an input signal. Next, we discuss testing the Volume
Unit—using a tool from the SDK—and also validation, using
Apple's Audio Units Validation Tool. Finally, the example Volume Unit is
applied as an effect to an instrument in GarageBand.
What You Need to Get Started
To build Audio Units you need the Core Audio SDK, found on
the ADC Development Kits page; note that this
article refers to the Core Audio SDK v.1.3.3.
The Core Audio SDK also includes the Audio Unit templates, which are
used in this article, that make it very easy to create a simple Audio Unit.
Once the disk images are mounted, step through the installer, which will
place the SDK files under /Developer. The directory
/Developer/Examples/CoreAudio/Documentation/ contains HTML
pages covering the Audio Unit classes provided by the SDK, plus an
overview of Audio Units development.
AU Lab (/Developer/Applications/Audio) is an application that can
be used to test your Audio Unit. /Developer/Examples/CoreAudio/Services contains several
helper apps. /Developer/Examples/CoreAudio/AudioUnits
contains sample Xcode projects, including SampleAUs>SampleEffect,
that will provide additional ideas for your own Audio Units.
Introduction to Audio Units
An Audio Unit is a software component that manipulates audio data. It
may generate, modify, amplify, or perform other processing on an audio
data stream. Structurally, an Audio Unit is a software component
containing several well-defined entry points or functions, similar to a
library. These entry points may be called by applications or by other
components. Using Xcode, you build an Audio Unit as a bundle. That
bundle gets loaded by the Component Manager and made available to all
Audio Unit -aware applications.
An Audio Unit may be used singly or in combination with other Audio
Units as part of an audio signal graph. The output of one Audio Unit may
form the input to another Audio Unit. The data stream comprising the
signal moves from one Audio Unit to the next, with each manipulating the
stream by applying an internal algorithm.
Audio Units must specify codes for the manufacturer, type and
subtype. These are four character (32-bit) sequences that help
distinguish one Audio Unit from another. The codes should be declared in
a resource file (.r extension), from which they will be
automatically compiled and included in the built product.
- The manufacturer code distinguishes your Audio Unit from those of
other vendors, and prevents namespace collisions between Audio Units of
the same type. You should register a manufacturer code at the Creator Code Registration
page.
- The type defines the general category of Audio Unit, including
effect, music device ("instrument"), output, and others. An effect
receives and processes audio input, and produces audio output. An
instrument receives a MIDI note or command, and generates the audio for
that note. This article discusses the creation of an effect unit (type
'aufx').
- The subtype is an arbitrary code: use it in any way you see fit. One
use may be to label the Audio Unit's purpose, such as delay, low pass
filter, high pass filter, and so on. The subtype used in the example for
this article is pass-through (
'Pass'). Various
four-character codes for types and subtypes are enumerated in the file
AUComponent.r.
The Component Manager loads Audio Units along with other components
during startup. It automatically looks in several locations, including
the system /Library/Audio/Plug-Ins/Components folder and
the current user's ~/Library/Audio/Plug-Ins/Components
folder. After building your Audio Unit, you can copy the bundle to
either location for system-wide or user-specific use, respectively.
The Component Manager is a Carbon Manager, and uses Resource
Manager-style resources. You will notice in the sample code several
resource types, such as 'thng', that are required for your Audio Unit to
load properly. The .r files contain resource definitions, and Xcode
invokes Rez to compile the .r files.
An Audio Unit may also specify parameters and properties:
- Parameters are values that may change over time. A parameter is
always of type
Float32, but its meaning is internal to the
Audio Unit. The gain, discussed in this article, is an example of a
parameter.
- A property may contain arbitrary data. Properties are used to
configure an Audio Unit, and the property mechanism is used to pass
values to and from an Audio Unit. For example, user interface
information may be returned as a property by the Audio Unit, and used to
create the visual display in a hosting app. There are a number of
defined properties that an Audio Unit must support, depending on the
Audio Unit type; see
AudioUnitProperties.h for an
enumerated list. Developers may also define custom properties, using ID
values at or above 64000.
A view may be included with an Audio Unit to allow the user to easily
manipulate the unit's parameters. The view, or user interface, is a
separate entity from the Audio Unit. This allows you to reuse the Audio
Unit signal processing code and provide either a Carbon or Cocoa view.
Additional information regarding the relationship between an Audio Unit
and its view (in this case, a Cocoa view) may be found in the document
WritingCocoaAUUIs.pdf, which is included in the Core Audio
SDK.
You can provide the user interface for your Audio Unit in a nib file,
or dynamically construct one using function calls from within a view
class. If you do not provide either, the hosting app can query your
Audio Unit's parameters (using the property mechanism) and construct an
appropriate interface on-the-fly. Audio Unit host applications provide a
window within which your AU's view will draw. In GarageBand, for
example, when the user selects your Audio Unit as an effect for an
instrument or track, and chooses to modify the settings, GarageBand pops
up a window containing your Audio Unit's user interface.
Because a component cannot run independently, the
AudioUnitHosting test tool (which is a hosting application,
like GarageBand) from the CoreAudio SDK is used in this article to test
both the Volume Unit's user interface and its internal processing.
AudioUnitHosting presents the user interface within a test
harness. AudioUnitHosting also plays audio files of various
formats (including AIFF and WAV); you simply drop files onto the app for
playback. It will first pass the data stream to your component for
processing, then play the result over the current default output device
(typically the speaker or headphones).
Creating an Audio Unit Using Xcode
This section shows how to create a Volume Unit, which is a type of
audio effect. A Volume Unit allows the user to adjust the gain on an
input signal. This example presents a user interface that allows the
user to select the gain by using a slider. The slider values are marked
in tenths. The lowest value on the slider is 0.0, and the highest value
is 1.0. A value of 1.0 will pass the signal through at the same level
("unity gain"). Zero effectively reduces the signal to 0. A value less
than 1.0 attenuates the signal. A value greater than 1.0 amplifies, or
increases, the volume. (Although the example in this article stops at
1.0, you can modify the slider in Interface Builder, and also the
function MyVolumeUnit::GetParameterInfo in
MyVolumeUnit.cpp, to accept a greater maximum value.)
You should provide reasonable default values for the Audio Unit's
parameters, just in case the user does not check the settings before
use. You need to be careful to avoid "clipping" when amplifying a
signal. Clipping occurs when an amplifier is asked to increase the
output current (delivered to the speakers) to a level beyond its
capabilities. The amplifier responds by eliminating, or "clipping", the
upper and lower portions of the waveform, which distorts the sound and
may result in speaker damage.
Several C++ classes provide common functionality and a default
implementation for Audio Units. These classes are automatically included
in your Xcode project when you use the templates to create the project
(more on this later). ComponentBase is the base class, and
contains a constructor, a destructor, a version method, and so on.
AUBase inherits from ComponentBase, and adds
support for rendering, property and parameter retrieval, presets, and
other functionality relating to Audio Units. Some of these methods are
declared virtual, and are implemented in child classes, such as
AUEffectBase. The Volume Unit presented in this article is
an audio effect unit. Its implementation class is
MyVolumeUnit, which inherits from
AUEffectBase.
Graphically, this relationship looks like:
ComponentBase > AUBase > AUEffectBase > MyVolumeUnit
As you look through the header files you will notice an associative
relationship between AUEffectBase and the class
AUKernelBase: AUEffectBase maintains an
internal reference to one or more AUKernelBase instances.
However, AUKernelBase is not a part of the hierarchy.
Signal Processing
The processing algorithm is contained in the
AUKernelBase::Process function. In the sample, that
function simply gets the current value of the slider, and applies it to
the input signal. The signal is automatically routed to the next
component for processing.
Because the Volume Unit inherits from AUEffectBase, you
must provide an implementation for its internal
AUKernelBase instance. AUKernelBase contains
two functions responsible for rendering the audio signal:
// Your signal processing code should go in an override of this function.
virtual void Process(
const Float32* inSourceP,
Float32* inDestP,
UInt32 inFramesToProcess,
UInt32 inNumChannels,
bool& ioSilence
) = 0;
// Override this function to reset your unit for restarting rendering.
virtual void Reset();
Notice that the input signal is presented to Process as
a series of discrete Float32 pointer values. This allows
you to perform the same data manipulation without worrying about the
location of the value in the data stream. On the other hand, for a more
exotic result, such as varying the output level in a wave-like fashion,
you could apply variations of the effect to every ten, hundred, or
thousand data elements. Go wild!
For those readers unfamiliar with C++, the keyword
virtual means that this function may be overridden.
Assigning a value of 0 to the function, such as for
Process, marks this as a pure virtual function.
The compiler requires that an implementation be provided for pure
virtual functions in any derived class.
The implementation of Process created by the template
applies the current gain setting from the user interface to the input
signal. Reset does nothing.
Creating an Audio Unit Using Xcode
This Audio Unit will be built using the Xcode Audio Unit Cocoa View
project template (Audio_Unit_Effect_with_Cocoa_View in
Figure 1). The Xcode templates save you a lot of time by putting in
place the appropriate C++ class and resource file scaffolding. If you
prefer, you can build either the Carbon or generic
(Audio_Unit_Effect with no suffix) View instead.
Figure 1: Audio Unit Project Templates
You can name the project whatever you like (here,
MyVolumeUnit). Xcode generates a project that looks like
Figure 2. Notice the automatic inclusion of the SDK classes at the top
of the file list.
Figure 2: MyVolumeUnit Project in Xcode
The template adds just enough functionality to allow hosting,
testing, and verification of your Audio Unit. It performs minimal signal
manipulation, and the user interface is very simple. You can find
additional implementation details and ideas for enhancements in the
SampleEffect project in the CoreAudio SDK, in
/Developer/Examples/CoreAudio/AudioUnits/SampleAUs.
You construct the user interface with Interface Builder. If you start
from the Cocoa project template, a nib file containing a simple gain
slider in a CocoaView will be automatically included. Refer
to Figure 3. The Carbon template constructs the same slider, but uses
the Control Manager API from within an AUCarbonViewBase
subclass. The generic view does not create any user interface class or
controls, but at runtime the hosting app will query the Audio Unit and
build an interface based on the Audio Unit's properties.
Figure 3: Initial Volume Unit User Interface in Interface Builder
Implementation
The real work of the Audio Unit is performed in the
Process function, as shown in Listing 1. The Volume Unit
applies the user selected gain to the audio data. Note that the data is
presented in discrete chunks of type Float32 (the native
format for Core Audio data), which greatly simplifies the math in this
particular Audio Unit.
Listing 1: Applying the Gain Value
void MyVolumeUnit::MyVolumeUnitKernel::Process( const Float32 *inSourceP,
Float32 *inDestP,
UInt32 inFramesToProcess,
UInt32 inNumChannels,
bool &ioSilence )
{
UInt32 nSampleFrames = inFramesToProcess;
const Float32 *sourceP = inSourceP;
Float32 *destP = inDestP;
Float32 gain = GetParameter( kGainParam );
while (nSampleFrames-- > 0) {
Float32 inputSample = *sourceP;
sourceP += inNumChannels;
// here's where you do your DSP work
Float32 outputSample = inputSample * gain;
*destP = outputSample;
destP += inNumChannels;
}
}
Testing in AudioUnitHosting
After building, copy the MyVolumeUnit.component bundle
to either /Library/Audio/Plug-Ins/Components or the current
user's ~/Library/Audio/Plug-Ins/Components folder. You may
need to log out and back in the first time, but try running
AudioUnitHosting first and see what happens. After the
first time, changes will be picked up by the Component Manager, though
you still need to copy the updated bundle to the destination folder and
restart the test application.
Now build and run the application AudioUnitHosting, located in
/Developer/Examples/Services/AudioUnitHosting/. Figure 4
shows MyVolumeUnit selected in the running hosting application. This is
prior to any resource modifications, so the company name is set to the
default.
Figure 4: MyVolumeUnit Loaded in AudioUnitHosting
Notice how the interface of Figure 4 differs from the sparse view
shown in Figure 3. This is because AudioUnitHosting hosts
Carbon views. But since you created an Audio Unit with a Cocoa View,
AudioUnitHosting queried the Audio Unit for information
about its parameters, and generated a generic view. In this case, there
is only one parameter, but it is accurately identified as a linear gain
component, and the string "Gain" has been appended to the text box
containing the current slider value. How does this happen?
Listing 2 shows MyVolumeUnit::GetParameterInfo. This function gets invoked when the hosting app queries the Audio Unit about specific parameters. In this case, it sets the unit field of the AudioUnitParameterInfo structure to the linear gain type (defined in AudioUnitProperties.h). The caller, CAAUParameter::CAAUParameter in Listing 3, uses this value to set the string suffix for the user interface element.
Listing 2: The Audio Unit Returns Info about the Gain Parameter
ComponentResult MyVolumeUnit::GetParameterInfo(AudioUnitScope inScope,
AudioUnitParameterID inParameterID,
AudioUnitParameterInfo &outParameterInfo)
{
ComponentResult result = noErr;
outParameterInfo.flags = kAudioUnitParameterFlag_IsWritable
| kAudioUnitParameterFlag_IsReadable;
if (inScope == kAudioUnitScope_Global) {
switch(inParameterID)
{
case kParam_Gain:
AUBase::FillInParameterName (outParameterInfo, kParameterOneName, false);
outParameterInfo.unit = kAudioUnitParameterUnit_LinearGain;
outParameterInfo.minValue = 0.0;
outParameterInfo.maxValue = 1;
outParameterInfo.defaultValue = kDefaultValue_Gain;
break;
default:
result = kAudioUnitErr_InvalidParameter;
break;
}
} else {
result = kAudioUnitErr_InvalidParameter;
}
return result;
}
Listing 3: The Parameter Type is used to Determine the Suffix for the Slider
CAAUParameter::CAAUParameter(AudioUnit au,
AudioUnitParameterID param,
AudioUnitScope scope,
AudioUnitElement element)
: mParamTag (0),
mNumIndexedParams (0),
mNamedParams (0)
{
mAudioUnit = au;
mParameterID = param;
mScope = scope;
mElement = element;
UInt32 propertySize = sizeof(mParamInfo);
OSStatus err = AudioUnitGetProperty(au, kAudioUnitProperty_ParameterInfo,
scope, param, &mParamInfo, &propertySize);
// ...
char* str = 0;
switch (mParamInfo.unit)
{
case kAudioUnitParameterUnit_Boolean:
str = "T/F";
break;
case kAudioUnitParameterUnit_Percent:
case kAudioUnitParameterUnit_EqualPowerCrossfade:
str = "%";
break;
// ...
case kAudioUnitParameterUnit_MixerFaderCurve1:
case kAudioUnitParameterUnit_LinearGain:
str = "Gain";
break;
default:
str = NULL;
break;
}
// ...
if (str)
mParamTag = CFStringCreateWithCString(NULL, str, kCFStringEncodingUTF8);
else
mParamTag = NULL;
}
The custom Cocoa view you created in Interface Builder can only be
viewed in an application that can host Audio Units with a custom Cocoa
view, such as GarageBand. There is a testing application in the
CoreAudio SDK that supports Cocoa views located at
/Developer/Examples/CoreAudio/Services/CocoaAUHost.
Versioning
Undoubtedly you will end up creating, testing, and possibly shipping
multiple versions of your Audio Unit. Using the templates, you only need
to increment version number in your Audio Unit's Version.h
file (here, MyVolumeUnitVersion.h). Since Xcode will invoke
Rez to compile the resources, and each of the
.r files in the project use the values defined in
MyVolumeUnitVersion.h, this is a great convenience.
Listing 4 defines the version number, manufacturer code and subtype.
Notice the "suggestion" to change the subtype and manufacturer values in
order to avoid namespace collisions. Also note that if you place
multiple versions of the same manufacturer and subtype in the same
folder, the Component Manager will only access the highest version.
Listing 4: File MyVolumeUnitVersion.h
#ifdef DEBUG
#define kMyVolumeUnitVersion 0xFFFFFFFF
#else
#define kMyVolumeUnitVersion 0x00010000
#endif
//~~~~~~~~~~~~~~ Change!!! ~~~~~~~~~~~~~~~~~~~~~//
#define MyVolumeUnit_COMP_SUBTYPE 'Pass'
#define MyVolumeUnit_COMP_MANF 'Demo'
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~//
Validating with the AU Validation Tool
The next step is to run the validation tool,
AUValidation. This command tests the Audio Unit for
conformance with expected interfaces. (It cannot determine that the
Audio Unit works as advertised; that task is your responsibility.)
First, check that the Component Manager has discovered
MyVolumeUnit. The command auval invokes
AUValidation. The -a flag lists all available
Audio Units of any type.
Listing 5: Output from the AU Validation Tool
Mertz:~ asd$ auval -a
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
AU Validation Tool
Version: 1.1.1b11
Copyright 2003-4, Apple Computer, Inc.
Specify -h (-help) for command options
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
aufx Pass Demo - Acme Inc: MyVolumeUnit
aufx bpas appl - Apple: AUBandpass
aufx dcmp appl - Apple: AUDynamicsProcessor
aufx dely appl - Apple: AUDelay
aufx greq appl - Apple: AUGraphicEQ
aufx hpas appl - Apple: AUHipass
aufx hshf appl - Apple: AUHighShelfFilter
aufx lmtr appl - Apple: AUPeakLimiter
aufx lpas appl - Apple: AULowpass
aufx lshf appl - Apple: AULowShelfFilter
aufx mcmp appl - Apple: AUMultibandCompressor
aufx mrev appl - Apple: AUMatrixReverb
aufx pmeq appl - Apple: AUParametricEQ
aumu dls appl - Apple: DLSMusicDevice
aumx 3dmx appl - Apple: AUMixer3D
aumx mxmx appl - Apple: AUMatrixMixer
aumx smxr appl - Apple: AUMixer
aufc conv appl - Apple: AUConverter
aufc vari appl - Apple: AUVarispeed
auou ahal appl - Apple: AudioDeviceOutput
auou def appl - Apple: DefaultOutputUnit
auou genr appl - Apple: GenericOutput
auou sys appl - Apple: SystemOutputUnit
Mertz:~ asd$
It's there, so test MyVolumeUnit using:
Mertz:~ asd$ auval -v aufx Pass Demo
The -v flag indicates that the next arguments specify
the type and manufacturer information. aufx is the
component type, and indicates an Audio Unit effect. Pass is
the subtype, as defined in the resource file
MyVolumeUnit.r, and Demo is the manufacturer
name, also defined in the resource file.
Listing 5 shows partial output from the above command. Each step or
phase results in either a PASS or FAIL. The file
AUValidationReadMe.rtf (included with the AUValidation
download) discusses the phases, possible outcomes, and options for
AUValidation.
Listing 5: Output from the AU Validation Tool
Mertz:~ asd$ auval -v aufx Pass Demo
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
AU Validation Tool
Version: 1.1.1b11
Copyright 2003-4, Apple Computer, Inc.
Specify -h (-help) for command options
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
--------------------------------------------------
VALIDATING AUDIO UNIT: 'aufx' - 'Pass' - 'Demo'
--------------------------------------------------
Manufacturer String: Acme Inc
AudioUnit name: MyVolumeUnit
Component Info: Acme Inc's Revolutionary Volume Unit
Component Version: 1.0.0 (0x10000)
* * PASS
--------------------------------------------------
TESTING OPEN TIMES:
COLD:
Time to open AudioUnit: 28.872ms
WARM:
Time to open AudioUnit: 0.091ms
* * PASS
--------------------------------------------------
VERIFYING DEFAULT SCOPE FORMATS:
Input Scope Bus Configuration:
Default Bus Count:1
Default Format: AudioStreamBasicDescription: 2 ch, 44100 Hz, 'lpcm' (0x0000002B) 32-bit big-endian
float, deinterleaved
Output Scope Bus Configuration:
Default Bus Count:1
Default Format: AudioStreamBasicDescription: 2 ch, 44100 Hz, 'lpcm' (0x0000002B) 32-bit big-endian
float, deinterleaved
* * PASS
--------------------------------------------------
VERIFYING REQUIRED PROPERTIES:
VERIFYING PROPERTY: Sample Rate
PASS
VERIFYING PROPERTY: Stream Format
PASS
VERIFYING PROPERTY: Maximum Frames Per Slice
PASS
VERIFYING PROPERTY: Last Render Error
PASS
* * PASS
--------------------------------------------------
VERIFYING RECOMMENDED PROPERTIES:
VERIFYING PROPERTY: Latency
PASS
VERIFYING PROPERTY: Tail Time
WARNING: Recommended Property is not supported
VERIFYING PROPERTY: Bypass Effect
PASS
* * PASS
--------------------------------------------------
VERIFYING OPTIONAL PROPERTIES:
VERIFYING PROPERTY Host Callbacks
PASS
* * PASS
--------------------------------------------------
VERIFYING SPECIAL PROPERTIES:
VERIFYING CUSTOM UI
Carbon View Components Available: 0
Cocoa Views Available: 1
CocoaFactoryView
PASS
VERIFYING CLASS INFO
PASS
TESTING HOST CALLBACKS
PASS
* * PASS
// output truncated...
--------------------------------------------------
AU VALIDATION SUCCEEDED.
--------------------------------------------------
Mertz:~ asd$
Now that MyVolumeUnit has been built and validated, you can use it in
GarageBand, Logic, and other audio applications.
Integrating with GarageBand
Once the Component Manager has located and registered your Audio Unit
(which occurs during startup and login), GarageBand will automatically
display it as an effect that can be applied to either the Master Track
or any of the Instruments. Use the menu command Track > Show Track
Info or <Command-I> to display the Track Info window. Figure 5
shows MyVolumeUnit selected as an effect for the instrument
named "Gain-Adjusted Grand Piano". Clicking the modify icon in the
far-right column displays the Volume Unit user interface. Notice that it
lacks the adornments added by AudioUnitHosting, and instead
displays the custom Cocoa View created in Interface Builder. You may
want to verify that your interface looks correct in any apps you expect
it to work with.
Figure 5: MyVolumeUnit in GarageBand
For More Information
Using the techniques discussed in this article, you can create your own Audio Units for audio signal processing. The references below contain additional information on this and other related topics.
Tools
- You will need the current version of the Core Audio SDK (note that this article was written with the CoreAudio SDK version 1.3.3—later versions may not exactly match the contents of this article).
- Download and use the Audio Units Validation Tool.
- Download the Audio Unit Effect Templates
which are available separately from the SDK at the time of this
writing.
- You will find additional useful information in the
AudioToolbox.framework, AudioUnit.framework, and CoreAudio.framework header files. Definitely look through them.
- Be sure to register your creator code(s).
Documents
Marketing
- Display the Audio Units logo with your completed
AU or hosting application. You must first complete
and return the Audio Units Logo Agreement, which is
linked off the Software Licensing & Trademark
Agreements page.
Updated: 2006-01-17
|
