Using GNU Autoconf, Automake, and Autoheader

If you are bringing a preexisting command-line utility to OS X that uses GNU autoconf, automake, or autoheader, you will probably find that it configures itself without modification (though the resulting configuration may be insufficient). Just run configure and make as you would on any other UNIX-based system.

If running the configure script fails because it doesn’t understand the architecture, try replacing the project’s config.sub and config.guess files with those available in /usr/share/automake-1.6. If you are distributing applications that use autoconf, you should include an up-to-date version of config.sub and config.guess so that OS X users don’t have to do anything extra to build your project.

If that still fails, you may need to run /usr/bin/autoconf on your project to rebuild the configure script before it works. OS X includes autoconf in the BSD tools package. Beyond these basics, if the project does not build, you may need to modify your makefile using some of the tips provided in the following sections. After you do that, more extensive refactoring may be required.

Some programs may use autoconf macros that are not supported by the version of autoconf that shipped with OS X. Because autoconf changes periodically, you may actually need to get a new version of autoconf if you need to build the very latest sources for some projects. In general, most projects include a prebuilt configure script with releases, so this is usually not necessary unless you are building an open source project using sources obtained from CVS or from a daily source snapshot.

However, if you find it necessary to upgrade autoconf, you can get a current version from http://www.gnu.org/software/autoconf/. Note that autoconf, by default, installs in /usr/local/, so you may need to modify your PATH environment variable to use the newly updated version. Do not attempt to replace the version installed in /usr/.

For additional information about using the GNU autotoolset, see http://autotoolset.sourceforge.net/tutorial.html and the manual pages autoconf, automake, and autoheader.

Compiling for Multiple CPU Architectures

Because the Macintosh platform includes more than one processor family, it is often important to compile software for multiple processor architectures. For example, libraries should generally be compiled as universal binaries even if you are exclusively targeting an Intel-based Macintosh computer, as your library may be used by a PowerPC binary running under Rosetta. For executables, if you plan to distribute compiled versions, you should generally create universal binaries for convenience.

When compiling programs for architectures other than your default host architecture, such as compiling for a ppc64 or Intel-based Macintosh target on a PowerPC-based build host, there are a few common problems that you may run into. Most of these problems result from one of the following mistakes:

• Assuming that the build host is architecturally similar to the target architecture and will thus be capable of executing intermediate build products

• Trying to determine target-processor-specific information at configuration time (by compiling and executing small code snippets) rather than at compile time (using macro tests) or execution time (for example, by using conditional byte swap functions)

Whenever cross-compiling occurs, extra care must be taken to ensure that the target architecture is detected correctly. This is particularly an issue when generating a binary containing object code for more than one architecture.

In many cases, binaries containing object code for more than one architecture can be generated simply by running the normal configuration script, then overriding the architecture flags at compile time.

For example, you might run

./configure

followed by

make CFLAGS="-isysroot /Developer/SDKs/MacOSX10.4u.sdk -arch ppc \

    -arch i386" LDFLAGS="-syslibroot /Developer/SDKs/MacOSX10.4u.sdk \

    -arch ppc -arch i386"

to generate a universal binary (for Intel-based and PowerPC-based Macintosh computers). To generate a 4-way universal binary that includes 64-bit versions, you would add -arch ppc64 and -arch x86_64 to the CFLAGS and LDFLAGS.

However, applications that make configuration-time decisions about the size of data structures will generally fail to build correctly in such an environment (since those sizes may need to be different depending on whether the compiler is executing a ppc pass, a ppc64 pass, or an i386 pass). When this happens, the tool must be configured and compiled for each architecture as separate executables, then glued together manually using lipo.

In rare cases, software not written with cross-compilation in mind will make configure-time decisions by executing code on the build host. In these cases, you will have to manually alter either the configuration scripts or the resulting headers to be appropriate for the actual target architecture (rather than the build architecture). In some cases, this can be solved by telling the configure script that you are cross-compiling using the --host, --build, and --target flags. However, this may simply result in defaults for the target platform being inserted, which doesn’t really solve the problem.

The best fix is to replace configure-time detection of endianness, data type sizes, and so on with compile-time or run-time detection. For example, instead of testing the architecture for endianness to obtain consistent byte order in a file, you should do one of the following:

• Use C preprocessor macros like __BIG_ENDIAN__ and __LITTLE_ENDIAN__ to test endianness at compile time.

• Use functions like htonl, htons, ntohl, and ntohs to guarantee a big-endian representation on any architecture.

• Extract individual bytes by bitwise masking and shifting (for example, lowbyte=word & 0xff; nextbyte = (word >> 8) & 0xff; and so on).

Similarly, instead of performing elaborate tests to determine whether to use int or long for a 4-byte piece of data, you should simply use a standard sized type such as uint32_t.

There are a few other caveats when working with universal binaries:

• The library archive utility, ar, cannot work with libraries containing code for more than one architecture (or single-architecture libraries generated with lipo) after ranlib has added a table of contents to them. Thus, if you need to add additional object files to a library, you must keep a separate copy without a TOC.

• The -M switch to gcc (to output dependency information) is not supported when multiple architectures are specified on the command line. Depending on your makefile, this may require substantial changes to your makefile rules. For autoconf-based configure scripts, the flag --disable-dependency-tracking should solve this problem.

  For projects using automake, it may be necessary to run automake with the -i flag to disable dependency checks or put no-dependencies in the AUTOMAKE_OPTIONS variable in each Makefile.am file.

• If you run into problems building a universal binary for an open source tool, the first thing you should do is to get the latest version of the source code. This does two things:

  • Ensures that the version of autoconf and automake used to generate the configuration scripts is reasonably current, reducing the likelihood of build failures, execution failures, backwards or forwards compatibility problems, and other idiosyncratic or downright broken behavior.

  • Reduces the likelihood of building a version of an open source tool that contains known security holes or other serious bugs.

• Older versions of autoconf do not handle the case where --target, --host, and --build are not handled gracefully. Different versions also behave differently when you specify only one or two of these flags. Thus, you should always specify all three of these options if you are running an autoconf-generated configure script with intent to cross-compile.

• Some earlier versions of autoconf handle cross-compiling poorly. If your tool contains a configure script generated by an early autoconf, you may be able to significantly improve things by replacing some of the config.* files (and config.guess in particular) with updated copies from the version of autoconf that comes with OS X.

  This will not always work, however, in which case it may be necessary to actually regenerate the configure script by running autoconf. To do this, simply change into the root directory of the project and run /usr/bin/autoconf. It will automatically detect and use the configure.in file and use it to generate a new configure script. If you get warnings, you should first try a web search for the error message, as someone else may have already run into the problem (possibly on a different tool) and found a solution.

  If you get errors about missing AC_ macros, you may need to download a copy of libraries on which your tool depends and copy their .m4 autoconf configuration files into /usr/share/autoconf. Alternately, you can add the macros to the file acinclude.m4 in your project’s main directory and autoconf should automatically pick up those macros.

  You may, in some cases, need to rerun automake and/or autoheader if your tool uses them. Be prepared to run into missing AM_ and AH_ macros if you do, however. Because of the added risk of missing macros, this should generally only be done if running autoconf by itself does not correct a build problem.

• Different makefiles and configure scripts handle command-line overrides in different ways. The most consistent way to force these overrides is to specify them prior to the command. For example:

  make CFLAGS="-isysroot /Developer/SDKs/MacOSX10.4u.sdk -arch i386 -arch ppc"

  should generally result in the above being added to CFLAGS during compilation. However, this behavior is not completely consistent across makefiles from different projects.

For additional information about autoconf, automake, and autoheader, you can view the autoconf documentation at http://www.gnu.org/software/autoconf/manual/index.html.

For additional information on compiler flags for Intel-based Macintosh computers, modifying code to support little-endian CPUs, and other porting concerns, you should read Universal Binary Programming Guidelines, Second Edition, available from the ADC Reference Library.

Conditional Compilation on OS X

You will sometimes find it necessary to use conditional compilation to make your code behave differently depending on whether certain functionality is available.

Older code sometimes used conditional statements like #ifdef __MACH__ or #ifdef __APPLE__ to try to determine whether it was being compiled on OS X or not. While this seems appealing as a quick way of getting ported, it ultimately causes more work in the long run. For example, if you make the assumption that a particular function does not exist in OS X and conditionally replace it with your own version that implements the same functionality as a wrapper around a different API, your application may no longer compile or may be less efficient if Apple adds that function in a later version.

Apart from displaying or using the name of the OS for some reason (which you can more portably obtain from the uname API), code should never behave differently on OS X merely because it is running on OS X. Code should behave differently because OS X behaves differently in some way—offering an additional feature, not offering functionality specific to another operating system, and so on. Thus, for maximum portability and maintainability, you should focus on that difference and make the conditional compilation dependent upon detecting the difference rather than dependent upon the OS itself. This not only makes it easier to maintain your code as OS X evolves, but also makes it easier to port your code to other platforms that may support different but overlapping feature sets.

The most common reasons you might want to use such conditional statements are attempts to detect differences in:

• processor architecture

• byte order

• file system case sensitivity

• other file system properties

• compiler, linker, or toolchain differences

• availability of application frameworks

• availability of header files

• support for a function or feature

Instead it is better to figure out why your code needs to behave differently in OS X, then use conditional compilation techniques that are appropriate for the actual root cause.

The misuse of these conditionals often causes problems. For example, if you assume that certain frameworks are present if those macros are defined, you might get compile failures when building a 64-bit executable. If you instead test for the availability of the framework, you might be able to fall back on an alternative mechanism such as X11, or you might skip building the graphical portions of the application entirely.

For example, OS X provides preprocessor macros to determine the CPU architecture and byte order. These include:

• __i386__—Intel (32-bit)

• __x86_64__—Intel (64-bit)

• __ppc__—PowerPC (32-bit)

• __ppc64__—PowerPC (64-bit)

• __BIG_ENDIAN__—Big endian CPU

• __LITTLE_ENDIAN__—Little endian CPU

• __LP64__—The LP64 (64-bit) data model

In addition, using tools like autoconf, you can create arbitrary conditional compilation on nearly any practical feature of the installation, from testing to see if a file exists to seeing if you can successfully compile a piece of code.

For example, if a portion of your project requires a particular application framework, you can compile a small test program whose main function calls a function in that framework. If the test program compiles and links successfully, the application framework is present for the specified CPU architecture.

You can even use this technique to determine whether to include workarounds for known bugs in Apple or third-party libraries and frameworks, either by testing the versions of those frameworks or by providing a test case that reproduces the bug and checking the results.

For example, in OS X, poll does not support device files such as /dev/tty. If you just avoid poll if your code is running on OS X, you are making two assumptions that you should not make:

• You are assuming that what you are doing will always be unsupported. OS X is an evolving operating system that adds new features on a regular basis, so this is not necessarily a valid assumption.

• You are assuming that OS X is the only platform that does not support using poll on device files. While this is probably true for most device files, not all device files support poll in all operating systems, so this is also not necessarily a valid assumption.

A better solution is to use a configuration-time test that tries to use poll on a device file, and if the call fails, disables the use of poll. If using poll provides some significant advantage, it may be better to perform a runtime test early in your application execution, then use poll only if that test succeeds. By testing for support at runtime, your application can use the poll API if is supported by a particular version of any operating system, falling back gracefully if it is not supported.

A good rule is to always test for the most specific thing that is guaranteed to meet your requirements. If you need a framework, test for the framework. If you need a library, test for the library. If you need a particular compiler version, test the compiler version. And so on. By doing this, you increase your chances that your application will continue to work correctly without modification in the future.

Choosing a Compiler

OS X ships two compilers and their corresponding toolchains. The default compiler is based on GCC 4.2. In addition, a compiler based on GCC 4.0 is provided. Older versions of Xcode also provide prior versions. Compiling for 64-bit PowerPC and Intel-based Macintosh computers is only supported in version 4.0 and later. Compiling 64-bit kernel extensions is only supported in version 4.2 and later.

Always try to compile your software using GCC 4 because future toolchains will be based on GCC version 4 or later. However, because GCC 4 is a relatively new toolchain, you may find bugs that prevent compiling certain programs.

Use of the legacy GCC 2.95.2-based toolchain is strongly discouraged unless you have to maintain compatibility with OS X version 10.1.

If you run into a problem that looks like a compiler bug, try using a different version of GCC. You can run a different version by setting the CC environment variable in your Makefile. For example, CC=gcc-4.0 chooses GCC 4.0. In Xcode, you can change the compiler setting on a per-project basis or a per-file basis by selecting a different compiler version in the appropriate build settings inspector.

Setting Compiler Flags

When building your projects in OS X, simply supplying or modifying the compiler flags of a few key options is all you need to do to port most programs. These are usually specified by either the CFLAGS or LDFLAGS variable in your makefile, depending on which part of the compiler chain interprets the flags. Unless otherwise specified, you should add these flags to CFLAGS if needed.

Some common flags include:

-flat_namespace (in LDFLAGS)

Changes from a two-level to a single-level (flat) namespace. By default, OS X builds libraries and applications with a two-level namespace where references to dynamic libraries are resolved to a definition in a specific dynamic library when the image is built. Use of this flag is generally discouraged, but in some cases, is unavoidable. For more information, see Understanding Two-Level Namespaces.

-bundle (in LDFLAGS)

Produces a Mach-O bundle format file, which is used for creating loadable plug-ins. See the ld man page for more discussion of this flag.

-bundle_loader executable (in LDFLAGS)

Specifies which executable will load a plug-in. Undefined symbols in that bundle are checked against the specified executable as if it were another dynamic library, thus ensuring that the bundle will actually be loadable without missing symbols.

-framework framework (in LDFLAGS)

Links the executable being built against the listed framework. For example, you might add -framework vecLib to include support for vector math.

-mmacosx-version-min version

Specifies the version of OS X you are targeting. You must target your compile for the oldest version of OS X on which you want to run the executable. In addition, you should install and use the cross-development SDK for that version of OS X. For more information, see SDK Compatibility Guide.

Note: OS X also uses this value to determine the UNIX conformance behavior of some APIs. For more information, read Unix 03 Conformance Release Notes.

More extensive discussion for the compiler in general can be found at http://developer.apple.com/releasenotes/DeveloperTools/.

Understanding Two-Level Namespaces

By default, OS X builds libraries and applications with a two-level namespace. In a two-level namespace environment, when you compile a new dynamic library, any references that the library might make to other dynamic libraries are resolved to a definition in those specific dynamic libraries.

The two-level namespace design has many advantages for Carbon applications. However, it can cause problems for many traditional UNIX applications if they were designed to work in a flat namespace environment.

For example, suppose one library, call it libfoo, uses another library, libbar, for its implementation of the function barIt. Now suppose an application wants to override the use of libbar with a compressed version, called libzbar. Since libfoo was linked against libbar at compile time, this is not possible without recompiling libfoo.

To allow the application to override references made by libfoo to libbar, you would use the flag -flat_namespace. The ld man page has a more detailed discussion of this flag.

If you are writing libraries from scratch, it may be worth considering the two-level namespace issue in your design. If you expect that someone may want to override your library’s use of another library, you might have an initializer routine that takes pointers to the second library as its arguments, and then use those pointers for the calls instead of calling the second library directly.

Alternately, you might use a plug-in architecture in which the calls to the outside library are made from a plug-in that could be easily replaced with a different plug-in for a different outside library. See Dynamic Libraries and Plug-ins for more information.

For the most part, however, unless you are designing a library from scratch, it is not practical to avoid using -flat_namespace if you need to override a library’s references to another library.

If you are compiling an executable (as opposed to a library), you can also use -force_flat_namespace to tell dyld to use a flat namespace when loading any dynamic libraries and bundles loaded by the binary. This is usually not necessary, however.

Executable Format

The only executable format that the OS X kernel understands is the Mach-O format. Some bridging tools are provided for classic Macintosh executable formats, but Mach-O is the native format. It is very different from the commonly used Executable and Linking Format (ELF). For more information on Mach-O, see OS X ABI Mach-O File Format Reference.

Dynamic Libraries and Plug-ins

Dynamic libraries and plug-ins behave differently in OS X than in other operating systems. This section explains some of those differences.

ld a.o b.o c.o ... -L/path/to/lib -lmytool

gcc -bundle a.o b.o c.o -o mybundle.bundle \

    -bundle_loader /Applications/MyApp.app/Contents/MacOS/MyApp

Bundles

In the OS X file system, some directories store executable code and the software resources related to that code in one discrete package. These packages, known as bundles, come in two varieties: application bundles and frameworks.

There are two basic types of bundles that you should be familiar with during the basic porting process: application bundles and frameworks. In particular, you should be aware of how to use frameworks, since you may need to link against the contents of a framework when porting your application.

Handling Multiply Defined Symbols

A multiply defined symbol error occurs if there are multiple definitions for any symbol in the object files that you are linking together. You can specify the following flags to modify the handling of multiply defined symbols under certain circumstances:

-multiply_defined treatment

Specifies how multiply defined symbols in dynamic libraries should be treated when -twolevel_namespace is in effect. The values for treatment must be one of:

• error—Treat multiply defined symbols as an error.

• warning—Treat multiply defined symbols as a warning.

• suppress—Ignore multiply defined symbols.

The default behavior is to treat multiply defined symbols in dynamic libraries as warnings when -twolevel_namespace is in effect.

-multiply_defined_unused treatment

Specifies how unused multiply defined symbols in dynamic libraries should be treated when -twolevel_namespace is in effect. An unused multiply defined symbol is a symbol defined in the output that is also defined in one of the dynamic libraries, but in which but the symbol in the dynamic library is not used by any reference in the output. The values for treatment must be error, warning, or suppress. The default for unused multiply defined symbols is to suppress these messages.

Predefined Macros

The following macros are predefined in OS X:

__OBJC__

This macro is defined when your code is being compiled by the Objective-C compiler. By default, this occurs when compiling a .m file or any header included by a .m file. You can force the Objective-C compiler to be used for a .c or .h file by passing the -ObjC or -ObjC++ flags.

__cplusplus

This macro is defined when your code is being compiled by the C++ compiler (either explicitly or by passing the -ObjC++ flag).

__ASSEMBLER__

This macro is defined when compiling .s files.

__NATURAL_ALIGNMENT__

This macro is defined on systems that use natural alignment. When using natural alignment, an int is aligned on sizeof(int) boundary, a short int is aligned on sizeof(short) boundary, and so on. It is defined by default when you're compiling code for PowerPC architecutres. It is not defined when you use the -malign-mac68k compiler switch, nor is it defined on Intel architectures.

__MACH__

This macro is defined if Mach system calls are supported.

__APPLE__

This macro is defined in any Apple computer.

__APPLE_CC__

This macro is set to an integer that represents the version number of the compiler. This lets you distinguish, for example, between compilers based on the same version of GCC, but with different bug fixes or features. Larger values denote later compilers.

__BIG_ENDIAN__ and __LITTLE_ENDIAN__

These macros tell whether the current architecture uses little endian or big endian byte ordering. For more information, see Compiling for Multiple CPU Architectures.

Other Porting Tips

This section describes alternatives to certain commonly used APIs.