Why Byte Ordering Matters

The example in this section is designed to show you why byte ordering matters. Take a look at the C data structure defined in Listing 3-1. It contains a four-byte integer, a character string, and a two-byte integer. The listing also initializes the structure.

Listing 3-1  A data structure that contains multibyte and single-byte data

typedef struct {

    uint32_t myOptions;

    char     myStringArray [7];

    short    myVariable;

} myDataStructure;

myDataStructure aStruct;

aStruct.myOptions = 0xfeedface;

strcpy(aStruct.myStringArray, "safari");

aStruct.myVariable = 0x1234;

Figure 3-1 compares how this data structure is stored in memory on big-endian and little-endian systems. In a big-endian system, memory is organized with the address of each data byte increasing from most significant to least significant. In a little-endian system, memory is organized with the address of each data byte increasing from the least significant to the most significant.

As you look at Figure 3-1, note the following:

• Multibyte data, such as the 32-bit and 16-bit variables shown in the figure, are stored differently between big-endian and little-endian systems. As you can see in the figure, big-endian systems store data in memory so that the most significant byte of the data is stored in the address with the lowest value. Little-endian systems store data in memory so that the most significant byte of the data is in the address with the highest value. Hence, the least significant byte of the myOptions variable (0xce) is stored in memory location 0x00000003 on the big-endian system while it is stored in memory location 0x00000000 on the little-endian system.

• Single-byte data, such as the char values in the myStringArray character array, are stored in the same memory location on either system regardless of the byte ordering format of the system.

• Each system pads bytes to maintain four-byte data alignment. Padded bytes in the figure are designated by a shaded box that contains an asterisk.

The byte ordering of multibyte data in memory matters if you are reading data written on one architecture from a system that uses a different architecture and you access the data on a byte-by-byte basis. For example, if your application is written to access the second byte of the myOptions variable, then when you read the data from a system that uses the opposite byte ordering scheme, you’ll end up retrieving the first byte of the myOptions variable instead of the second one.

Suppose the example data values that are initialized by the code shown in Listing 3-1 are generated on a little-endian system and saved to disk. Assume that the data is written to disk in byte-address order. When read from disk by a big-endian system, the data is again laid out in memory as shown in Figure 3-1. The problem is that the data is still in little-endian byte order even though it is interpreted on a big-endian system. This difference causes the values to be evaluated incorrectly. In this example, the value of the field myOptions should be 0xfeedface, but because of the incorrect byte ordering it is evaluated as 0xcefaedfe.

Guidelines for Swapping Bytes

The following guidelines, along with the strategies provided later in this chapter, will help ensure optimal byte-swapping code in your application.

• Keep data structures in native byte-order while in memory. Only swap bytes when you read data from disk or write it to disk.

• When possible, let the compiler do the work for you. For example, when you use function calls such as the Core Foundation function CFSwapInt16BigToHost, the compiler determines whether the function call does something for the processor you are targeting. If the code does nothing, the compiler won’t call the function. Letting the compiler do the work is more efficient than using #ifdef statements.

• If you must access a large file, consider arranging the data in a way that limits the byte swapping that you must perform. For example, you can arrange the most frequently accessed data contiguously in the file. Then, you need to read and swap bytes only for that chunk of data instead of for the entire data file.

• Use the __BIG_ENDIAN__ and __LITTLE_ENDIAN__ macros only if you must. Do not use macros that check for a specific processor type, such as __i386__ and __ppc__.

• Choose a consistent byte-order approach and stick with it. That is, if you are reading and writing data from disk on a regular basis, choose the endian format you want to use. This eliminates the need for you to check the byte ordering of the data, and then to possibly have to swap the byte order.

• Be aware of which functions return big-endian data, and use them appropriately. These include the BSD Sockets networking functions, the DNSServiceDiscovery functions (for example, TCP and UDP ports are specified in network byte order), and the ColorSync profile functions (for which all data is big-endian). The IconFamilyElement and IconFamilyResource data types (which also include the data types IconFamilyPtr and IconFamilyHandle) are always big-endian. There may be other functions and data types that are not listed here. Consult the appropriate API reference for information on data returned by a function. For more information see Network-Related Data.

• Keep in mind that swapping bytes comes at a performance cost so swap them only when absolutely necessary.

Byte-Swapping Routines

The APIs that provide byte-swapping routines are listed below. For most situations it’s best to use the routines that match the framework you’re programming in. The Core Foundation and Foundation APIs have functions for swapping floating-point values, while the other APIs listed do not.

• POSIX (Portable Operating System Interface) byte ordering functions (ntohl, htonl, ntohs, and htons) are documented in Mac OS X Man Pages.

• Darwin byte ordering functions and macros are defined in the header file libkern/OSByteOrder.h. Even though this header is in the kernel framework, it is acceptable to use it from high-level applications.

• Core Foundation byte-order functions are defined in the header file CoreFoundation/CFByteOrder.h and described in the Byte-Order Utilities Reference. For details on using these functions, see the Byte Swapping article in Memory Management Programming Guide for Core Foundation.

• Foundation byte-order functions are defined in the header file Foundation/NSByteOrder.h and described in Foundation Framework Reference.

• The Core Endian API is defined in the header file CarbonCore/Endian.h and described in Core Endian Reference.

Byte-Swapping Strategies

The strategy for swapping bytes depends on the format of the data; there is no universal routine that can take care of all byte ordering differences. Any program that needs to swap data must know the data type, the source data endian order, and the host endian order.

This section lists byte-swapping strategies, organized alphabetically, for the following data:

• Constants

• Custom Apple Event Data

• Custom Resource Data

• Floating-Point Values

• Integers

• Network-Related Data

• OSType-to-String Conversions

• Unicode Text Files

• Values in an Array

double d = 3.0;

CFSwappedFloat64 swappedDouble;

// Encode the floating-point value.

swappedDouble = CFConvertFloat64HostToSwapped(d);

// Call the appropriate routine to write swappedDouble to disk,

// send it to another process, etc.

write(myFile, &swappedDouble, sizeof(swappedDouble));

float f;

CFSwappedFloat32 swappedFloat;

// Call the appropriate routine to read swappedFloat from disk,

// receive it from another process, etc.

read(myFile, &swappedFloat, sizeof(swappedFloat));

f = CFConvertFloat32SwappedToHost(swappedFloat)

SInt16  bigEndian16;

SInt16  swapped16;

// Swap a 16-bit value read from the network.

swapped16 = CFSwapInt16BigToHost(bigEndian16);

// Swap the bytes of the values if necessary.

aStruct.int1 = CFSwapInt32LittleToHost(aStruct.int1)

aStruct.int2 = CFSwapInt32LittleToHost(aStruct.int2)

CFDataRef addr = CFSocketCopyAddress(myCFSocketRef);

struct sockaddr_in my_sockaddr;

memmove(&my_sockaddr, CFDataGetBytePtr(addr), sizeof(my_sockaddr));

DNSServiceRegister( ... , my_sockaddr.sin_port, ...);

printf("%c%c%c%c", (char) (val >> 24), (char) (val  >> 16),

                    (char) (val >> 8), (char) val)

printf("%4.4s", (const char*) &val)

static inline void SwapUInt32ArrayBigToHost(UInt32 *array, UInt32  count) {

    int i;

    for(i = 0; i < count; i++) {

        array[i] = CFSwapInt32BigToHost(array[i]);

    }

}

Writing a Callback to Swap Data Bytes

You can provide a byte-swapping callback routine, also referred to as a flipper, to the system for custom resource data, custom pasteboard data, and custom Apple event data. When you install a byte-swapping callback, you specify which domain that the data type belongs to. There are two data domains—Apple event and resource. The resource data domain specifies custom pasteboard data or custom resource data. If the callback can be applied to either domain (Apple event and resource), you can specify that as well.

The Core Endian API defines a callback that you provide to swap bytes for custom resource and Apple event data. You must provide one callback for each type of data you want to swap bytes. The prototype for the CoreEndianFlipProc callback is:

typedef CALLBACK_API (OSStatus, CoreEndianFlipProc)

    (OSType dataDomain,

    OSType dataType,

    short id,

    void *dataPtr,

    UInt32 dataSize,

    Boolean currentlyNative,

    void *refcon

);

The callback takes the following parameters:

• dataDomain—An OSType value that specifies the domain to which the flipper callback applies. The value kCoreEndianResourceManagerDomain signifies that the domain is resource or pasteboard data. The value kCoreEndianAppleEventManagerDomain signifies that the domain is Apple event data.

• dataType—The type of data that needs the callback to swap bytes for. This is the four-character code of the resource type, pasteboard type, or Apple event.

• id—The resource id of the data type. This field is ignored if the dataDomain parameter is not kCoreEndianResourceManagerDomain.

• dataPtr—On input, points to the data to be flipped. On output, points to the byte swapped data.

• dataSize—The size of the data pointed to by the dataPtr parameter.

• currentlyNative—A Boolean value that indicates the direction to swap bytes. The value true specifies the data pointed to by the dataPtr parameter uses the byte ordering of the currently executing code. On a PowerPC Macintosh, true specifies that the data is in big-endian format. On an Intel-based Macintosh, true specifies that the data is in little-endian format.

• refcon—A 32-bit value that contains, or refers to, data needed by the callback.

The callback returns a result code that indicates whether bytes are swapped successfully. Your callback should return noErr if the data is byte swapped without error and the appropriate result code to indicate an error condition—errCoreEndianDataTooShortForFormat, errCoreEndianDataTooLongForFormat, or errCoreEndianDataDoesNotMatchFormat. The result code you return is propagated through the appropriate manager (Resource Manager (ResError) or Apple Event Manager) to the caller.

You do not need to swap bytes for quantities that are not numerical (such as strings, byte streams, and so forth). You need to provide a callback only to swap bytes data types for which the order of bytes in a word or long word are important. (For the preferred way to handle Unicode strings, see Unicode Text Files.)

Your callback should traverse the data structure that contains the data and swap bytes for:

• All counts and lengths so that array indexes are associated with the appropriate value

• All integers and longs so that when you read them into variables of a compatible type, you can operate correctly on the values (such as numerical, offset, and shift operations)

The Core Endian API provides these functions for working with your callback:

• CoreEndianInstallFlipper registers your callback for the specified data type (custom resource or custom Apple Event). After you register a byte-swapping callback for an application-defined resource data type, then any time you call a Resource Manager function that operates on that resource type, the system invokes your callback if it is appropriate to do so. (If your callback operates on pasteboard data, the system also invokes the callback at the appropriate time.) Similarly, if you specify Apple event as the domain for your callback, then any time you call an Apple Event Manager function that operates on that data type, your callback is invoked when it is appropriate to do so.

• CoreEndianGetFlipper obtains the callback that is registered for the specified data type. You can call this function to determine whether a flipper is available for a given data type.

• CoreEndianFlipData invokes the callback associated with the specified data type. You shouldn’t need to call this function, because the system invokes your callback whenever it’s needed.

As an example, look at a callback for the custom resource type ('PREF') defined in Listing 3-7. The MyPreferences structure is used to store preferences data on disk. The structure contains a number of values and includes two instances of the RGBColor data type and an array of RGBColor values.

Listing 3-7  A declaration for a custom resource

#define kMyPreferencesType      'PREF'

struct MyPreferences {

                SInt32          fPrefsVersion;

                Boolean         fHighlightLinks;

                Boolean         fUnderlineLinks;

                RGBColor        fHighlightColor;

                RGBColor        fUnderlineColor;

                SInt16          fZoomValue;

                char            fCString[32];

                SInt16          fCount;

                RGBColor        fPalette[];

};

You can handle the RGBColor data type by writing a function that swaps bytes in an RGBColor data structure, such as the function MyRGBSwap, shown in Listing 3-8. This function calls the Core Endian macro EndianS16_Swap to swap bytes for each of the values in the RGBColor data structure. The function doesn’t need to check for the currently executing system because the function is never called unless the values in the RGBColor data type need to have their bytes swapped. The MyRGBSwap function is called by the byte-swapping callback routine (shown in Listing 3-9) that’s provided to handle the custom 'PREF' resource (that is defined in Listing 3-7).

Listing 3-8  A flipper function for RGBColor data

static void MyRGBSwap (RGBColor *p)

{

    p->red = Endian16_Swap(p->red);

    p->blue = Endian16_Swap(p->blue);

    p->green = Endian16_Swap(p->green);

}

Listing 3-9 shows a byte-swapping callback for the custom 'PREF' resource. An explanation for each numbered line of code appears following the listing. Note that the flipper checks for data that is malformed or is of an unexpected length. If the data passed into the flipper routine is a shorter length than the flipped type is normally, or (for example) contains garbage data instead of an array count, the flipper must be careful not to read or write data beyond the end of the passed-in data. Instead, the routine returns an error.

Listing 3-9  A flipper for the custom 'PREF' resource

#define kCurrentVersion    0x00010400

static OSStatus MyFlipPreferences (OSType dataDomain,  // 1

                    OSType dataType,   // 2

                    short id,   // 3

                    void * dataPtr,   // 4

                    UInt32 dataSize,   // 5

                    Boolean currentlyNative,  // 6

                    void* refcon)  // 7

{

    UInt32  versionNumber;

    OSStatus status = noErr;

    MyPreferences* toFlip = (MyPreferences*) dataPtr;       // 8

    int count, i;

    if (dataSize < sizeof(MyPreferences))

        return errCoreEndianDataTooShortForFormat;  // 9

    if (currentlyNative)   // 10

    {

        count = toFlip->fCount;

        versionNumber = toFlip->fPrefsVersion;

        toFlip->fPrefsVersion = Endian32_Swap (toFlip->fPrefsVersion);

        toFlip->fCount = Endian16_Swap (toFlip->fCount);

        toFlip->fZoomValue = Endian16_Swap (toFlip->fZoomValue);

    }

    else   // 11

    {

        toFlip->fPrefsVersion = Endian32_Swap (toFlip->fPrefsVersion);

        versionNumber = toFlip->fPrefsVersion;

        toFlip->fCount = Endian16_Swap (toFlip->fCount);

        toFlip->fZoomValue = Endian16_Swap (toFlip->fZoomValue);

        count = toFlip->fCount;

    }

    if (versionNumber != kCurrentVersion)  // 12

                return errCoreEndianDataDoesNotMatchFormat;

    MyRGBSwap (&toFlip->fHighlightColor);  // 13

    MyRGBSwap (&toFlip->fUnderlineColor);  // 14

    if (dataSize < sizeof(MyPreferences) + count * sizeof(RGBColor))

        return errCoreEndianDataTooShortForFormat;   // 15

    for(i = 0; i < count; i++)

    {

        MyRGBSwap (&toFlip->fPalette[i]);  // 16

    }

    return status;     // 17

}

Here’s what the code does:

1. The system passes to your callback the domain to which the callback applies. You define the domain when you register the callback using the function CoreEndianInstallFlipper.

2. The system passes to your callback the resource type you defined for the data. In this example, the resource type is 'PREF'.

3. The system passes to your callback the resource ID of the data type. If the data is not a resource, this value is 0.

4. The system passes to your callback a pointer to the resource data that needs to have its bytes swapped. In this case, the pointer refers to a MyPreferences data structure.

5. The system passes to your callback the size of the data pointed to by the pointer described in the previous step.

6. The system passes to your callback true if the data in the buffer passed to the callback is in the byte ordering of the currently executing code. On a PowerPC Macintosh, when currentlyNative is true, the data is in big-endian order. On a Macintosh that uses an Intel microprocessor, when currentlyNative is true, the data is in little-endian order. Your callback needs to know this value, because if your callback uses a value in the data buffer to decide how to process other data in the buffer (for example, the count variable shown in the code), you must know whether that value needs to be flipped before the value can be used by the callback.

7. The system passes to your callback a pointer that refers to application-specific data. In this example, the callback doesn’t require any application-specific data.

8. Defines a variable for the MyPreferences data type and assigns the contents of the data pointer to the newly-defined toFlip variable.

9. Checks the static-length portion of the structure. If the size is less than it should be, the routine returns the error errCoreEndianDataTooLongForFormat.

10. If currentlyNative is true, saves the count value to a local variable and then swaps the bytes for the other values in the MyPreferences data structure. You must save the count value before you swap because you need it for an iteration later in the function. The fact that currentlyNative is true indicates that the value does not need to be byte swapped if it is used in the currently executing code. However, the value does need to be swapped to be stored to disk.

    The values are swapped using the appropriate Core Endian macros.

11. If currentlyNative is false, flips the values in the MyPreferences data structure before it saves the count value to a local variable. The fact that currentlyNative is false indicates that the count value needs to have its bytes swapped before it can be used in the callback.

12. Checks to make sure the version of the data structure is supported by the application. If the version is not supported, then your callback would not swap bytes for the data and would return the result errCoreEndianDataDoesNotMatchFormat.

13. Calls the MyRGBSwap function (shown in Listing 3-8) to swap the bytes of the fHighlightColor field of the data structure.

14. Calls the MyRGBSwap function to swap the bytes of the fUnderlineColor field of the data structure.

15. Checks the data size to make sure that it is less than it should be. If not, the routine returns the error errCoreEndianDataTooLongForFormat.

16. Iterates through the elements in the fPalette array, calling the MyRGBSwap function to swap the bytes of the data in the array.

17. Returns noErr to indicate that the data is flipped without error.

Although the sample performs some error checking, it does not include all the error-handling code that it could. When you write a flipper you may want to include such code.

You register a byte-swapping callback routine by calling the function CoreEndianInstallFlipper. You should register the callback when your application calls its initialization routine or when you open your resources. For example, you would register the flipper callback shown in Listing 3-9 using the following code:

OSStatus status = noErr;

status = CoreEndianInstallFlipper (kCoreEndianResourceManagerDomain,

                        kMyPreferencesType,

                        MyFlipPreferences,

                        NULL);

The system invokes the callback for the specified resource type and data domain when currentlyNative is false at the time a resource is loaded and true at the time the resource is set to be written. For example, the sample byte-swapping callback gets invoked any time the following line of code is executed in your application:

MyPreferences** hPrefs = (MyPreferences**) GetResource ('PREF',  128);

After swapping the bytes of the data, you can modify it as much as you’d like.

When the Resource Manager reads a resource from disk, it looks up the resource type (for example, 'PREF') in a table of byte-swapping routines. If a callback is installed for that resource type, the Resource Manager invokes the callback if it is appropriate to do so. Similar actions are taken when the Resource Manager writes a resource to disk. It finds the appropriate routine and invokes the callback to swap the bytes of the resource if it is appropriate to do so.

When you copy or drag custom data from an application that has a callback installed for pasteboard data, the system invokes your callback at the appropriate time. If you copy or drag custom data to a native application, the data callback is not invoked. If you copy or drag custom data to a nonnative application, the system invokes your callback to swap the bytes of the custom data. If you paste or drop custom data into your application from a nonnative application, and a callback exists for that custom data, the system invokes the callback at the time of the paste or drop. If the custom data is copied or dragged from another native application, the callback is not invoked.

Note that different pasteboard APIs use different type specifiers. The Scrap Manager and Drag Manager use OSType data types. The Pasteboard Manager uses Uniform Type Identifiers (UTI), and the NSPasteboard class uses its own type mechanism. In each case, the type is converted by the system to an OSType data type to discover if there is a byte-swapping callback for that type.

Apple event data types are typically swapped to network byte order when sent over a network. The callback you install is called only if a custom data type that you define is sent to another machine, or if another machine sends Apple event data to your application. The byte ordering of Apple events on the network is big-endian.

For cases in which the system would not normally invoke your byte-swapping callback, you can call the function CoreEndianFlipData to invoke the callback function installed for the specified data type and domain.

See Also

The following resources are available in the ADC Reference Library:

• Byte-Order Utilities Reference describes the Core Foundation byte order utilities API.

• Byte Swapping, in Core Foundation Memory Management, shows how to swap integers and floating-point values using Core Foundation byte-order utilities.

• File-System Performance Guidelines provides information useful for mapping Unicode files to memory.