It should come as no surprise that Mac OS 9 and Mac OS X have different threading characteristics. While Mac OS X does support the Mac OS 9 threading APIs (as part of Carbon), it does so using an entirely different core OS; this core OS brings with it a number of subtle changes in behavior of those threading APIs.
This technote explains the basics of the threading architecture on both platforms. By understanding the fundamental design you can understand the subtle differences in how each platform handles threads. The ultimate goal is to make it easier for you to tune your threaded application to match the behavior of each platform. This should yield better performance for both your application and other applications on the system.
This technote describes the threading architecture for Mac OS 9.1 and Mac OS X 10.0.x, specifically. Things are different on older versions of traditional Mac OS and may be different on future versions of Mac OS X. Mac OS X is a rapidly evolving system. You should use this document as a guide to thinking about threading on Mac OS X, not as the final word about threading on that platform.
Mac OS 9 Threading
This section describes threading on Mac OS 9. Mac OS 9 has two threading APIs.
In addition to these threading APIs, it's impossible to understand Mac OS 9 threading without also understanding Mac OS 9 process scheduling, as implemented by the Process Manager.
Links to programming documentation for each of these APIs are given in the References section at the end of this technote.
Mac OS 9 Without MP Tasks
If you temporarily ignore MP tasks, the Mac OS 9 threading architecture is shown in Figure 1.
In this architecture, all scheduling is done
cooperatively. Each process is round-robin scheduled by the
Process Manager when the process calls a yielding function
You can draw a number of conclusions from this diagram:
One important feature of Mac OS 9 threading is that Process Manager processes are very expensive. They consume a lot of resources (processes have fixed-size memory partitions) and, for historical reasons, the context switch time between processes is very expensive.
Mac OS 9 With MP Tasks
If you consider MP tasks, Mac OS 9's threading architecture is only slightly more complex, as shown in Figure 2.
The entire cooperative environment runs within a single MP task. This task is known as the blue task. All Process Manager processes and all Thread Manager threads are executed by the blue task. Other MP tasks, created either by the system or by an application, are executed as separate entities. The nanokernel schedules tasks to run on the processor (or processors) in a preemptive fashion.
You should note the following consequences of this architecture:
Mac OS X Threading
This section describes threading on Mac OS X, which provides five different threading APIs:
The Carbon Specification describes some of the issues you might encounter when porting MP task or Thread Manager code to Carbon on Mac OS X.
One useful way to think about Mac OS X's various threading APIs is to arrange them in an layered hierarchy. For example, each MP task is layered on top of a pthread, and each pthread is layered on top of a Mach thread. This hierarchy is shown in Figure 3.
Links to programming documentation for each of these APIs are given in the References section at the end of this technote.
Thread API Choice
With five public threading APIs to choose from, it can be hard to decide which threading API to use. Here are some general guidelines:
In contrast to Mac OS 9, all threads have roughly equal cost on Mac OS X. The overhead of wrapping a pthread up as an NSThread, MP task, or cooperative thread is small compared to the overhead of the base Mach thread. This also means that cooperative threads are more expensive (relative to other types of threads) than they are on Mac OS 9. For more details on the cost of threads in Mac OS X, see Inside Mac OS X: Performance.
Thread context switch time can vary depending on the threading API you choose. The overhead for an involuntary context switch and to block for I/O in the kernel is the same regardless of how you created the thread. The overhead for other context switches (for example, waiting on a semaphore) depends on the API you choose, with higher-level APIs generally being slower.
Mac OS X Thread Scheduling
One consequence of the thread hierarchy described earlier is that ultimately all threads are represented by Mach threads. As far as Mach is concerned, all threads are equal. When choosing a thread to run, the Mach scheduler always follows its scheduling policies to choose the next thread to run. It does not care whether the Mach thread was created as a pthread, an MP task, a cooperative thread, and so on. Thus you can represent the Mach scheduling as a single flat ring, as shown in Figure 4.
To understand how cooperative threads are implemented, you need to turn this diagram on its side and look at how abstractions are layered on top of the fundamental Mach threads. Figure 5 shows this. At the bottom layer of each abstraction stack is the Mach thread that is actually scheduled by the kernel. Some Mach threads exist in their raw form (typically these threads are created to execute within the kernel). All user space Mach threads have a pthread layered on top of them. Threads created by Cocoa applications have an NSThread layered on top of that. Carbon can created threads both for itself and to implement higher-level threading APIs. If the thread is created for internal use by Carbon, it is implemented a very simple wrapper around a pthread. If the thread is created by Carbon at the behest of a Carbon application, it is also a pthread but with either a Thread Manager thread or an MP task layered on top of it.
Carbon ensures that all Thread Manager threads created
within a single process are scheduled cooperatively. Each
process has a special synchronization token (the token is
implemented as a Mach message) that is passed between the
cooperative threads within the process. If the cooperative
thread has the token it is allowed to run. If it does not
have the token, it must block waiting for it. When the
cooperative thread that has the token calls to
This token passing arrangement is represented by the red lines in Figure 5.
A Mach thread's policy controls the algorithm used to schedule the thread. The recommended thread policies on Mac OS X are:
For each policy there are a number of policy parameters that control how the thread acts within that policy. These parameters are roughly equivalent to the common idea of thread priority. For example, threads using the precedence policy have an "importance" parameter that controls the thread's priority with respect to other threads within that task. However, this simplistic equivalence breaks down for more the more complex time constraint policy, where the policy parameters are actually a number of real-time values.
Each threading API has its own idea of thread priorities.
For example, MP tasks have the concept of task weight (as
To learn more about Mac OS's recommended thread policies, see <mach/thread_policy.h>. Mac OS X also supports some older, deprecated thread policies, which are defined in <mach/policy.h>.
Mac OS X Thread Miscellanea
This section covers a number of miscellaneous topics related to threading on Mac OS X.
The main thread is a valid thread for all threading APIs.
For example, you can call
Traditional Mac OS makes heavy use of asynchronous I/O with I/O completion routines that are delivered at interrupt time. However, the Mac OS X core OS does not support a callback-based asynchronous I/O model, and user space code never runs at interrupt time. Carbon simulates asynchronous I/O completion routines using preemptive threads.
For example, when you make an asynchronous File Manager request, Carbon simply puts the request on an internal queue and wakes up its asynchronous file I/O thread. That thread pulls the first item off the queue, executes it synchronously, and then calls its completion routine. If there are no more items on the queue the thread goes to sleep. The upshot of this is that the operations appear to operate asynchronously even though the underlying core OS does not have an asynchronous file system API.
Carbon uses techniques like this for many different interrupt time callback-based APIs, including File Manager, Time Manager, Deferred Task Manager, and Open Transport. Each subsystem creates its own pthread for making callbacks.
This design has a number of interesting consequences.
With all of these threading APIs available, it's inevitable that at some point you will want to mix and match threads and their APIs. For example, you might want to call a Mach thread API on an MP task, or maybe an MP call on a pthread. If one thread is layered on top of another thread (assuming the layering hierarchy from Figure 3), you can, in general, do low-level operations on the high-level thread. For example, most pthread calls are safe to make on MP tasks. There are, however, a number of issues to consider.
The first issue is how to get a reference to the
low-level thread. Only the pthread API provides an explicit
mechanism to get a reference to the underlying thread
Another question is whether low-level operations actually work on a high-level thread. In general, most pthread APIs are safe on pthreads created by the higher-level thread APIs. On the other hand, you should be very careful when making Mach thread calls on pthreads (and hence on threads created by any high-level API). In general, it is not appropriate to make Mach thread calls on pthreads. Some exceptions are the Mach APIs for getting thread information, setting up thread exception handlers, and death notification, which should work for any Mach thread regardless of how it was created.
Finally, given that most high-level threads are
implemented in terms of pthreads, is it safe to use
high-level thread APIs on an arbitrary pthread? For example,
is it safe to call MP API routines on a pthread? This
approach definitely breaks the thread layering hierarchy and
is not recommended. One notable, and useful, exception is
that it is safe to make MP API synchronization calls
Mac OS X Kernel Threading
On traditional BSD-based systems the kernel is divided into upper and lower halves. The upper half is executed by the user thread that made a particular kernel call. It can block (sleep) while waiting for I/O to complete or for access to shared kernel resources. The lower half is executed as the result of hardware interrupts. Interrupts must not block. The upper half synchronizes with the lower half by disabling interrupts. McKusick, et al., provide a more in depth explanation of this design.
While Mac OS X uses a large part of the BSD kernel source code, it is not a traditional BSD system. Its Mach underpinnings require a rework of the BSD kernel synchronization mechanism. For example, Mach supports multiprocessor systems, so disabling interrupts is no longer a sufficient synchronization guarantee.
The upper half of the Mac OS X BSD kernel still runs in
the context of the user thread that made the kernel call.
However, the lower half is no longer executed in a hardware
interrupt context. Hardware interrupts have a tiny scope
within Mac OS X. When the hardware interrupt occurs it
simply wakes up an IOKit workloop thread. These workloop
threads are kernel threads; they are owned
by the kernel, not by any BSD process. The workloop thread
is the entity that executes the lower half of the BSD
kernel. This means that the BSD kernel synchronization
The solution to this problem is the kernel funnel. The kernel funnel is a mutex that prevents more than one thread from running inside the BSD portions of the kernel at the same time. Each thread acquires the kernel funnel when it enters the BSD portion of the kernel, and releases it when it leaves. In addition, if a thread blocks (sleeps) while holding a funnel, it automatically drops the funnel, and thus allows other threads to enter the kernel.
Mac OS X 10.0.x implements split funnels. There is one funnel for the networking part of the kernel, and one funnel for the other BSD parts of the kernel (file system, process management, device management, and so on). This split results in a significant performance improvement for tasks that use both the disk and the network. Future versions of Mac OS X may use even more funnels, allowing even greater kernel re-entrancy, and even greater performance.
As far as the Mach scheduler is concerned, threads running within the BSD kernel are like any other Mach thread. One particularly interesting consequence of this is kernel preemption. When a high priority thread wakes up it will preempt a thread running within the BSD kernel. This happens regardless of whether the high priority thread is a kernel thread or a user thread. As long as the high priority thread does not attempt to acquire a kernel funnel (that is, it makes no BSD system calls), it can do its job despite the limited re-entrancy of the BSD parts of the kernel. This design allows Mac OS X to meet the real-time goals required by its real-time components, such as the highly responsive audio playback engine.
With five different threading APIs, Mac OS X threading looks complicated, but it's actually simpler than Mac OS 9 threading. All you have to do is understand the fundamental concepts:
DTS Q&A PS 06 Yielding Time Without Getting Events -- This describes why it is impossible to yield to other processes without handling user interface events on traditional Mac OS.
DTS Q&A 1061 RunApplicationEventLoop
and Thread Manager -- This discusses the problems with
integrating cooperative threads into
DTS Technote 1104 Interrupt-Safe Routines
DTS Technote 2006 MP-Safe Routines
Carbon Specification -- Describes some of the issues you might encounter when porting MP task or Thread Manager code to Carbon on Mac OS X.
Mach 3 Kernel Principles and Interfaces -- While these documents are somewhat out of date, they're still a valuable introduction to the Mach 3 kernel and its programming APIs.
Darwin -- The source code for Mac OS X's kernel (CVS module "xnu") and pthreads library (CVS module "Libc") is available as part of Apple's open source efforts.
The Open Group's Single UNIX Specification -- This includes reference documentation for the pthreads API.
David R Butenhof , Programming with POSIX Threads, Addison-Wesley, 1997, ISBN: 0201633922 -- A good introduction to pthreads programming for UNIX systems.
NSThread Class -- This is the reference documentation for the NSThread class.
Inside Macintosh: Processes -- This book describes the Process Manager and its APIs.
Macintosh: Macintosh Toolbox Essentials -- This book
includes a chapter on the Event Manager, which describes the
Inside Macintosh: Thread Manager -- This book describes the Mac OS 9 Thread Manager.
Inside Carbon: Thread Manager -- This describes the Carbon Thread Manager.
Adding Multitasking Capability to Applications Using Multiprocessing Services -- This describes the Mac OS 9 MP task API.
Adding Multitasking Capability to Applications Using Multiprocessing Services -- This describes the Carbon MP task API.
Inside Mac OS X: Performance -- This book contains information about the memory cost of threads on Mac OS X.
Marshall Kirk McKusick, et al., The Design and Implementation of the 4.4BSD Operating System, Addison-Wesley, 1996, ISBN: 0201549794