A common way for a hacker to gain control of a system is to exploit a buffer overflow in a running program. A buffer overflow occurs when a program does not validate its input and accepts more data than can fit in the memory that the program reserved for that data. The data then overwrites memory owned by the system or by some other program. In some circumstances, the hacker can insert executable code directly into memory this way; in other cases, the hacker can cause a jump of execution to another location in memory. For more information how such a buffer overflow is exploited, see Types of Security Vulnerabilities in Secure Coding Guide.
In order to make such exploits more difficult, starting in Mac OS X v10.4, a change was made to prevent the execution of code in the region of memory known as the stack on Intel-based Macintosh computers by default. In Mac OS X v10.5, the restrictions on executing code were extended in two ways: First, the system now disallows the execution of stack-based code on both PowerPC-based and Intel-based Macintosh computers by default. Second, for 64-bit programs, the system now disallows attempts to execute code in any portion of memory unless that portion is explicitly marked as executable. Most developers may not notice these changes because code compiled and linked statically is automatically marked as executable by the linker. A 64-bit application that generates code dynamically, however, must explicitly mark that code as executable or the program receives a SIGBUS signal and exits when trying to execute that code. A program can use the mprotect system call with the PROT_EXEC option to grant execute permissions to a block of memory containing dynamically generated code. For information on how to use this call, see the mprotect manual page.
Last updated: 2008-02-08