Inside Macintosh: Memory /

Chapter 2 - Memory Manager

Using the Memory Manager

This section discusses the techniques you can use both to deal with memory outside of your application's partition and to manipulate your own application's partition.

You can use the techniques in this section to

• read and write the values of system global variables when there is no Toolbox routine that would accomplish the work for you
• check for the availability of temporary memory and use it to speed operations that depend on memory buffers
• allocate memory in the system heap
• install code into the system heap
• allocate memory at the high end of the available RAM from within a system extension during the startup process
• initialize new heap zones within your application heap zone, on your application's stack, or in the application global variables area
• install a purge-warning procedure for your application heap zone

Reading and Writing System Global Variables

In general, you should avoid relying on the values of system global variables whenever possible. However, you might occasionally need to access the value of one of these variables. Because the actual values associated with global variables in MPW's SysEqu.p interface file are memory locations, you can access the value of a low-memory variable simply by dereferencing a memory location.

Many system global variables are process-independent, but some are process-specific. The Operating System swaps the values of the process-specific variables as it switches processes. If you write interrupt code that reads low memory, that code could execute at a time when another process's system global variables are installed. Therefore, before reading low memory from interrupt code, you should call the Process Manager to ensure that your process is the current process. If it is not, you should not rely on the value of system global variables that could conceivably be process-specific.

Note

No available documentation distinguishes process-specific from process-independent system global variables.

The routine defined in Listing 2-1 illustrates how you can read a system global variable, in this case the system global variable BufPtr, which gives the address of the highest byte of allocatable memory.

Listing 2-1 Reading the value of a system global variable

FUNCTION FindHighestByte: LongInt;
TYPE
   LongPtr = ^LongInt;
BEGIN
   FindHighestByte := LongPtr(BufPtr)^;
END;

In Pascal, the main technique for reading system global variables is to define a new data type that points to the variable type you want to read. In this example, the address is stored as a long integer. Thus, the memory location BufPtr is really a pointer to a long integer. Because of Pascal's strict typing rules, you must cast the low-memory address into a pointer to a long integer. Then, you can dereference the pointer and return the long integer itself as the function result.

You can use a similar technique to change the value of a system global variable. For example, suppose you are writing an extension that displays a window at startup time. To maintain compatibility with pre-Macintosh II systems, you need to clear the system global variable named DeskHook. This global variable holds a ProcPtr that references a procedure called by system software to paint the desktop. If the value of the pointer is NIL, the system software uses the standard desktop pattern. If you do not set DeskHook to NIL, the system software might attempt to use whatever random data it contains to call an updating procedure when you move or close your window. The procedure defined in Listing 2-2 sets DeskHook to NIL.

Listing 2-2 Changing the value of a system global variable

PROCEDURE ClearDeskHook;
TYPE
   ProcPtrPtr = ^ProcPtr;                 {pointer to ProcPtr}
VAR
   deskHookProc: ProcPtrPtr;              
BEGIN
   deskHookProc := ProcPtrPtr(DeskHook);  {initialize variable}
   deskHookProc^ := NIL;                  {clear DeskHook proc}
END;

You can use a similar technique to change the value of any other documented system global variable.

Extending an Application's Memory

Rather than using your application's 'SIZE' resource to specify a preferred partition size that is large enough to contain the largest possible application heap, you should specify a smaller but adequate partition size. When you need more memory for temporary use, you can use a set of Memory Manager routines for the allocation
of temporary memory.

By using the routines for allocating temporary memory, your application can request some additional memory for occasional short-term needs. For example, the Finder uses these temporary-memory routines to secure buffer space for use during file copy operations. Any available memory (that is, memory currently unallocated to any application's partition) is dedicated to this purpose. The Finder releases this memory as soon as the copy is completed, thus making the memory available to other applications or to the Operating System for launching new applications.

Because the requested amount of memory might not be available, you cannot be sure that every request for temporary memory will be honored. Thus, you should make sure that your application will work even if your request for temporary memory is denied. For example, if the Finder cannot allocate a large temporary copy buffer, it uses a reserved small copy buffer from within its own heap zone, prolonging the copying but performing it nonetheless.

Temporary memory is taken from RAM that is reserved for (but not yet used by) other applications. Thus, if you use too much temporary memory or hold temporary memory for long periods of time, you might prevent the user from being able to launch other applications. In certain circumstances, however, you can hold temporary memory indefinitely. For example, if the temporary memory is used for open files and the user can free that memory simply by closing those files, it is safe to hold onto that memory as long as necessary.

Temporary memory is tracked (or monitored) for each application, and so you must use it only for code that is running on an application's behalf. Moreover, the Operating System frees all temporary memory allocated to an application when the application quits or crashes. As a result, you should not use temporary memory for VBL tasks, Time Manager tasks, or other procedures that should continue to be executed after your application quits. Similarly, it is wise not to use temporary memory for an interprocess buffer (that is, a buffer whose address is passed to another application in a high-level event) because the originating application could crash, quit, or be terminated, thereby causing the temporary memory to be released before (or even while) the receiving application uses that memory.

Although you can usually perform ordinary Memory Manager operations on temporary memory, there are two restrictions. First, you must never lock temporary memory across calls to GetNextEvent or WaitNextEvent. Second, although you can determine the zone from which temporary memory is generated (using the HandleZone function), you should not use this information to make new blocks or perform heap operations on your own.

Allocating Temporary Memory

You can request a block of memory for temporary use by calling the Memory Manager's TempNewHandle function. This function attempts to allocate a new relocatable block of the specified size for temporary use. For example, to request a block that is one-quarter megabyte in size, you might issue this command:

myHandle := TempNewHandle($40000, myErr); {request temp memory}

If the routine succeeds, it returns a handle to the block of memory. The block of memory returned by a successful call to TempNewHandle is initially unlocked. If an error occurs and TempNewHandle fails, it returns a NIL handle. You should always check for NIL handles before using any temporary memory. If you detect a NIL handle, the second parameter (in this example, myErr) contains the result code from the function.

Instead of asking for a specific amount of memory and then checking the returned handle to find out whether it was allocated, you might prefer to determine beforehand how much temporary memory is available. There are two functions that return information on the amount of free memory available for temporary allocation. The first is the TempFreeMem function, which you can use as follows:

memFree := TempFreeMem;    {find amount of free temporary memory}

The result is a long integer containing the amount, in bytes, of free memory available for temporary allocation. It usually isn't possible to allocate a block of this size because of fragmentation. Consequently, you'll probably want to use the second function, TempMaxMem, to determine the size of the largest contiguous block of space available. To allocate that block, you can write

mySize := TempMaxMem(grow);
myHandle := TempNewHandle(mySize, myErr);

The TempMaxMem function returns the size, in bytes, of the largest contiguous free block available for temporary allocation. (The TempMaxMem function is analogous to the MaxMem function.) The grow parameter is a variable parameter of type Size; after the function returns, it always contains 0, because the temporary memory does not come from the application's heap. Even when you use TempMaxMem to determine the size of the available memory, you should check that the handle returned by TempNewHandle
is not NIL.

Determining the Features of Temporary Memory

Only computers running system software version 7.0 and later can use temporary memory as described in this chapter. For this reason, you should always check that the routines are available and that they have the features you require before calling them.

Note

The temporary-memory routines are available in some earlier system software versions when MultiFinder is running. However, the handles to blocks of temporary memory are neither tracked nor real.

The Gestalt function includes a selector to determine whether the temporary-memory routines are present in the operating environment and, if they are, whether
the temporary-memory handles are tracked and whether they are real. If temporary-memory handles are not tracked, you must release temporary memory before your next call to GetNextEvent or WaitNextEvent. If temporary-memory handles are not real, then you cannot use normal Memory Manager routines such as HLock to manipulate them.

To determine whether the temporary-memory routines are implemented, you can check the value returned by the TempMemCallsAvailable function, defined in Listing 2-3.

Listing 2-3 Determining whether temporary-memory routines are available

FUNCTION TempMemCallsAvailable: Boolean;
VAR
   myErr:   OSErr;               {Gestalt result code}
   myRsp:   LongInt;             {response returned by Gestalt}
BEGIN
   TempMemCallsAvailable := FALSE;
   myErr := Gestalt(gestaltOSAttr, myRsp);
   IF myErr <> noErr THEN  
      DoError(myErr)          {Gestalt failed}
   ELSE                       {check bit for temp mem support}
      TempMemCallsAvailable :=
            BAND(myRsp, gestaltTempMemSupport) <> 0;
END;

You can use similar code to determine whether temporary-memory handles are real and whether the temporary memory is tracked.

Using the System Heap

The system heap is used to store most of the information needed by the Operating System and other system software components. As a result, it is ideal for storing information needed by a system extension (which by definition extends the capabilities of system software). You might also need to use the system heap to store a task record and the code for an interrupt task that should continue to be executed when your application is not the current application.

Allocating blocks in the system heap is straightforward. Most ordinary Memory Manager routines have counterparts that allocate memory in the system heap zone instead of the current heap zone. For example, the counterpart of the NewPtr function is the NewPtrSys function. The following line of code allocates a new nonrelocatable block of memory in the system heap to store a Time Manager task record:

myTaskPtr := QElemPtr(NewPtrSys(SizeOf(TMTask)));

Alternatively, you can change the current zone and use ordinary Memory Manager operations, as follows:

SetZone(SystemZone);
myTaskPtr := QElemPtr(NewPtr(SizeOf(TMTask)));
...
SetZone(ApplicationZone);

You might also need to store the interrupt code itself in the system heap. For example, when an application that installed a vertical retrace task with the VInstall function is in the background, the Vertical Retrace Manager executes the task only if the vblAddr field of the task record points to a routine in the system heap.

Unfortunately, manually copying a routine into the system heap is difficult in Pascal.
The easiest way to install code into the system heap is to place the code into a separate stand-alone code resource in your application's resource fork. You should set the system heap bit and the locked bit of the code resource's attributes. Then, when you need to use the code, you must load the resource from the resource file and cast the resource handle's master pointer into a procedure pointer (a variable of type ProcPtr), as follows:

myProcHandle := GetResource(kProcType, kProcID);
IF myProcHandle <> NIL THEN
   myTaskPtr^.vblAddr := ProcPtr(myProcHandle^);

Because the resource is locked in memory, you don't have to worry about creating
a dangling pointer when you dereference a handle to the resource. If you want the
code to remain in the system heap after the user quits your application, you can call
the Resource Manager procedure DetachResource so that closing your application's resource fork does not destroy the resource data. Note, however, that if you do so and your application crashes, the code still remains in the system heap.

Once you have loaded a code resource into memory and created a ProcPtr that references the entry point of the code resource, you can use that ProcPtr just as you can use any such variable. For example, you could assign the value of the variable to the vblAddr field of a vertical retrace task record (as shown just above). If you are programming in assembly language, you can then call the code directly. To call the routine from a high-level language such as Pascal, you'll need to use some inline assembly-language code. Listing 2-4 defines a routine that you can use to execute a procedure by address.

Listing 2-4 Calling a procedure by address

PROCEDURE CallByAddress (aRoutine: ProcPtr);
   INLINE   $205F,            {MOVE.L (SP)+,A0}
            $4ED0;            {JMP (A0)}

Allocating Memory at Startup Time

If you are implementing a system extension, you might need to allocate memory at startup time. As explained in the previous section, an ideal place to allocate such memory is in the system heap. To allocate memory in the system heap under system software version 7.0 and later, you merely need to call the appropriate Memory Manager routines, and the system heap expands dynamically to meet your request. In earlier versions of system software, you must use a 'sysz' resource to indicate how much the Operating System should increase the size of the system zone.

Alternatively, however, you can allocate blocks in high memory. The global variable BufPtr always references the highest byte in memory that might become part of an application partition. You can lower the value of BufPtr and then use the memory between the old and new values of BufPtr.

Note

In general, if you are implementing a system extension, you should allocate memory in the system heap instead of high memory. In this way, you avoid the problems associated with lowering the value of BufPtr too far (described in the following paragraphs) and ensure that the extension is not paged out if virtual memory is operating.

Lowering the value of BufPtr too far can be dangerous for several reasons. In 128K ROM Macintosh computers running system software version 4.1, you must avoid lowering the value of BufPtr so that it points in the system startup blocks. The highest byte of these blocks can always be found relative to the global variable MemTop, at MemTop DIV 2 + 1024.

In later versions of the Macintosh system software, the system startup blocks were no longer barriers to BufPtr, but new barriers arose, including Macintosh IIci video storage, for example. To maintain compatibility with extensions that rely on the ability to lower BufPtr relative to MemTop, the system software simply adjusts MemTop so that the formula still holds. Thus, at startup, the MemTop global variable currently does not reference any memory location in particular. Instead, it holds a value that guarantees that the formula allowing you to lower BufPtr as low as MemTop DIV 2 + 1024 but no further still holds.

Beginning in system software version 7.0, the Operating System can detect excessive lowering of BufPtr, but only after the fact. When the Operating System does detect
that the value of BufPtr has fallen too low, it generates an out-of-memory system error.

WARNING

Although the above formula has been true since system software version 4.1, a bug in the Macintosh IIci and later ROMs made it invalid in certain versions of system software 6.x.

Because there is no calling interface for lowering BufPtr, you must do it manually, by changing the value of the system variable, as explained in "Reading and Writing System Global Variables" on page 2-8. To obtain the value of the MemTop global variable, you can use the TopMem function.

Creating Heap Zones

You can create heap zones as subzones of your application heap zone or (in rare instances) either in space reserved for the application global variables or on the stack. You can also create heap zones in a block of temporary memory or within the system heap zone. This section describes how to create new heap zones by calling the InitZone procedure.

Note

Most applications do not need to create heap zones.

To create a new heap zone in the application heap, you must allocate nonrelocatable blocks in your application heap to hold new subzones of the application heap. In addition to being able to create subzones of the application zone, you can create subzones of any other zone to which you have access, including a zone that is itself
a subzone of another zone.

You create a heap zone by calling the InitZone procedure, which takes four parameters. The first parameter specifies a grow-zone function for the new zone, or NIL if you do not want the zone to have a grow-zone function. The second parameter specifies the number of new master pointers that you want each block of master pointers in the zone to contain. The InitZone procedure allocates one such block to start with, and you can allocate more by calling the MoreMasters procedure. The third and fourth parameters specify, respectively, the first byte beyond the end of the new zone and the first byte of the zone.

When initializing a zone with the InitZone procedure, make sure that you are subdividing the current zone. When InitZone returns, the new zone becomes
current. Thus, if you subdivide the application zone into several subzones, you must
call SetZone(ApplicationZone) before you create the second and each of the subsequent subzones. Listing 2-5 shows a technique for creating a single subzone of the original application zone, assuming that the application zone is the current zone. The technique for subdividing subzones is similar.

Listing 2-5 Creating a subzone of the original application heap zone

FUNCTION CreateSubZone: THz;
CONST
   kZoneSize = 10240;            {10K zone}
   kNumMasterPointers = 16;      {num of master ptrs for new zone}
VAR
   start:   Ptr;                 {first byte in zone}
   limit:   Ptr;                 {first byte beyond zone}
BEGIN
   start := NewPtr(kZoneSize);   {allocate storage for zone}
   IF MemError <> noErr THEN
   BEGIN                         {allocation successful}
      limit := Ptr(ORD4(start) + kZoneSize);
                                 {compute byte beyond end of zone}
      InitZone(NIL, kNumMasterPointers, limit, start);
                                 {initialize zone header, trailer}
   END;
   CreateSubZone := THz(start);  {cast storage to a zone pointer}
END;

To create a subzone in the system heap zone, you can call SetZone(SystemZone) at the beginning of the procedure in Listing 2-5. You might find this technique useful if you are implementing a system extension but want to manage your extension's memory much as you manage memory in an application. Instead of simply allocating blocks in the system heap, you can make your zone current whenever your extension is executed. Then, you can call regular Memory Manager routines to allocate memory in your subzone of the system heap, and you can compact and purge your subzone without compacting and purging the entire system heap zone.

When you allocate memory for a subzone, you must allocate that memory in a nonrelocatable block (as in Listing 2-5) or in a locked relocatable block. If you create a subzone within an unlocked relocatable block, the Memory Manager might move your entire subzone during memory operations in the zone containing your subzone. If so, any references to nonrelocatable blocks that you allocated in the subzone would become invalid. Even handles to relocatable blocks in the subzone would no longer be valid, because the Memory Manager does not update the handles' master pointers correctly. This happens because the Memory Manager views a subzone of another zone as a
single block. If that subzone is a relocatable block, the Memory Manager updates only that block's master pointer when moving it, and does not update the block's contents (that is, the blocks allocated within the subzone).

If you use a block of temporary memory as a heap zone, you must lock the temporary memory immediately after allocating it. Then, you can pass to InitZone a dereferenced copy of a handle to the temporary memory. If you find (after a call to the Gestalt function) that temporary memory handles are not real, then you must dispose of the new zone before any calls to GetNextEvent or WaitNextEvent. You must dispose of the new zone because you cannot lock a handle to temporary memory across event calls if the handle is not real.

Once you have created a subzone as a nonrelocatable block or a locked relocatable block, you can allocate both relocatable and nonrelocatable blocks within it. Although the Memory Manager can move such relocatable blocks only within the subzone, it correctly updates those blocks' master pointers, which are also in the subzone.

Installing a Purge-Warning Procedure

You can define a purge-warning procedure that the Memory Manager calls whenever it is about to purge a block from your application heap. You can use this procedure to save the data in the block, if necessary, or to perform other processing in response to this notification.

Note

Most applications don't need to install a purge-warning procedure. This capability is provided primarily for applications that require greater control over their heap. Examples are applications that maintain purgeable handles containing important data and applications that for any other reason need notification when a block is about to be purged.

When your purge-warning procedure is called, the Memory Manager passes it a handle to the block about to be purged. In your procedure, you can test the handle to determine whether it contains data that needs to be saved; if so, you can save the data (possibly by writing it to some open file). Listing 2-6 defines a very simple purge-warning procedure.

Listing 2-6 A purge-warning procedure

PROCEDURE MyPurgeProc (h: Handle);
VAR
   theA5:   LongInt;             {value of A5 when procedure is called}
BEGIN
   theA5 := SetCurrentA5;        {remember current value of A5; install ours}
   IF BAND(HGetState(h), $20) = 0 THEN
      BEGIN                      {if the handle isn't a resource handle}
         IF InSaveList(h) THEN
            WriteData(h);        {save the data in the block}
      END;
   theA5 := SetA5(theA5);        {restore previous value of A5}
END;

The MyPurgeProc procedure defined in Listing 2-6 inspects the handle's properties (using HGetState) to see whether its resource bit is clear. If so, the procedure next determines whether the handle is contained in an application-maintained list of
handles whose data should be saved before purging. If the handle is in that list, the purge-warning procedure writes its data to disk. (The file into which the data is written should already be open at the time the procedure is called, because opening a file might cause memory to move.)

Note that MyPurgeProc sets up the A5 register with the application's A5 value upon entry and restores it to its previous value before exiting. This is necessary because you cannot rely on the A5 register within a purge-warning procedure.

WARNING

Because of the optimizations performed by some compilers, the actual work of the purge-warning procedure and the setting and restoring of the A5 register might have to be placed in separate procedures. See the chapter "Vertical Retrace Manager" in Inside Macintosh: Processes for an illustration of how you can do this.

To install a purge-warning procedure, you need to install the address of the
procedure into the purgeProc field of your application's heap zone header.
Listing 2-7 illustrates one way to do this.

Listing 2-7 Installing a purge-warning procedure

PROCEDURE InstallPurgeProc;
VAR
   myZone:  THz;
BEGIN
   myZone := GetZone;                     {find the current zone header}
   gPrevProc := myZone^.purgeProc;        {remember previous procedure}
   myZone^.purgeProc := @MyPurgeProc;     {install new procedure}
END;

The InstallPurgeProc procedure defined in Listing 2-7 first obtains the address of the current heap zone by calling the GetZone function. Then it saves the address of any existing purge-warning procedure in the global variable gPrevProc. Finally, InstallPurgeProc installs the new procedure by putting its address directly into the purgeProc field of the zone header. (For more information on zone headers, see "Heap Zones" on page 2-19.)

Keep in mind that the Memory Manager calls your purge-warning procedure each time it decides to purge any purgeable block, and it might call your procedure far more often than you would expect. Your purge-warning procedure might be passed handles not only to blocks that you explicitly mark as purgeable (by calling HPurge), but also to resources whose purgeable attribute is set. (In general, applications don't need to take any action on handles that belong to the Resource Manager.) Because of the potentially large number of times your purge-warning procedure might be called, it should be able to determine quickly whether a handle that is about to be purged needs additional processing.

Remember that a purge-warning procedure is called during the execution of some Memory Manager routine. As a result, your procedure cannot cause memory to be moved or purged. In addition, it should not dispose of the handle it is passed or change the purge status of the handle. See "Purge-Warning Procedures" on page 2-90 for a complete description of the limitations on purge-warning procedures.

WARNING

If your application calls the Resource Manager procedure SetResPurge with the parameter TRUE (to have the Resource Manager automatically save any modified resources that are about to be purged), you should avoid using a purge-warning procedure. This is because the Resource Manager installs its own purge-warning procedure when you call SetResPurge in this way. If you must install your own purge-warning procedure, you should remove your procedure, call SetResPurge, then reinstall your procedure as shown in Listing 2-7. You then need to make sure that your procedure calls the Resource Manager's purge-warning procedure (which is saved in the global variable gPrevProc) before exiting. Most applications do not need to call SetResPurge at all.

If your application does call SetResPurge(TRUE), you should use the version of MyPurgeProc defined in Listing 2-8. It is just like the version defined in Listing 2-6 except that it calls the Resource Manager's purge-warning procedure before exiting.

Listing 2-8 A purge-warning procedure that calls the Resource Manager's procedure

PROCEDURE MyPurgeProc (h: Handle);
VAR
   theA5:   LongInt;             {value of A5 when procedure is called}
BEGIN
   theA5 := SetCurrentA5;        {remember current value of A5; install ours}
   IF BAND(HGetState(h), $20) = 0 THEN
      BEGIN                      {if the handle isn't a resource handle}
         IF InSaveList(h) THEN
            WriteData(h);        {save the data in the block}
      END
   ELSE IF gPrevProc <> NIL THEN
      CallByAddress(gPrevProc);
   theA5 := SetA5(theA5);        {restore previous value of A5}
END;

See Listing 2-4 on page 2-13 for a definition of the procedure CallByAddress.