CONTENTS

• Updating Firmware
• Installing System Software
  • Classic Mac OS
  • Mac OS X
• Installing MacsBug*
• Resetting PRAM
• Rebuilding the Desktop*

• Identifying and Reproducing a Bug
• Isolating a Bug
• How to Properly Report a Bug
  • Steps to Reproduce
  • Apple System Profiler*
  • MacsBug*
  • GDB+
  • Screen Shots*
  • Grab.app+
• Bug Reporting Template
• Submitting Apple Technology Bug Reports

Quality Assurance is the process of comparing what is required of a product and what is actually being provided to the users of that product. Apple encourages you to invest time planning a systematic approach to your testing. This will help ensure greater customer satisfaction with your products.

In this Note, you will find tips, tools and a template for producing a complete problem report for your programmers. Many times, a problem report is ignored until the required information is obtained and the issues are clearly understood. This wastes time. Starting with complete bug reports will save time in your development process.

This Note is directed at quality assurance engineers who are looking for tips and tools for compatibility testing with Apple technologies.

Note: Compatibility testing is a 3rd Party responsibility. This Note contains general testing highlights and is not meant to be complete testing guidelines.

Identifying a Target Test Environment (Hardware & Software)

Before you begin testing your software or hardware with the Mac OS, it is important to determine what your minimum requirements are.

• What is the minimum operating system requirement?
• What is the minimum hardware requirement?

You will certainly want to test a wide spectrum of Apple hardware, as well as with different versions of the Mac OS. Different versions of an Apple technology may or may not work as expected with your product. The only way to find out how compatible your product is with different versions of an Apple technology is to test with these different versions. Determining what your minimum requirements are will tell you how far back you need to test.

Keep in mind that if your product has dependencies on an Apple technology or a 3rd Party product, your product will share the same dependencies as those technologies.

To find the system requirements for an Apple technology, consider searching the Apple Support web pages. To find specifications for Apple hardware, consider searching the Apple Spec Database.

In example

Properly Preparing your Hardware for Testing

• Updating Firmware

  Your computer's firmware should always be updated to the latest version available for your computer. Search the Apple Support Software Downloads web pages for updates specific to your computer.

• Installing System Software
  
  
  • Classic Mac OS Install For the purposes of this Note, Classic Mac OS refers to any version of Mac OS that is less than Mac OS X.

    When installing a new version of the Classic Mac OS, it is strongly recommended that you perform a clean install. A clean install preserves your current System Folder and creates a new one. This is especially useful when trying to isolate and reproduce a problem. Sometimes a corrupt System Folder will mislead you into thinking there is a problem in your software or the Mac OS when there really is not.

    The best way to perform a clean install is to boot your computer from your system installation disks or CD. If you have floppy disks, insert the first system installation disk in your floppy disk drive, then restart your computer. If you have a CD-ROM, put it in your CD-ROM drive, restart your computer, and immediately hold down the 'C' key. The computer will use the floppy or CD as its startup disk and you should be able to begin your clean install.

    Note: In certain models, especially Macintosh clones, you may need to hold down the keyboard combination instead of the 'C' key to boot from your CD-ROM.

    • Clean install for Mac OS 7.6 to Mac OS 9.x
      1. Insert the first system installation disk or the system installation CD-ROM in the appropriate drive, then restart your computer(Note: Mac OS 8.5 and higher are only available on CD-ROM).
      2. Start up the installer, then click Options... in the main screen.
      3. Check the box that says 'Create new System Folder (clean installation...)' (Mac OS 7.6) or 'Perform clean installation' (Mac OS 8.0 and greater).
      
      
    • Clean install for Mac OS 7.5.x
      1. Insert the first system installation disk or the system installation CD-ROM in the appropriate drive, then restart your computer.
      2. Launch the system software installer, then press
      3. Select the clean system installation option.
      4. Install the software.
      
      
    • Clean install for Mac OS prior to 7.5
      1. Rename the System Folder to 'Previous System Folder'.
      2. Drop the System file into the Preferences folder.
      3. Insert the first system installation disk or the system installation CD-ROM in the appropriate drive, then restart your computer.
      4. Click the installer and install your system. Then run any updates to the system.
    
    

    After you do a clean install, you will need to reinstall any software to ensure that your applications have all the pieces they require in the System Folder. This includes any fonts, control panels, extensions, and other system software that is not a part of a standard Mac OS installation.

    Some people find it useful to duplicate a clean System Folder and test with a copy. This way you will always have a clean System Folder on your hard drive. You can make a copy of that System Folder instead of reinstalling it.

    Back to top

  • Mac OS X Install
    
    
    • Partitioning your Hard Disk

      Partitioning allows you to create multiple volumes on the same hard disk. If you are interested in doing compatibility testing with both Mac OS X and the Classic Mac OS on the same hard disk, you will want to create partitions for each.

      To partition your hard disk:
      1. Launch Apple's "Drive Setup" or any other disk formatting and partitioning utility.
      2. Divide your disk up into at least 2 HFS+ partitions.
         - For a thin install of Mac OS X, you will need about 600-700MB.
         - For a full install, you will need 1.2 GB.
      3. Name your partitions clearly.
         - Usually, the first partition is reserved for Mac OS 9.x.
         - The second is reserved for Mac OS X.
      
      
    • CD Install
      
      
      1. After putting the Mac OS X Install CD in your drive, restart with the "C" key down, so that you boot from the CD.
      2. Launch the installer and follow the on screen directions.
      3. Select the destination disk or partition for Mac OS X.
         - The destination disk or partition will be erased in the process.
         - If you wish to deselect packages, use the "Customize" button in the lower left corner of the installer window.
         - If you wish to add languages other than English, use the "Languages" button and check the desired languages.
      4. Click "Continue" and then click "Restart".
         - After the base system has been copied, your machine will restart again.
         - After the machine restarts, Installer.app will launch and begin copying your preselected packages.
         - Your machine will restart again, and the Setup Assistant will appear.

      Note: If you have installed Mac OS X and the Classic Mac OS on different partitions, you will need to hold down the option key after a restart to boot with the Classic Mac OS.

  Back to top

• Installing MacsBug (available for Classic Mac OS only)

  You should install MacsBug after performing a clean install of the Mac OS. MacsBug is Apple's assembly-level 680x0 and PowerPC debugger for Mac OS. It can be used to debug code running in most execution environments, from applications to drivers and everything in between. It is also a very valuable bug reporting tool used by many Mac OS software developers. Please see the documentation on Using MacsBug for more information.

  MacsBug can be downloaded from http://developer.apple.com/tools/debuggers/MacsBug/. The minimal installation requires that you take the file called MacsBug and move it into the System Folder. You need to restart the computer after installing MacsBug.

  Back to top

• Resetting PRAM

  Your computer stores certain preferences in an area of memory called Parameter RAM (PRAM pronounced "p-ram"). Resetting PRAM replaces the preferences stored there with the default (original) settings.

  Note: When you reset PRAM, preferences for Open Firmware, serial ports, video, and sound are lost. This affects the settings in some of your control panels, including the following:
  • Memory (RAM disk and other settings are lost)
  • Monitors
  • Sound
  • Keyboard
  • Mouse (or Trackpad)
  • AppleTalk
  • Power Manager
  • Map

  Some third-party software programs also store information in PRAM.

  To reset PRAM:
  1. Restart your computer and immediately hold down:
     
     Note: Make sure the Caps Lock key is not engaged.
  2. Hold the keys down until you hear the startup sound a second time, then release the keys.
  3. If you had special settings in any control panels, open those control panels to restore the settings you want.
  
  

  Back to top

• Rebuilding the Desktop (available for Classic Mac OS only)

  Rebuilding the Desktop file is useful when desktop information becomes corrupted (for example, when icons take on a "generic" appearance).

  To rebuild the desktop:
  1. Restart you computer and immediately hold down:
     
  2. Hold the keys down until a window comes up asking if you would like to rebuild the desktop.
  3. Click OK.

Bug Reports

• Identifying and Reproducing a Bug

  A bug is any behavior that you encounter in hardware or software that is not expected. Make sure that you can reproduce the bug before trying to communicate the bug to your engineer. You may want to consider restarting the machine and trying to reproduce the bug again. If you cannot properly document how to reproduce a bug, it will be difficult for your developer to isolate and fix the problem.

  Back to top

• Isolating a Bug

  After you have found steps to reproduce your bug, you want to try to find out how widespread your bug is. This should involve testing with different hardware and/or software. Consider all the dependencies your product has and then use those dependencies as ways to isolate your bug.

  Isolation Suggestions:
  • Try different versions of the Mac OS on the same computer.
  • Try different machines.
  • Try more or less RAM.
  • Try with virtual memory turned on or off.
  
  

  Back to top

• How to Properly Report a Bug

  A well written bug report will contain as much of the following information as possible.

  • Steps to Reproduce

    Always include a list of steps that are necessary to cause the bug to happen. They should be written in such a way that someone else can try to follow your steps to reproduce the bug on their own. Pretend that they have just restarted their machine. What steps will they need to take to reproduce your bug?

  • Apple System Profiler (available for Classic Mac OS only)

    The Apple System Profiler provides information about your computer's software and installed hardware devices. It includes information about the version of system software you're using, how many and what kind of devices are connected to your computer, and what applications are installed on your computer. Please see the documentation on Apple System Profiler for more information.

  • MacsBug (available for Classic Mac OS only)

    If you have found a bug that causes a crash, you will want to capture a MacsBug Standard Log(stdlog). The Standard Log that MacsBug generates contains debugging information that is useful to programmers and should be included in your bug reports. Please see the documentation on Using MacsBug for more information.

  • GDB (available for Mac OS X only)

    GDB allows you to see what your application is doing programmatically while it runs. GDB can be an invaluable tool to trace the execution of an application in order to find the cause of an encountered problem, and to let developers know where their code is failing. If GDB is running and your application crashes, you can capture a GDB stack backtrace(bt). A stack backtrace will allow you to see the last few instructions the program executed. Please see the documentation on Using GDB for more information.

  • Screen Shots (available for Classic Mac OS only)

    You can create a picture file of a window or of the entire screen. This is called taking a screen shot. Screen shots are especially helpful for capturing error messages instead of writing them down. Screen shots are also useful for capturing cosmetic problems that come up while you are testing.

    Taking a screen shot of the entire screen:

    Hold down the keys at the same time. You will hear a camera click and a screen shot will be created on your startup disk. If you have multiple monitors, the picture will be of all the content on all of your monitors.

    Taking a screen shot of a window:

    Hold down the keys at the same time. The arrow cursor will turn into cross-hairs. Select a section of the screen by clicking the mouse button and then dragging. When you let go of the mouse button, you will hear a camera click and a screen shot will be created on your startup disk. Taking a screen shot of a window is only supported in Mac OS 7.6 and greater.

    The first screen shot you take is always labeled 'Picture 1', unless a file or folder with the same name already exists. Subsequent screen shots will be called 'Picture 2', 'Picture 3', etc.

    Note: You will not be able to take a screen shot if your system has crashed.

  • Grab.app (available for Mac OS X only)

    Grab.app is significantly more powerful than Classic Mac OS's built-in screen shot functionality. Grab.app will allow you to take screen shots by cursor selection, on a timer, of a selected window, or various other options.

    Grab.app can be found at
    /Local/Users/YourUserName/Applications/

  Back to top

• Bug Reporting Template
  • TITLE and SUMMARY

    Create problem title and/or include more descriptive summary information. Readers should be able to quickly understand the problem report from this summary.

  • STEPS TO REPRODUCE
    
    
    1. Describe setup or preparation work.
    2. Include explicit and accurate steps to reproduce. Do not include extraneous or irrelevant steps.
    
    
  • RESULTS

    Describe your results and how they differed from what you expected.

  • REGRESSION

    Provide information on steps taken to isolate the problem. Under what conditions or circumstances does the problem occur or not occur?

  • DOCUMENTATION

    If a screen shot or special files are relevant to the problem, describe them here. Include any relevant MacsBug stdlogs or GDB backtraces here.

  • WORKAROUND

    If there is a workaround for the problem, describe it in detail.

  • NOTES

    Document any additional information that might be useful in resolving the problem including references to related problems, leads on diagnosis work already underway, or other people who should participate in the resolution.

  • CONFIGURATION

    If you can not get an Apple System Profiler report, you will want to at least provide this information.

    • CPU, memory, mass storage, displays, peripherals
    • Operating System, Control Panel settings, extensions
    • Network connection, third-party software
    • Other setup information or instructions
  
  

  Back to top

• Submitting Apple Technology Bug Reports

  For complete instructions on submitting bug reports to Apple's Worldwide Developer Relations, please http://developer.apple.com/bugreporter.