Next Previous

Retired Document

Important: This sample code may not represent best practices for current development. The project may use deprecated symbols and illustrate technologies and techniques that are no longer recommended.

ReadMe.html

<HTML>

<HEAD>

   <TITLE>Read Me</TITLE>

</HEAD>

<BODY BGCOLOR="#FFFFFF">

<H1>Read Me About MPFileCopy</H1>

<P>1.0b1</P>

<P>This sample demonstrates a) how to copy a folder and its contents

using the HFS Plus APIs, and b) how to call the File Manager from MP

tasks on Mac OS 9.0 and above. The copy engine preserves long Unicode

names, larger than 2 GB forks, and forks beyond the resource and data

forks (if present).</P>

<P>This sample requires Mac OS 9.0 or above.</P>

<P>Read the <A HREF="#Caveats">Caveats</A> section for some important

caveats.</P>

<H2>Packing List</H2>

<P>The sample contains the following items:</P>

<UL>

   <LI>ReadMe.html -- This document.

   <LI>MPFileCopy.mcp -- A project file for the sample.

   <LI>MPFileCopy.c -- Source code to the sample.

   <LI>MPFileCopy-PPC -- A compiled version of the above.

   <LI>MoreIsBetterParts -- The parts of the DTS sample code library

   MoreIsBetter that are required to compile this sample.

</UL>

<H2>Using the Sample</H2>

<P>To use the sample, simply launch it and first choose a source item

and then choose a destination folder. The program will then copy the

source items to the destination folder, displaying some console

window progress as it goes.</P>

<H2>Building the Sample</H2>

<P>The sample was built using the standard MoreIsBetter build

environment (CodeWarrior Pro 2 compiler) with Universal Interfaces

3.3. You should be able to build the project in CodeWarrior Pro 4

without difficulty as long as you have a late version of Universal

Interfaces 3.3 or later. To build the project, select the "C-PPC"

target, and choose Make from the Project menu. This will build the

MPFileCopy-PPC application.</P>

<H2>How it Works</H2>

<P>Copying a folder is a surprisingly difficult thing to do, although

the HFS Plus APIs make it somewhat easier. I wrestled with the

following key issues while working on this sample.</P>

<UL>

   <LI>Efficiency -- As a rule, File Manager calls are expensive,

   especially when running over a high-latency network link. I wanted

   to avoid making more File Manager than absolutely necessary. To

   this end, I was careful to use <CODE>FSGetCatalogInfoBulk</CODE>

   (introduced by the HFS Plus API), and to avoid getting catalogue

   information more often than necessary.

   <LI>Stack Growth -- The copy engine uses a recursive algorithm. I

   wanted to guarantee that I wouldn't run off the end of my stack,

   especially in the MP task where I am responsible for explicitly

   requesting the stack space I need. For details on the solution,

   see the comments for the <CODE>kCopyFileTaskStackSize</CODE>

   constant in the source.

   <LI><A NAME="Asynchronicity"></A>Asynchronicity -- While Mac OS

   9.0 allows you to make File Manager calls from MP tasks, it

   doesn't guarantee a good user experience if you do so,

   specifically on some older machines with synchronous-only I/O

   buses. See the "True Async Tests" comment in the source for

   details.

   <LI>Transfer Buffer Size -- File Manager is always more efficient

   when you use large transfers. However, on traditional Mac OS the

   File Manager is single-threaded. A large transfer on a slow link

   can take a long time, and thus starve other File Manager clients.

   See the "Transfer Buffer Size" comment in the source for my take

   on solving this problem.

   <LI>File Sharing Bug -- I found and worked around a bug in File

   Sharing in Mac OS 9.0. For details, look for the bug number,

   2397324, in the source code.

   <LI>Multiple Forks -- The code copies all forks of all items it

   copies, including forks associated with a folder. This

   significantly complicates the code, especially when dealing with

   drop folders (see below). I also wanted to avoid the extra

   overhead associated with multiple forks in the most common case

   (items that have just a resource and a data fork). Testing this

   support is also tricky because no current file systems support

   forks beyond the classic resource and data forks. To test the

   code, I throw a switch, <CODE>kSpecialCaseClassicForks</CODE>,

   which forces the code to treat the resource and data forks as if

   they were named forks, which exercises many different code paths.

   <LI>Locked Files -- Copying locked files is tricky because you

   have to create the destination file unlocked, copy the forks, and

   then lock the destination file.

   <LI>Modification Dates -- Copying items without changing their

   modification dates requires copying the item and then setting the

   modification date after the copy is finished.

   <LI>Marking Items as Busy --While copying items, you should mark

   them as busy so that the Finder won't process them until you've

   finished copying. I do this by setting a file's type to a special

   value (<CODE>kFirstMagicBusyFiletype</CODE>) and any items

   creation date to another special value

   (<CODE>kMagicBusyCreationDate</CODE>). Once I've finished copying

   the item, I reset these values.

   <LI>Drop Folders -- Copying items into drop folders is tricky.

   Firstly, the rules for drop folders require that you open all the

   forks of the file before you start writing to any of them. This is

   tricky when there is just a resource and data fork, but gets

   really nasty when you have to deal with multiple forks. Also, when

   copying into a drop folder, you can't change an items file type,

   locked status, or dates after any fork in the file has content, so

   many of the techniques described above don't work properly. The

   upshot is that items copied into a drop folder are always

   unlocked, have the current time as their modification date and

   can't be marked as busy. This is the same behaviour as the Finder.

</UL>

<H2><A NAME="Caveats"></A>Caveats</H2>

<P>This sample still has a number of problems. I shipped with these

problems because I thought it was more important to get the sample

out the door than to make it perfect. I will attempt to address these

in a future release.</P>

<UL>

   <LI>Performance versus Responsiveness -- The current code is

   neither as fast as I would like it to be, nor does it allow the

   system to respond well to user actions while a copy to a fast

   volume from a preemptive task. I can improve the performance by

   increasing the transfer buffer size, but that makes the

   responsiveness worse. I'm not sure what the right solution is

   here.

   <LI>Performance versus Finder -- The code currently copies

   somewhat slower than Finder 9.0. Increasing the transfer buffer

   size brings me almost to par with the Finder for copying a small

   number of large files, at the cost of the lowered system

   responsiveness described above. Also, Finder's copying code beats

   this code in two important aspects. The upshot is that this code

   would require significant tuning to match the copying performance

   of Finder.

   <UL>

      <LI>My current algorithm for copying (copying one item at a

      time via a recursive descent of the folder hierarchy) is

      inherently slower than the Finder's algorithm (read items until

      you fill the transfer buffer, then write items until the

      transfer buffer is empty) for large numbers of small files on

      the same volume. This is hard to fix without a complete rewrite

      of the code.

      <LI>Finder bypasses the File Manager to overlap I/O when

      copying between a local and an AppleShare volume. The technique

      for doing this is not documented to third parties, so can't be

      used in this sample.

   </UL>

   <LI>Carbon -- The code compiles for Carbon but I don't have a

   Carbon-compatible version of MSL with which to link it. Long term,

   MSL will be available for Carbon, but I wanted to get this "out

   the door".

   <LI>CarbonLib and Synchronous I/O -- I discussed the problems with

   asynchronicity above, along with my solution. That solution is not

   possible under Carbon because some of the required APIs are not

   Carbon-compatible. Under Mac OS X this isn't a problem (all

   drivers will be asynchronous on Mac OS X), but if you build a

   Carbon version of this code and run it on traditional Mac OS on

   some older machines, you will encounter this problem again.

   <LI>Destination Names -- The sample does not make the name of the

   destination item unique. This is tricky code correctly. I have the

   code lying around (in Pascal, as part of Internet Config) but I

   haven't had time to integrate it here.

</UL>

<H2>Credits and Version History</H2>

<P>If you find any problems with this sample, mail

&lt;DTS@apple.com&gt; with "Attn: Quinn" as the first line of your

mail and I'll try to fix them up.</P>

<P>1.0d1 (Oct 1999) was an Apple internal release for reviewers.</P>

<P>1.0d2 (Oct 1999) was another Apple internal release for reviewers.

</P>

<P>1.0b1 (Nov 1999) was the first shipping version.</P>

<P>Share and Enjoy.</P>

<P>Quinn "The Eskimo!"<BR>

Apple Developer Technical Support<BR>

Networking, Communications, Hardware</P>

<P>15 Nov 1999</P>

<P>&nbsp;</P>

</BODY>

</HTML>

Next Previous