Monitoring File Changes
with the File System Events API

Apple’s File System Events API gives applications the ability to detect changes to paths containing relevant files recursively within an entire directory tree, both during execution and between launches. The File System Events API tackles the problem of detecting changes within a large chunk of the file system. A prime example of a use of this capability is Time Machine, which periodically launches and backs up any files modified since it last checked. Alternatively, this API could be used for syncing two directory trees on different machines over a network, or giving the user near-instant feedback about changes to specific files, for example in applications where many people collaborate on the same set of files.

Prior to the existence of the File System Events API, developers could use kernel queues to monitor files. Kernel queues (also known as kqueues), are more general than the File System Events API in that they can be used to monitor activity on not just the file system, but other types of activity as well, including network socket activity and changes to running processes. However, it has some limitations compared to the File System Events API. The kqueue API is best at monitoring individual files or directories during the execution of an application, and does not scale as well as the File System Events API.

This article will walk through the creation of a sample application called "ImageHistory" that uses the File System Events API to provide an up-to-date list of the most recently added and edited image files in the user’s home directory. It assumes basic familiarity with Cocoa development using Objective-C.

The three primary components of this application are:

• the user interface
• the File System Events API
• an algorithm that indicates which file in a given path has been modified.

The User Interface will be very simple. It will consist of a table of images in the user's home directory that have changed since its first launch, with the most-recent images at the top. The File System Events API will cause modified and newly created images to appear at the top of the table.

The File System Events API generates notifications of changes within a file system path, which in this case will be the current user’s home directory. The API will pass groups of events to a callback that will determine, given the path provided by the API, which files have changed and whether or not they are images.

While the File System Events API gives us the path to the directory in which events have occurred, it does not tell us specifically which files have changed. In order to determine specifically which files have been created or modified, we need our application to also keep track of the time at which the most recent event was received.

Putting Together the User Interface

in Xcode, create a new Cocoa application. Go to Project -> Edit Project Settings. Choose the 'Build' tab and set the 'Objective-C Garbage Collection' option to 'Supported.' Now we won't have to worry so much about memory management (i.e., retaining and releasing objects).

Now open your new application’s nib file in Interface Builder. Drag an NSTableView object from the object palette of the library window onto the empty window and size it to resemble Figure 1. Delete the extra column and set the title on the remaining column to ‘Recent Image Activity.’  We need to set the type of data that will be displayed in the columns: The default is text, but we want to display images. To allow this, drag an NSImageCell object from the Library and drop it onto the NSTableView. Finally, set the table to adjust automatically to fit the containing window in the Size Inspector.

Figure 1: Creating a Table View

Now we just need to hook the table up to an NSArray and start storing some images in it. First, let’s go over the AppController class, which we will use to define the behavior of the application. 

As this application has a very simple user interface, we can implement all of the controller functionality in a single class, which we’ll call AppController. We’ll need to declare some instance variables:

• the table variable, which we will bind to an NSTableView in Interface Builder.
• fm, an instance of NSFileManager, which we will use to retrieve the file modification date of files in paths passed to us via the File System Events API.
• images, an NSMutableArray, which we will use to store paths to modified images to be displayed.
• stream, this is our interface to the File System Events API. We’ll discuss it more later.
• appStartedTimestamp, we will use this to determine which file in a directory has been modified (more discussion later).
• lastEventId, we ’ll use this variable to remember where we left off last time the app was run.
• pathModificationDates, we'll keep track of the modification dates for individual directories here.

Here’s the header file with the declarations we’ll need:

#import <Cocoa/Cocoa.h>
@interface AppController : NSObject {
    NSFileManager* fm;
    IBOutlet NSTableView* table;
    NSMutableArray* images;
    NSMutableDictionary* pathModificationDates;
    NSDate* appStartedTimestamp;
    NSNumber* lastEventId;
    FSEventStreamRef stream;
}

- (void) registerDefaults;
- (void) initializeEventStream;
- (void) addModifiedImagesAtPath: (NSString *)path;
- (void)updateLastEventId: (uint64_t) eventId;
- (BOOL)fileIsImage: (NSString *)path;

@end

Having saved AppController.h, we need to hook a few things up in the nib using Interface Builder. Drag and drop an instance of NSObject from the library to the nib. Select the instance and click on the identity button in the properties panel. Set the class to AppController. Connect the delegate outlet of File's Owner to AppController. Connect the table outlet of AppController to the NSTableView. Finally, connect the NSTableView's dataSource outlet to AppController.

We want to make sure the rows are tall enough to show the image at a sufficiently big size. Add the following line to awakeFromNib:

    [table setRowHeight:100.0];

Next let’s implement the data source methods necessary to properly display images in the table. We need to implement two methods - numberOfRowsInTableView, and tableView:objectValueForTableColumn:row:. 

- (int)numberOfRowsInTableView: (NSTableView *)aTable
{
	return [images count];
}

- (id)tableView: (NSTableView *)aTable
      objectValueForTableColumn: (NSTableColumn *)aCol
      row: (int)aRow
{
    NSImage *image =
        [[NSImage alloc] initByReferencingFile:
            [images objectAtIndex:([images count] - 1) - aRow]];
    return image;
}

The first method, numberOfRowsInTableView, returns the size of the images array. The second method, tableView, initializes a new NSImage object with the path most recently added to images.

Handling Events as they Occur

In order to receive File System Events, we first must create an event stream and register a callback. Let’s write a method called initializeEventStream that we can call from awakeFromNib that will accomplish this task.

- (void) initializeEventStream
{
    NSString *myPath = NSHomeDirectory();
    NSArray *pathsToWatch = [NSArray arrayWithObject:myPath];
    void *appPointer = (void *)self;
    FSEventStreamContext context = {0, appPointer, NULL, NULL, NULL};
    NSTimeInterval latency = 3.0;
    stream = FSEventStreamCreate(NULL,
                                 &fsevents_callback,
                                 &context,
                                 (CFArrayRef) pathsToWatch,
                                 [lastEventId unsignedLongLongValue],
                                 (CFAbsoluteTime) latency,
                                 kFSEventStreamCreateFlagUseCFTypes
    );

    FSEventStreamScheduleWithRunLoop(stream,
                                     CFRunLoopGetCurrent(),
                                     kCFRunLoopDefaultMode);
    FSEventStreamStart(stream);
}

In this method we pass in a reference to a callback called fsevents_callback. This callback will be called by the File System Events API every time there is a group of events in the queue ready to be processed. We also pass in the variable context, through which will will pass the callback a reference to the AppController object (appPointer).

Notice that we pass in the result of the call of unsignedLongLongValue on lastEventId. This parameter specifies the event after which we would like to begin receiving new events. We discuss this further in the section entitled “Monitoring Persisted Events.”  The latency variable, set to 3 seconds, is passed in to indicate the amount of time to wait after receiving an event before passing it to the callback. Waiting longer can increase efficiency.

Finally, we pass in the constant kFSEventStreamCreateFlagNone, which tells the API to watch for events in the given directories and subdirectories.

Following is a listing of fsevents_callback, which will be called by the File System Events API as events are received.

void fsevents_callback(ConstFSEventStreamRef streamRef,
                       void *userData,
                       size_t numEvents,
                       void *eventPaths,
                       const FSEventStreamEventFlags eventFlags[],
                       const FSEventStreamEventId eventIds[])
{
    AppController *ac = (AppController *)userData;
    size_t i;
    for(i=0; i < numEvents; i++){
        [ac addModifiedImagesAtPath:[(NSArray *)eventPaths objectAtIndex:i]];
        [ac updateLastEventId:eventIds[i]];
    }

}

The File System Events API passes in to the callback function several useful parameters. The userData parameter can be used to pass in arbitrary information (we’re using it to pass in a reference to the application controller in this case). The eventPaths argument is a pointer to an array of paths corresponding to the eventIds argument, which is a pointer to an array of event IDs. Using a for-loop along with numEvents parameter, we can iterate through each path and pass it to the insertModifiedImageForPath method.

Directory Hierarchy Caching

As mentioned before, the File System Events API only gives us the path to a directory in which an event has occurred; it doesn’t tell us exactly which file was modified. One technique we can use to determine the exact file that has been modified is to examine the timestamp of each file in the directory, checking it against the last time a file in the directory was modified. If we find a file with a timestamp later than the previous modification, we know that it must be the file to which an event has occurred.

- (void) addModifiedImagesAtPath: (NSString *)path
{
	NSArray *contents = [fm directoryContentsAtPath:path];
	NSString* fullPath = nil;
	BOOL addedImage = false;

    for(NSString* node in contents) {
        fullPath = [NSString stringWithFormat:@"%@/%@",path,node];
        if ([self fileIsImage:fullPath])
        {
            NSDictionary *fileAttributes =
              [fm attributesOfItemAtPath:fullPath error:NULL];
            NSDate *fileModDate =
              [fileAttributes objectForKey:NSFileModificationDate];
            if([fileModDate compare:[self lastModificationDateForPath:path]] ==
              NSOrderedDescending) {
                [images addObject:fullPath];
                addedImage = true;
            }
        }
    }

    if(addedImage){
        [table reloadData];
    }

    [self updateLastModificationDateForPath:path];
}

- (BOOL)fileIsImage: (NSString *)path
{
    NSWorkspace *sharedWorkspace =
      [NSWorkspace sharedWorkspace];
    return [sharedWorkspace type:
      [sharedWorkspace typeOfFile:path error:NULL] conformsToType:@"public.image"];
}

The above listing shows how to check if a file has been modified. Notice that it depends on the lastModificationDateForPath method. This method accesses the pathModificationDates dictionary. If the path is in the dictionary, it returns the time at which the path was last modified.   If it is not in the dictionary, it returns the time the app was started. Note also that we’re also checking to make sure the file is an image file by checking for UTI (Universal Type Identifier) equivalence of the file with the typeOfFile:error: and type:conformsToType: methods provided by NSWorkspace.

Monitoring Persisted Events

Persisted events are File System Events that the OS records even when your application is not running. In order to monitor persisted events, we need to:

• store the last received event ID to the disk before the application exits, and load it when the application starts; and then,
• store a dictionary of paths with modification dates.

Using Cocoa, both of these tasks can be accomplished through the NSUserDefaults class. In the case of the event ID, we first need to convert the event ID (which is an unsigned long long) into an NSNumber using the numberWithUnsignedLongLong method. We then need to store the resulting NSNumber object using the setObject method in order to properly serialize it. Storing the dictionary of paths and timestamps is straightforward.

- (NSApplicationTerminateReply)applicationShouldTerminate: (NSApplication *)app
{
    NSUserDefaults *defaults = [NSUserDefaults standardUserDefaults];
    [defaults setObject:lastEventId forKey:@"lastEventId"];
    [defaults setObject:pathModificationDates forKey:@"pathModificationDates"];
    [defaults synchronize];
    FSEventStreamStop(stream);
    FSEventStreamInvalidate(stream);
    return NSTerminateNow;
}

Here we implement the applicationShouldTerminate method, which is called before an application exits. Notice that besides storing path timestamps and the last event ID before shutting down, we also clean up our stream reference by calling FSEventStreamStop and FSEventStreamInvalidate. When the app starts, we load the last event ID and the dictionary of path modification dates in awakeFromNib.

     pathModificationDates =
       [[[NSUserDefaults standardUserDefaults] dictionaryForKey:@"pathModificationDates"] mutableCopy];
     lastEventId = [[NSUserDefaults standardUserDefaults] objectForKey:@"lastEventId"];

Testing Things Out

Build and run the ImageHistory Xcode project. As this is the first time the app has run, there will no images in the table. Using the command line, navigate to the project directory and duplicate the file named "motorcycle.png".

$ cp motorcycle.png motorcycle-copy1.png

Within three seconds, you should see the image pop into view in the ImageHistory application. Now, shut down the application. Use the cp command again to copy the file again.

$ cp motorcycle.jpg motorcycle-copy2.jpg

Start the application up again. Within a few seconds, the newly copied image will appear in the table.

Wrapping Up

In this tutorial we have worked through the process of initializing an event stream, implementing a callback to handle events, determining which files have been affected by an event, as well has how to check for persisted events.

Download the sample Xcode project ImagesHistory (DMG, 4.1 MB) in its entirety.

Posted: 2008-10-21