Error when materializing files

Hello,

we have a file provider based macOS app. Around June we started receiving reports that our users have problems when opening files. Sometimes they get "Invalid argument" alerts

after double-clicking on a dataless file. We receive similar errors when trying to materialize the files programmatically from our app (not FP extension)(*):

"The operation could not be completed. Invalid argument"
code: 22
domain: "NSPOSIXErrorDomain"
underlyingError: 
    "cannotMaterialize"
     code: 33
     domain: "libfssync.VFSFileError"

We also see those errors with matching timestamps in the output from fileproviderctl dump:

> (...) update-item: 🔶 last:(...) (-1min27s) (...) error:'NSError: POSIX 22 "The operation couldn’t be completed. Invalid argument" Underlying={NSError: libfssync.VFSFileError 33 "cannotMaterialize" }}' domain:none category:<nil> (...)

At the same time our file provider extension receives fetchPartialContents call or no call at all. If it receives the call it finishes with success and returns correct range:

requestedRange: "{0, 15295}"
returnedRange: "{0, 524288}"
alignment: "16384"

Sadly we don't know how to reproduce those issues. We will be grateful for any hints that could be useful in debugging.

Some more context:

  • if materializing a file fails with this error, subsequent materialization attempts fail similarly
  • in local testing when downloading a file with the code below, or double-click, we only get regular fetchContents call
  • file returned by fetchPartialContents should have correct size
  • the app is built with XCode 16.1, customers reporting this issue have OS/FP versions: 24F74/2882.120.74 and 24G90/2882.140.30

(*) More or less the code that we use to materialize files

func materializeURL(_ url: URL) throws {
    if try url.isDataless() {
        var error: NSError? = nil
        let coordinator = NSFileCoordinator(filePresenter: nil)

        coordinator.coordinate(readingItemAt: url, error: &error) { _ in }

        if let error {
            throw error
        }
    }
}

private extension URL {
    func isDataless() throws -> Bool {
        let downloadStatus = try self
            .resourceValues(forKeys: [.ubiquitousItemDownloadingStatusKey])
            .ubiquitousItemDownloadingStatus

        return downloadStatus == .notDownloaded || downloadStatus == .none
    }
}

In this situation, you might contact your users who can reproduce the issue and are willing to help, ask them to install the file provider profile, follow the guide to reproduce the issue and capture a sysdiagnose, and then provide sysdiagnose to you. From there, you can look into the system logs, and try to figure how the system behaves differently from the workable case (your case).

The debugging process may take time, but given that you don't have a reproducible case, that's probably the way you can move forward.

Best,
——
Ziqiao Chen
 Worldwide Developer Relations.

We found the cause - this happens when extension returns content of different size (even larger) than the range for fetchPartialContents.

In our case it happened quite rarely and the content was always larger - which is quite surprising looking at the docs in NSFileProviderPartialContentFetching.h:

On-disk layout:

(...)

The file contents outside of the fetched range are ignored by the system. (...) For instance, if the fetchedRange is {offset:0x100000, length:0x1000} (...) The ranges {0, 0x100000}, and {0x101000, EOF} can be anything including sparse ranges.

So actually in this example, if I understand correctly what fetched range refers to, if EOF > 0x101000 (i.e. fetched data extends beyond the range), the issue reproduces.

FWIW leaving some more details in case it's useful for someone reading this:

Analyzing sysdiagnose logs was quite helpful - in logs from log show --archive system_logs.logarchive we noticed this

extent length is not as expected (524488 != 524288)

and based on that quickly found that we are actually returning 524488 bytes of data instead of 524288.

The issue can be quite easily simulated / reproduced with FruitBasket by applying attached patch. Steps to reproduce

  • create big enough file (mkfile 10m test.txt)
  • remove download
  • read some first bytes (head -c 4096 test.txt)
  • read fails and the file is now permanently broken, cannot be downloaded

Error when materializing files
 
 
Q