FileDescriptor writing to an unexpected file

Question

Created 2w

Replies 8

Boosts 0

Participants 3

I'm using a file descriptor to write into a file. I've encountered a problem where if the underlying file is removed or recreated, the file descriptor becomes unstable. I have no reliable way to confirm if it's writing on the expected file.

let url = URL(fileURLWithPath: "/path/")
try FileManager.default.removeItem(at: url)
FileManager.default.createFile(atPath: url.path, contents: .empty)

let filePath = FilePath(url.path)
var fileDescriptor = try FileDescriptor.open(filePath, .readWrite)

// The file is recreated - may be done from a different process.
try FileManager.default.removeItem(at: url) // L9
FileManager.default.createFile(atPath: url.path, contents: .empty) // L10

let dataToWrite = Data([1,1,1,1])
try fileDescriptor.writeAll(dataToWrite) // L13
let dataWritten = try Data(contentsOf: url)
print(dataToWrite == dataWritten) // false

I would expect L13 to result in an error. Given it doesn't:

Is there a way to determine where fileDescriptor is writing?
Is there a way to ensure that fileDescriptor is writing the content in the expected filePath?

Answered by on-d-go in 788983022

Is there a way to ensure that fileDescriptor is writing the content in the expected filePath?

This can be done by comparing their inodes.

var fdStat = stat()
var fpStat = stat()
let isValid = fstat(fileDescriptor.rawValue, &fdStat) == 0 && stat(filePath.string, &fpStat) == 0 && fdStat.st_ino == fpStat.st_ino

If it's not pointing to filePath, recreate the file descriptor.

if !isValid {
    fileDescriptor = try FileDescriptor.open(filePath, .readWrite)
}

This works, but is there a better way? Also, is there a high-level API in FileManager that does fstat and stat?

Boost

Answer 1

on-d-go OP

2w

Accepted Answer

Is there a way to ensure that fileDescriptor is writing the content in the expected filePath?

This can be done by comparing their inodes.

var fdStat = stat()
var fpStat = stat()
let isValid = fstat(fileDescriptor.rawValue, &fdStat) == 0 && stat(filePath.string, &fpStat) == 0 && fdStat.st_ino == fpStat.st_ino

If it's not pointing to filePath, recreate the file descriptor.

if !isValid {
    fileDescriptor = try FileDescriptor.open(filePath, .readWrite)
}

This works, but is there a better way? Also, is there a high-level API in FileManager that does fstat and stat?

0

Answer 2

endecotp OP

2w

I would expect L13 to result in an error

Why?

I don’t know what happens when all Apple’s extra layers are added on top, but the underlying behaviour of POSIX file descriptors is well-defined. If the file is deleted, the file descriptor remains valid and you can continue to read and write to what is now a “anonymous“ file. Only once you close the last open file descriptor for a file is it actually deleted.

2

Answer 3

DTS Engineer OP

Apple

2w

What endecotp said plus one clarification regarding this:

I don’t know what happens when all Apple’s extra layers are added on top

FileDescriptor is a very thin layer over the BSD APIs, so you should expect FileDescriptor.open(…) to behave like the open system call.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

0

Answer 4

on-d-go OP

2w

Here's another example where the System.FileDescriptor doesn't read from the desired FilePath

let dataWrittenInFile1 = Data([1,1,1,1])
let file1 = URL(fileURLWithPath: "/path/to/file1")
try FileManager.default.removeItem(at: file1)
FileManager.default.createFile(atPath: file1.path, contents: dataWrittenInFile1)

let fileDescriptor1 = try FileDescriptor.open(file1.path, .readWrite)
try fileDescriptor1.close()

let dataWrittenInFile2 = Data([2,2,2,2])
let file2 = URL(fileURLWithPath: "/path/to/file2")
try FileManager.default.removeItem(at: file2)
FileManager.default.createFile(atPath: file2.path, contents: dataWrittenInFile2)
try FileDescriptor.open(file2.path, .readWrite)

var dataInFile1 = Data(count: 4)
try dataInFile1.withUnsafeMutableBytes({try fileDescriptor1.read(into: $0)})
print(dataInFile1 == dataWrittenInFile1) // false
print(dataInFile1 == dataWrittenInFile2) // true

I would expect fileDescriptor1 to read from file1. Instead, it reads from file2. In my view, reading from a closed file descriptor should have resulted in an error. This problem extends to Foundation.FileHandle as well.

Do you know if there's a high-level API, included with iOS, macOS, and tvOS that does:

func fileDescriptor(_ fileDescriptor: System.FileDescriptor, pointsTo filePath: System.FilePath) -> Bool {
    //
}

0

Answer 5

endecotp OP

2w

I would expect fileDescriptor1 to read from file1

As I wrote in my original reply, Why would you expect that?

The behaviour of file descriptors is well-defined. The behaviour you are seeing is allowed by the spec.

What’s actually happening is that the kernel is allocating the same FD for the second open as for the first, which it can do because you closed the first one. So when you read, you read from the second file.

What are you actually trying to achieve here? There may be other APIs that better match your requirements. If you really want to use file descriptors you need to understand them properly. You will be constantly surprised if you try to guess how you would expect them to behave.

1

Answer 6

on-d-go OP

2w

What’s actually happening is that the kernel is allocating the same FD for the second open as for the first, which it can do because you closed the first one. So when you read, you read from the second file.

I was (and still am) hesitant to make that assumption because the actual implementation is behind these APIs.

However, I see your point; these APIs don't necessarily need to read/write from the filePath, likely because they are eventually backed by the posix’s filedescriptor.

My use case requires that I read/write to a pre-defined filepath. If I've opened a file descriptor with a filepath, I need to know if or when it's not reading/writing from/to that filepath. My question still is:

Do you know if there's a high-level API, included with iOS, macOS, and tvOS that does:
func fileDescriptor(_ fileDescriptor: System.FileDescriptor, pointsTo filePath: System.FilePath) -> Bool {
    //
}

0

Answer 7

endecotp OP

2w

If I've opened a file descriptor with a filepath, I need to know if or when it's not reading/writing from/to that filepath. My question still is:
Do you know if there's a high-level API, included with iOS, macOS, and tvOS that does:
func fileDescriptor(_ fileDescriptor: System.FileDescriptor, pointsTo filePath: System.FilePath) -> Bool {
    //
}

I think your stat/fstat code is OK. I don’t see equivalent methods on the swift fd struct, so I guess there is no “high level API” in that sense.

Are you keeping this file descriptor open for a long time? If not, I suggest trying to write atomically. Create a temporary file, write your data to that file, close, and then rename to the destination. Rename is atomic; there is no opportunity for e.g. another process to rename the file while you are writing to it.

Otherwise… what is the underlying reason for this requirement? In the situation where the user has deleted the file, maybe the user doesn’t want it to reappear?

Most programs do not detect / handle this case.

0

Answer 8

DTS Engineer OP

Apple

1w

I was (and still am) hesitant to make that assumption because the actual implementation is behind these APIs.

Regarding the System framework, I think it’s fine to assume that it calls directly through to the Posix APIs. The entire framework exists to smooth above the bumps in calling those Posix APIs directly.

I’ll admit that the situation for FileHandle is more nuanced but, honestly, I wouldn’t worry about this. Remember that FileHandle has a fileDescriptor property, which constrains its behaviour considerably.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

1