Is there a way to track leaking file descriptors?

Question

My little network extension is running out of file descriptors. My suspicion is that something in the Security framework is not being deallocated, although even this doesn't make a great deal of sense:

The extension looks at each flow, and gets a SecStaticCodeRef for it, finds the pathname, makes a decision, and stores the result of that decision in an NSCache<NSData, NSNumber> where the key is flow.metaData.sourceAppUniqueIdentifier. This goes through a couple layers of abstractions (the cache is in one Swift class, and it calls another Swift class that gets the security info and then returns the pathname, or throws an error).

As an example, after running for a couple of days, it has 1074 open file descriptors for /System/Library/PrivateFrameworks/CloudKitDaemon.framework/Support/cloudd -- and only had 732 three hours ago.

362

Posted by

kithrup

Reply

Add a Comment

Answer 1

Hello!

One of the ways to tack file descriptors is using File Activity template in Instruments. It could be done with following steps:

Open Instruments;
Select the extension as a target;
Open File Activity template;
Start recording;
Try to reproduce the issue;
Stop recording;
Click on Filesystem Activity track on the top;
Click on Filesystem Statistics popup menu on the toolbar of details below;
Select File Descriptor History.

In details view file descriptor events with backtraces should be appeared. The data could by filtered out for event about suspected file by using Detail Filter.

Anton

Posted by

Developer Tools Engineer

Add a Comment

Answer 2

Hello!

One of the ways to tack file descriptors is using File Activity template in Instruments. It could be done with following steps:

Open Instruments;
Select the extension as a target;
Open File Activity template;
Start recording;
Try to reproduce the issue;
Stop recording;
Click on Filesystem Activity track on the top;
Click on Filesystem Statistics popup menu on the toolbar of details below;
Select File Descriptor History.

In details view file descriptor events with backtraces should be appeared. The data could by filtered out for event about suspected file by using Detail Filter.

Anton

Posted by

Developer Tools Engineer

Add a Comment

Answer 3

Oooh, that does seem to have done it. Meanwhile, after a hospital scare involving my mother, I put in a bunch of log messages. (BTW: is there a better way to count open file descriptors than doing a readdir on /dev/fd? FreeBSD has kern.proc.nfds, but xnu doesn't seem to, and I didn't see a similar one from a quick search.)

It is definitely being opened in the Security framework; however, I found that I was keeping a pointer to something using it when I only needed to get the pathname. (Specifically, I had a lazy var pathname; however, since that information was never going to change, I just set it during init, and am going to see if that helps. This does mean I seem to have a non-cleaned up flow somewhere, so I'll also have to look into that.)

Posted by

kithrup

Add a Comment

Answer 4

is there a better way to count open file descriptors

Yes. The code in this post shows the basic idea.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Posted by

eskimo

Add a Comment

Answer 5

Ouch, that is a much worse way to count open file descriptors: it takes N+1 system calls, where N is the limit (which could be a 64-bit MAXINT!). Compared to a readdir on /dev/fd, which is generally going to only be 3 or 4 system calls.

Posted by

kithrup

Although I haven't tried on iOS -- this was for macOS.

—
kithrup

Add a Comment

Answer 6

Is performance a concern here? I thought you were using this for debugging?

Some other notes:

The maximum number of file descriptions is not “64-bit MAXINT”. File descriptors are of type CInt, which on Apple platforms is Int32.
Moreover, the practical limit is much lower. For most code it’s 256, the default for RLIMIT_NOFILE.
/dev/fd is not available on iOS or its child platforms; it’s macOS only.
/dev/fd is not available to a sandboxed app.
The code in the post I referenced gets the paths, which readdir won’t give you.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Posted by

eskimo

Add a Comment

Answer 7

I had gotten confused because of rlim_t using 64-bit integers. Foolish on my part. I had in fact been increasing the process' number of file descriptors -- ironically because we'd been running out of file descriptors.

I didn't need the paths, and without the Instruments version, I was calling my function... a lot ;). In reality it didn't matter, but I still hesitate at doing a few thousand system calls instead of just a handful. But thats because I am old, and date back to when a boundary crossing could take over a thousand cycles.

Posted by

kithrup

Add a Comment

Is there a way to track leaking file descriptors?

Accepted Reply

Replies