Seeking clarification on macOS URLs with security scope

And one more little gotcha.

I started all of this from a Login Item. But I've seen references to Login Items being frowned upon by Apple, so I wanted to change that to a Launch Agent. Plus, XPC doesn't work with a Login Item.

And this is meant for the Mac App Store, so all of it has to be sandboxed. That meant putting the launch agent into an "app-like wrapper".

XPC doesn't work with a sandboxed launch agent either, in spite of it being in an app group. But that's OK. I configure the launch agent from the database and run it on a calendar interval, which is better regardless.

But it looks like I got carried away with sandbox restrictions. I have to give my launch agent the "user selected files" capability. Otherwise, it can't talk to the ScopedBookmarkAgent at all, even on Sequoia.

First off, on the bug side...

I think you filed this as FB20915052; however, this is also a known issue (r.161870449).

This is fixed in the current seed of 26.2 (25C5031i).

I understand about those kinds of race conditions. I'm not sure why it would be classified as an "attack".

Because one of the most common attacks the system is dealing with today involves passing an input path into one subsystem, then modifying that path "midstream" by replacing a directory with a symlink. Under the wrong circumstances, that can end up retargeting the action that was originally intended. The standard way of preventing this attack is by passing one of the "no follow" flags to open, but in a complex system that can be extremely difficult to guarantee and validate.

The new ".nofollow" construct effectively "attaches" the no follow flag to the path itself, forcing that flag on all open calls regardless of the actual flag passed in.

And I have no idea what that has to do with URL bookmarks.

Because bookmarks are critical infrastructure used throughout the system, which makes them a common starting point to TOC/TOU attacks.

I don't think we're talking about the same bug here.

We are. The basic issue here is that both of these are valid references to EXACTLY the same path:

/.nofollow/
/

They are most definitely invalid. I mean, they do exist.

No, that's not what I mean. The directories "/.nofollow" and "/.resolve" basically exist to prevent those locations from being modified, but the system does NOT treat them as normal directories. You can see this for yourself in Terminal.app. If you run the command:

cd /.nofollow/Applications/

...you'll find that you're now inside the "/Applications/" directory. That is because both these are valid paths to the same directory:

/.nofollow/Applications/
/Applications/

Similarly, the correct behavior is that this command:

cd /.nofollow

...changes to the ".nofollow" directory, but this command:

cd /.nofollow/

...actually changes to "/". And yes, that's exactly what Terminal.app will do.

The bug here is that the Foundation isn't handling this properly. It's converting this path:

/.nofollow/

to this path:

/.nofollow

...which is the wrong behavior. I'll note here that the actual issue here isn't actually with bookmark handling, but is actually with the URL system. More specifically, if you manually construct a path and list contents with NSFileManager, this:

/.nofollow/Applications/

...lists the contents of "/Applications/", but this:

/.nofollow/

...returns an empty directory. That's what the bug here actually is. Foundation is stripping the trailing slash from "/.nofollow/", which then changes the target.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

My confusion levels have now reached the saturation point.

The new ".nofollow" construct effectively "attaches" the no follow flag to the path itself, forcing that flag on all open calls regardless of the actual flag passed in.

What is the scope of this new construct? Is this going to be applied to all URLs? Only URLs decoded from a security-scoped bookmark? Only URLs to root directories? What about non-security-scoped bookmarks? What about external drives? What about non-APFS volumes? What about network volumes?

In the past, we've been specifically asked to avoid meddling in the affairs of implementation details. But now it seems like we're being thrown in head first.

You can see this for yourself in Terminal.app

I'm deploying to the Mac App Store with a sandboxed launch agent. Virtually every system-level API I've encountered behaves differently in this environment. I cannot test in Terminal.

cd /.nofollow/

...actually changes to "/". And yes, that's exactly what Terminal.app will do.

I assume you mean this is Terminal.app will do in 26.2, correct?

I'm not sure what you mean with respect to Foundation behaviours. I assume you are talking about the internals of the Foundation framework, far below anything that we can see.

I just did a little test. My app's primary functionality is doing a scan of a user-supplied folder. That folder could be /.

Based on previous posts here in the forum, I take the URL provided from NSOpenPanel, save it to a (non-secure) bookmark, and then resolve that bookmark and use that new URL. I don't need a security-scoped URL for this use, so I assumed a non-secure bookmark would provide the same functionality in this case.

(I haven't changed my code to issue multiple stopAccessingSecurityScopedResource calls on the implicitly security-scoped original URL, which has also been recommended here in the forums.)

But when I change that bookmark to a security-scoped URL, it resolves to a URL of "file:///.nofollow/". It does have the trailing slash in 26.1. Perhaps Foundation is interpreting it differently at a lower level. It most definitely points directly to the empty directory at /.nofollow/, which is not correct.

I understand that you've said the bug is fixed in 26.2. But now I'm more concerned about the scope of this new behaviour. I don't really need any security-scoped bookmarks in this app. Perhaps I should just avoid them entirely until this settles down.

PS: I now also see the wisdom in accepting the default Xcode deployment targets that include non-zero minor build versions.

So to summarize: 26.0 shipped with a bug, which evolved to an even more annoying bug in 26.1 which is supposedly fixed in 26.2 beta. I wish Apple considered issuing a fast patch rather than waiting for months until the next minor release, because this is affecting many users already, and we're only 1 week into 26.1 official release. So I'll have to reply to emails by frustrated users for another couple months...?

Unfortunately users of my different apps also started reporting another issue that could be related: paths on an external volume, such as one mounted by "NTFS for Mac", or a path on a Synology volume, are "converted" to / (Macintosh HD) after going through laundry. Could someone at Apple confirm whether this is also a known issue? I wasn't sure if I should report this here or in my recent post.

What is the scope of this new construct?

System-wide. This is actually happening in the kernel when in-process incoming paths. The critical syscall is "open" (and its variants), but the entire kernel will accept these paths.

Is this going to be applied to all URLs? Only URLs decoded from a security-scoped bookmark? Only URLs to root directories? What about non-security-scoped bookmarks? What about external drives? What about non-APFS volumes? What about network volumes?

Nothing really limits the scope here, but I expect them to be increasingly common. Indeed, I suspect that they're "missing" in some context is actually a misunderstanding of what's going on. For example, you probably won't see them with fileReferenceURLs, but that's actually because the fileReferenceURL isn't storing a path at all.

I'm deploying to the Mac App Store with a sandboxed launch agent. Virtually every system-level API I've encountered behaves differently in this environment. I cannot test in Terminal.

I suggested testing in Terminal as an easy way to understand how the core system behaves, not to replicate your app’s specific behavior.

I assume you mean this is Terminal.app will do in 26.2, correct?

No, actually this works in a much broader set of systems than that. The handling of "/" itself is inconsistent, but this command will move you to "/Applications/":

cd /.nofollow/Applications

...even on macOS 15.6.1.

I'm not sure what you mean with respect to Foundation behaviours. I assume you are talking about the internals of the Foundation framework, far below anything that we can see.

Unfortunately, it's not all that "far" below. The core bug here is actually in fileSystemRepresentation (both CF and Foundation). Putting the bug into code, if you run this:

#import <Foundation/Foundation.h>

void TestPath(NSString* path) {
    NSFileManager* fileMan = [NSFileManager defaultManager];
    NSError* err;
    NSArray* cont = [fileMan contentsOfDirectoryAtPath:path error: &err];
    NSLog(@"C: %ld Rep: %s",cont.count, path.fileSystemRepresentation);

}
int main(int argc, const char * argv[]) {
    @autoreleasepool {
        TestPath(@"/Applications/");
        TestPath(@"/.nofollow/Applications/");
        TestPath(@"/");
        TestPath(@"/.nofollow/");
    }
    return EXIT_SUCCESS;
}

You'll get this output:

C: 89 Rep: /Applications/
C: 89 Rep: /.nofollow/Applications/
C: 25 Rep: /
C: 0 Rep: /.nofollow/

That last return is simply wrong, as it should return the same value as "/":

C: 25 Rep: /.nofollow/

That's the core bug that we need to fix.

But when I change that bookmark to a security-scoped URL, it resolves to a URL of "file:///.nofollow/".

Security scope is relevant here because resolving the bookmark is both "security sensitive" and involves IPC, both of which are what led to a "/.nofollow/" path being returned. However, that system is full of cases like this (for example, "NSOpenPanel" meets that same basic criteria), which is why I noted that I'd expect these URLs to become more common.

I understand that you've said the bug is fixed in 26.2.

To be clear, our bookmark infrastructure is working around the issue in fileSystemRepresentation. I don't know when the underlying Foundation issue will be fixed (r.162781619).

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

System-wide....Nothing really limits the scope

My question is more about the fundamental nature of URL paths. Should we be treating these ".nofollow" components like "." components?

For example, in most cases, we don't have to worry much about "." and "..". The system handle them fines, they can be harmlessly standardized away, and they don't occur in the context of NSOpenPanel. They're only an issue when URLs are provided by the user. But we do have to worry about ".nofollow" components now? What do we do with them?

Should we be removing that (those, plural?) ".nofollow" component manually? Or will that break something?

I'm using an NSToken field to show the user their current location in the file system. I don't want to show ".nofollow" to the user.

you probably won't see them with fileReferenceURLs, but that's actually because the fileReferenceURL isn't storing a path at all.

That's an interesting idea. However, I'm not too confident in the future of fileReferenceURLs. They aren't even supported in Swift. I use them for one neat little trick, but I have to do it in Objective-C.

I suggested testing in Terminal as an easy way to understand how the core system behaves, not to replicate your app’s specific behavior.

I wasn't talking about my app's specific behaviour, but, rather, the behaviour of the sandboxed app environment, both when the user has provided access to a given URL and when not. In an earlier app, I wrote a tool just so I could execute system tools from the sandbox to see if they would work or not. Of course, in most cases, those tools aren't "API", so I shouldn't be doing that. Fair enough. But official APIs are really no different. How official APIs are documented to behave and how they work in the sandbox are two different things.

And that's one of my primary concerns there. You keep talking in terms of Terminal and Foundation. But in the sandbox, "file:///" and "file:///.nofollow/" are NOT the same thing. I don't even have to talk in theoretical terms about it possibly breaking one day. It's broken right now.

If the system resolves a security-scoped URL to "file:///.nofollow/", and I do some UI manipulation to avoid confusing the user, and at some point request "file:///". Will that break? You're talking about Foundation dealing with these paths. But I'm talking about the sandbox. Maybe Foundation does it right but then the sandbox blocks it.

You mentioned IPC. That's yet another potential point of breakage. I had to give my launch agent a user-facing entitlement just so I could resolve a security-scoped bookmark.

If there is going to be some new type of "special" component to URLs, then it needs to be fully documented. When will developers encounter this? How can they test it? Can it be safely removed?

That's really my question here. Can I just treat ".nofollow" like "." and strip it out? That would solve a lot of problems.

Unfortunately, users of my different apps also started reporting another issue that could be related: paths on an external volume, such as one mounted by "NTFS for Mac", or a path on a Synology volume, are "converted" to / (Macintosh HD) after going through laundry. Could someone at Apple confirm whether this is also a known issue?

I'm not sure. There was an issue (fixed in macOS 26.1*) that prevented SMB volumes from remounting during bookmark resolution, however:

• I believe resolution simply failed; it didn't lead back to "/".

• The issue effected remounting the volume, not the full resolution.

*Note that the bookmark will need to be recreated.

That also wouldn't explain "NTFS for Mac", as that's a local volume mount which presents very different issues.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

I'm not sure.

Luckily one of those users is a programmer and already ran a test project for me, confirming that the issue is related to bookmark resolution. Is there anything I can ask them to do to understand this issue better? I opened TSI 16931637 in case it helps.