Security scoped bookmarks not valid anymore in macOS 14.7.5 / 13.7.5

Security scoped bookmarks that were created before updating to macOS 14.7.5 cannot be resolved anymore after updating to macOS 14.7.5.

Have you filed a bug on this? If so, what is the bug number?

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Thanks for your reply, here's the report number: FB17100576 If you need additional info please let me know.

Unrelated to the issue: I'd like to express my frustration with the reporting process. I have reported several issues using Feedback Reporter in the past, but almost none of them get an acknowledgement or any reply. Some issues have been fixed, but the reports are still open without any interaction. So I no longer feel like reporting issues, as it appears the work I put into it is not valued. Why bother?

In the past personal TSIs were available and very helpful. Now when I try to contact support directly I'm referred to the dev forums to report it, such that other devs can read it. So I report here publicly and I'm again referred to the private Feedback Assistant, where I have to copy and paste what I've already written above. It's an incredibly frustrating experience. I've reported issues to other companies and the difference is night and day, offering personal replies and confirmations. Not sure if I'm the only one dissatisfied, but just wanted to give some feedback. Hope it can be improved.

Thanks for your reply, here's the report number: FB17100576 If you need additional info please let me know.

Looking at the log data, I think this is the failure you're talking about, though I can't be certain of this from the log alone:

2025-04-03 10:42:25.009851+0200    <app_name>-beta: (libxpc.dylib) [com.apple.xpc:connection] [0x600001704ff0] activating connection: mach=true listener=false peer=false name=com.apple.scopedbookmarksagent.xpc
2025-04-03 10:42:25.010050+0200    ScopedBookmarkAgent: (libxpc.dylib) [com.apple.xpc:connection] [0x15960a2c0] activating connection: mach=false listener=false peer=true name=com.apple.scopedbookmarksagent.xpc.peer[999].0x15960a2c0
2025-04-03 10:42:25.010940+0200    ScopedBookmarkAgent: (Security) SecTrustEvaluateIfNecessary
...
2025-04-03 10:42:25.012126+0200    kernel: (Sandbox) Sandbox: ScopedBookmarkAgent(721) deny(1) file-read-data /Library/Preferences/com.apple.security.plist
2025-04-03 10:42:25.012140+0200    ScopedBookmarkAgent: (Security) [com.apple.securityd:security_exception] UNIX error exception: 5
2025-04-03 10:42:25.012215+0200    ScopedBookmarkAgent: (Security) [com.apple.securityd:security_exception] UNIX error exception: 5
2025-04-03 10:42:25.012270+0200    ScopedBookmarkAgent: (Security) [com.apple.securityd:security_exception] UNIX error exception: 5
...
2025-04-03 10:42:25.013671+0200    ScopedBookmarkAgent: (Security) SecTrustEvaluateIfNecessary
2025-04-03 10:42:25.017631+0200    <app_name>-beta: (libxpc.dylib) [com.apple.xpc:connection] [0x600001704ff0] invalidated after the last release of the connection object
 

WHY this is happening is a question I can't answer. My read of it's sandbox profile is that it should be allowed access and, more to the point, I haven't found any change between 14.7.4 and 14.7.5 that would lead to that. I've raised the issue the engineer to how maintinas ScopedBookmarkAgent and I'll let you know if that turns anything up.

One suggestion on the mitigation side- what happens if you try resolving the bookmark without security scope? You won't be able to access the file through that URL, but if you can resolve the bookmark to a URL, you could then use that URL to point and open/save panel at the right location so the user can restore access.

Unrelated to the issue: I'd like to express my frustration with the reporting process. I have reported several issues using Feedback Reporter in the past, but almost none of them get an acknowledgement or any reply. Some issues have been fixed, but the reports are still open without any interaction. So I no longer feel like reporting issues, as it appears the work I put into it is not valued. Why bother?

So, let me first say that I absolutely understand where that frustration comes from. We obviously take secrecy very seriously and that makes it very hard to have a process that provides the kind of detail you'd really want. As you might imagine the volume of bugs is quite large, which also leaves plenty of room for mistakes. Lastly, scheduling and development time often means that it can take a year or more for an enhancement or fix to ship, which obviously doesn't feel all that responsive. Nonetheless, filing bugs IS the best tool you have for pushing information into Apple.

I'll also say that in the context of the developer forums, I ask for bugs to be filed because:

1. It's a convenient way to collect the data I need to look into an issue further, particularly large files (like sysdiagnose logs).

2. I'm going to move that bug directly to the engineering team so that they can help me figure out what's going on.

Both of those things have already happened here.

Finally, explaining a bit about what's going on here:

In the past personal TSIs were available and very helpful. Now when I try to contact support directly I'm referred to the dev forums to report it, such that other devs can read it. So I report here publicly and I'm again referred to the private Feedback Assistant, where I have to copy and paste what I've already written above.

So, the first thing I'll say here is that my role in DTS hasn't really changed. My goal is to help you make great software "now". While bugs are often part of that process (typically, because the problem your facing IS "a bug"), my primary focus is still on helping you with your product. Bugs are the reverse of that- they're the tool we use to improve our products. If you have an issue with our products, please file a bug. If you want help with your product, then you should talk to DTS.

However, what has changed is that we've made a very deliberate effort to make the work DTS does more public by focusing much more on the developer forums instead of TSIs. That's why you were redirected to the forums- not to avoid the conversation, but to make the conversation public so that other developers could benefit from it.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Thank you for looking into it, very much appreciated. I'm not sure if the errors from the logs you posted are related to the underlying issue. Those same error/exception logs always show up when resolving the first bookmark after app launch, regardless of whether that bookmark resolves/fails. And they also show up on macOS 15.

Here are the debug logs for a bookmark that was created before updating to 14.7.5:

default	11:20:08.512031+0200	<app-name>	Resolving bookmark: <private>
debug	11:20:08.512159+0200	<app-name>	bookmarkRef=<0x600001b2c660 <private>>  options=0x400 relativeToURL=(null) propertiesToInclude=(null) repIsStaleP=0x16db12a58 errorRefP=0x16db12898
debug	11:20:08.512214+0200	<app-name>	Client beginning scoped bookmark resolution.
debug	11:20:08.512233+0200	<app-name>	Resolution request message is prepared and about to be sent to the agent.
debug	11:20:08.512612+0200	<app-name>	0x6000036935c0 rule ^Contents$ added (weight 0, flags 0x12)
debug	11:20:08.512650+0200	<app-name>	0x6000036937a0 rule ^(\.LSOverride|\.DS_Store|Icon
|\.SoftwareDepot\.tracking)$ added (weight 0, flags 0x9)
debug	11:20:08.512666+0200	<app-name>	entering /Users/user/Desktop/<app-name>.app
debug	11:20:08.512703+0200	<app-name>	entering /Users/user/Desktop/<app-name>.app/Contents
debug	11:20:08.512713+0200	<app-name>	leaving /Users/user/Desktop/<app-name>.app/Contents
debug	11:20:08.512721+0200	<app-name>	leaving /Users/user/Desktop/<app-name>.app
debug	11:20:08.512881+0200	<app-name>	open(/Users/user/Desktop/<app-name>.app/Contents/MacOS/<app-name>,0x0,0x1b6) = 34
debug	11:20:08.512919+0200	<app-name>	open(/Users/user/Desktop/<app-name>.app/Contents/MacOS/<app-name>,0x0,0x1b6) = 35
debug	11:20:08.512933+0200	<app-name>	0x60000369a620 is a thin file (arm64)
debug	11:20:08.512943+0200	<app-name>	close(34) err: 0
debug	11:20:08.513209+0200	<app-name>	77006 signing bytes in 5 blob(s) from /Users/user/Desktop/<app-name>.app/Contents/MacOS/<app-name>(arm64)
debug	11:20:08.513361+0200	<app-name>	failed to fetch /Users/user/Desktop/<app-name>.app/Contents/_CodeSignature/CodeRequirements-1 error=-10
debug	11:20:08.513381+0200	<app-name>	SecStaticCode network default: YES
debug	11:20:08.513437+0200	<app-name>	close(35) err: 0
default	11:20:08.513464+0200	<app-name>	[0x600002a93930] activating connection: mach=true listener=false peer=false name=com.apple.scopedbookmarksagent.xpc
debug	11:20:08.513870+0200	ScopedBookmarkAgent	(peer pid=1119) new peer connection.
default	11:20:08.513921+0200	ScopedBookmarkAgent	[0x14d005080] activating connection: mach=false listener=false peer=true name=com.apple.scopedbookmarksagent.xpc.peer[1119].0x14d005080
debug	11:20:08.513986+0200	ScopedBookmarkAgent	(peer pid=1119) peer sent message. message_type=reso
debug	11:20:08.514010+0200	ScopedBookmarkAgent	Handling resolution of a scoped bookmark.
debug	11:20:08.514027+0200	ScopedBookmarkAgent	Collection file descriptor missing or unresolvable, inferring app scope.
debug	11:20:08.514072+0200	ScopedBookmarkAgent	bookmarkRef=<0x14d106ad0 <private>>  options=0x8000400 relativeToURL=<private> propertiesToInclude=(null) repIsStaleP=0x16bd09420 errorRefP=0x16bd092d8
debug	11:20:08.514137+0200	ScopedBookmarkAgent	CFURLCreateByResolvingBookmarkData failed with error [Error Domain=NSCocoaErrorDomain Code=259 "app-scope or collection-scope key doesn't match the key in the bookmark" UserInfo={NSDebugDescription=app-scope or collection-scope key doesn't match the key in the bookmark}].
debug	11:20:08.514167+0200	ScopedBookmarkAgent	(peer pid=1119) sent reply to peer.
debug	11:20:08.514213+0200	<app-name>	Received agent reply.
debug	11:20:08.514225+0200	<app-name>	No resolved path present in agent reply, aborting.
default	11:20:08.514254+0200	<app-name>	[0x600002a93930] invalidated after the last release of the connection object
default	11:20:08.514313+0200	ScopedBookmarkAgent	[0x14d005080] invalidated after getting a no-senders notification - client is gone
debug	11:20:08.514594+0200	<app-name>	Bundle: <private>, key: <private>, value: (null), table: FoundationErrors, localizationNames: (null), result: The file couldn’t be opened because it isn’t in the correct format.
debug	11:20:08.514767+0200	<app-name>	Bundle: <private>, key: <private>, value: (null), table: FoundationErrors, localizationNames: (null), result: The file couldn’t be opened because it isn’t in the correct format.
debug	11:20:08.516128+0200	<app-name>	Bundle: <private>, key: <private>, value: (null), table: FoundationErrors, localizationNames: (null), result: The file couldn’t be opened because it isn’t in the correct format.
default	11:20:08.516151+0200	<app-name>	Resolving bookmark fail: Error Domain=NSCocoaErrorDomain Code=259 --- <private>

This seems related to r.140342863 — Has the app scope key changed in 14.7.5? The key from the old bookmark does not seem valid anymore.

Logs for a new bookmark that succeeds, and the raw bookmark data in case it's useful are available here: https://gist.github.com/floorish/37f972c5411fa61025fbfbe114a84b35

I haven't tried resolving without security scope, but that is not useful in this case since the app already knows the path(s) which are stored separately. The app needs read/write access to these locations and the user may have imported multiple locations. It is possible to restore access by using the open panel again, but that is exactly what I want to prevent. Bookmark data is supposed to give persistent access to a user opened location, isn't it?

--

Regarding the issue reporting process: I'll keep this short since I don't want to lose focus from the original issue. I understand it and I just wanted to give my feedback. I really do appreciate you being here and actively interacting with posts, in particular this one, that's much appreciated. But it's hit or miss, some posts are just ignored, either here or in Feedback Reporter.

This seems related to r.140342863 — Has the app scope key changed in 14.7.5?

Sort of. The issues we had with bookmarks in the 14.7.x range were basically caused by issue with ScopedBookmarkAgent accessing the keys it's relied on. The "final" set of fixes we ended up with were designed to ensure that no matter what happened, ScopedBookmarkAgent would be able to self repair such that it would work correctly from that point "forward".

The key from the old bookmark does not seem valid anymore.

Yes, that is possible. The self repair logic could potentially force a key reset (which would break existing bookmarks), however, a one time failure that self correct is better than staying "stuck" in the broken state.

It is possible to restore access by using the open panel again, but that is exactly what I want to prevent. Bookmark data is supposed to give persistent access to a user opened location, isn't it?

Yes and no. The goal is obviously to provide persistent access, but that isn't something the system can actually guarantee. Setting bugs aside (and, yes, this issue is either a direct bug or a side effect of fixing our older issues), it's always going to be possible for the user to modify their configuration/files such that "logical file system" (meaning, the file hierarchy the user interacts with) looks "exactly" the same but the "physical file system" (meaning, the fs metadata that software actually relies on) is totally different.

That's issue is why the "isStale" argument in bookmark resolution exists at all. What "isStale" actually means is that there was enough data that the system was able to find the target file, but that the file object being returned doesn't have EXACTLY the same metadata configuration as the bookmark.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Thanks for the info!

It's not entirely clear to me when exactly this issue can occur, so I have a few more questions if you don't mind.

I have not received any bookmark related issue reports from users updating to 14.7.3 and 14.7.4. That could be a coincidence, but I assumed that the ScopedBookmarkAgent worked correctly since 14.7.3. I've also updated a VM from 14.4.0 to 14.7.5 and the app scope key was changed, so the self repair was apparently triggered.

1. Does this mean that the underlying issue (or some other issue) is back again in 14.7.5? Or was it never really fixed?

2. Is self repair considered an expected flow or is it a last-resort fallback mechanism in case there is still an unexpected bug? In other words: since self repair appears to be necessary for some users in 14.7.5, does that mean there is an unexpected issue that needs to be fixed?

3. Is it expected that the self repair may be necessary again for users when updating to future macOS 14.*.* versions?

4. Does the same apply for macOS 15.1+ ?

If self repair and therefore a key reset is expected then I have to seriously consider not using the App Sandbox. Because it will be very inconvenient if users have to potentially open their locations manually again on each macOS update.

--

I'm aware of the "isStale" mechanism, but those bookmarks are automatically recoverable by an app without user interaction. So I still consider that "persistent" access — it's actually a really nice feature.

I have not received any bookmark related issue reports from users updating to 14.7.3 and 14.7.4. That could be a coincidence, but I assumed that the ScopedBookmarkAgent worked correctly since 14.7.3.

So, trying to clarify a somewhat muddled situation:

• There was a scoped bookmark related issue that was originally identified in macOS 15.0. It was initially fixed in 15.1, but that initial fix created issues. Those issues were fixed in 15.2.

• That fix was also included in security updates to 13.7 and 14.7, and that fix also had issues. Those issues were "fully" resolved in 13.7.3.

• In both of the cases above, part of that "full" resolution was improving ScopedBookmarkAgent's internal logic with the goal being that it should be able to "self repair" no matter what.

Expanding on that last point, the issue in the earlier bugs wasn't just that ScopedBookmarkAgent could resolve bookmarks, it was that once it got in that state it you couldn't get it working again. Because of that, the final goal wasn't simply to deal with the specific scenario at hand, but also to make sure it would "sort itself out" no matter how/what state it ended up in.

Moving to your questions:

1. Does this mean that the underlying issue (or some other issue) is back again in 14.7.5? Or was it never really fixed

No. There are more details below, but the behavior you're seeing is not the same as the original issues (so it hasn't "returned") and is somewhat expected.

1. Is self repair considered an expected flow or is it a last-resort fallback mechanism in case there is still an unexpected bug? In other words: since self repair appears to be necessary for some users in 14.7.5, does that mean there is an unexpected issue that needs to be fixed?

First off, thanks to some help from the engineering team, let me provide a bit more details about what's going on here. Part of addressing theses security issues involved moving the scoped bookmark key from the file keychain (older) to the more secure data protection keychain (newer). As part of that fix, to provide a smoother transition, the existing key was copied from the file keychain to the data protection keychain.

However, that copy code could not stay present indefinitely, since it could potentially allow a new scoping key to be "injected" into the data protection keychain and that key replacement was exactly what we were trying to prevent. What all that means is:

1. When specifically updating from 14.7.4-> 14.7.5 (as well as other version), the transition I mentioned above should have already happened and so no key change will occur.

2. When updating from other versions, the key will not be present in the data protection keychain, which will cause an expected key change.

3. Per #1, that change should not occur on a system coming from 14.7.4. If you're able to consistently reproduce the problem then that's worth looking at more closely. We obviously can't do anything about transitioning from 14.7.4-> 14.7.5, but it would be nice to improve 14.7.4-> 14.7.5+ and we DEFINATELY want to eliminate any possibility of "another" transition (14.7.5-> 14.7.x).

Is it expected that the self repair may be necessary again for users when updating to future macOS 14.. versions?

No, it's not expected and it certainly shouldn't be happening with every software update.

Having said that, moving to the data protection does bind the bookmark to the machine more closely, which does create more scenarios which could force you to recreate the keychain.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Thanks for taking the time to explain in more detail.

So if I understand correctly 14.7.4 contains the key copy code, but 14.7.5 does not anymore. Therefore if a user skips over 14.7.4 they'll (potentially or always?) get a new key.

If a user is on 14.7.4 or above the key is already in the correct keychain, so no key change is expected anymore.

So far I've received issue reports after the following updates:

• 13.7.4 -> 13.7.5
• 13.7.4 -> 14.7.5
• 14.7.* -> 14.7.5 (not certain if previous version was 14.7.4 or below)

If I assume that 13.7.4 includes the key copy code similarly to 14.7.4, then those reports coming from 13.7.4 are a bit unexpected since the existing key should've been copied.

But to be honest I've received fewer reports than I initially expected, so that is a relief.

I was pretty reassured until that :) Should I be worried about this? What would be a scenario that would trigger that?

When this happens I assume that existing bookmarks are not valid anymore. So allowing the user to reopen/recreate the bookmarks would be the only solution?

If I assume that 13.7.4 includes the key copy code similarly to 14.7.4, then those reports coming from 13.7.4 are a bit unexpected since the existing key should've been copied.

Correct. I should have been more explicit on that, but all the version numbers make this a pain to write out. So, yes, these cases failing:

13.7.4 -> 13.7.5

13.7.4 -> 14.7.5

...is unexpected.

But to be honest I've received fewer reports than I initially expected, so that is a relief.

As I noted before, if you're able to reliable reproduce the issue going from *.7.4-> *.7.5, then that's something we'd definitely like to understand. However, I also expect the volume to relatively low and that the issue will self correct.

I was pretty reassured until that :) Should I be worried about this? What would be a scenario that would trigger that?

The main issue here is that the data protection keychain is "bound" to source machine in a way that the legacy keychain is not, which makes it harder to move the key data between machines. That's an issue the team is already thinking about but you'd already have the same problem today if you manually transferred data and did not move the keychain.

Having said that, I think the best approach here is to provide a good experience for resolving broken bookmarks. That's going to happen to some users no matter what (even without security scope), so improving that experience always has value even if this is never an issue again.

Exactly what that looks like depends on the specifics of your app, but it includes things like:

• Providing a UI option to search directories for "your" files instead of forcing indvidual files selection.

• Providing the option to select directories for "your" files instead of just individual files.

• Creating parallel standard bookmark (not just security scoped) so that you can attempt to find the file if scoped access fails.

The overall idea here is that the worst experience is forcing the user to correct files one by one and you may be able to avoid that by thinking again.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Understood. It looks like the data protection keychain should transfer when it's synced by iCloud, but not if it's local only — is that correct?

Thanks again for sharing your knowledge.

Understood. It looks like the data protection keychain should transfer when it's synced by iCloud, but not if it's local only — is that correct?

Correct, however, we also can't simply sync the current scoping key as that would end up breaking the user other machines. This is an area we're actively working on.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

OK that might indeed cause some confusion for users. Hopefully that can be improved in a future update.

Thanks, appreciate the support!