App group broken on Sequoia

The macOS 15 development cycle triggered a bunch of changes in app groups. See App Groups: macOS vs iOS: Working Towards Harmony for the backstory.

Going forward, I recommend that you:

1. Decide on the app group ID you want to use. This can use either the macOS style or iOS style. For new code I recommend the iOS style, although if you have an exist product that uses the macOS style it usually makes sense to stick with that.
2. For each program involved (so your app and your file provider appex)…
3. Make sure it has an explicit App ID.
4. Create a profile that authorises that App ID to access your app group.
5. Make sure that profile gets embedded in the program’s bundle.
6. Sign the program to claim access to that app group.

IMPORTANT Make sure to sign the program with com.apple.application-identifier so that the system can associated the program with its provisioning profile.

If you’re using Xcode, it will typically take care of this for you, although my experience is that this works better if you’re using an iOS style app group ID.

Will such a change break the app on pre-Sequoia machines?

I think this will back deploy just fine. Prior to macOS 15 the system doesn’t strictly [1] enforce app group ID claims [2] but, even if it did, the presence of a provisioning profile that authorises the claim should be sufficient.

But, as always, I recommend that you test on your each major macOS release that you support, just to be sure.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] I’m using “strictly” here because of the entitlement-validated flag, as explained in App Groups: macOS vs iOS: Working Towards Harmony.

[2] Although they were checked for apps submitted to the App Store [3].

[3] Sometimes |-: There’s a whole different backstory to that.

OK, I've successfully changed the app group to iOS style (group.com.myorg.MyApp)... and it hasn't fixed the problem. It's working on older versions, but still failing on Sequoia.

To confirm:

• Both the main app and extension have an explicit app ID (com.myorg.MyApp and com.myorg.MyApp.EMPFileProvider, respectively).
• I have added an App Group identifier on the website for group.com.myorg.MyApp -- the website won't let me add one in the old macOS format (I understand this is normal).
• On the website, in the Identifier entry for each component, I've added group.com.myorg.MyApp to the existing App Group set. This has required me to re-generate the provisioning profiles.
• On the Profiles page, I regenerated the profiles (by choosing Edit and then Save rather than deleting and re-creating them). I downloaded and installed the new profiles.
• The resulting app works fine on a pre-Sequoia machine. But on the Sequoia one, the main app runs successfully (and can log to the Group Container), and the FileProvider also runs, but the FileProvider can not write its logs, read its prefs, or access the connecting pipe.

We still get the following errors in the Console log (filtered on "fileprovider”), which indicate that even though it’s got the appropriate application-groups entitlement, it’s not able to access the prefs or create the log files:

default	16:26:01.341453-0700	EMPFileProvider	container_create_or_lookup_app_group_path_by_app_group_identifier: success
error	16:26:01.342562-0700	EMPFileProvider	Couldn't read values in CFPrefsPlistSource<0x7f83db71aab0> (Domain: group.com.mydomain.MyApp, User: kCFPreferencesAnyUser, ByHost: Yes, Container: (null), Contents Need Refresh: Yes): Using kCFPreferencesAnyUser with a container is only allowed for System Containers, detaching from cfprefsd
error	16:26:01.344858-0700	cfprefsd	rejecting read of { group.com.mydomain.MyApp, mikec, kCFPreferencesAnyHost, /Users/mikec/Library/Group Containers/group.com.mydomain.MyApp/Library/Preferences/group.com.mydomain.MyApp.plist, managed: 0 } from process 1791 (EMPFileProvider) because accessing preferences outside an application's container requires user-preference-read or file-read-data sandbox access
fault	16:26:01.346167-0700	EMPFileProvider	Couldn't read values in CFPrefsPlistSource<0x7f83db7104d0> (Domain: group.com.mydomain.MyApp, User: kCFPreferencesCurrentUser, ByHost: No, Container: (null), Contents Need Refresh: Yes): accessing preferences outside an application's container requires user-preference-read or file-read-data sandbox access
error	16:26:01.347646-0700	cfprefsd	rejecting read of { group.com.mydomain.MyApp, mikec, kCFPreferencesAnyHost, /Users/mikec/Library/Group Containers/group.com.mydomain.MyApp/Library/Preferences/group.com.mydomain.MyApp.plist, managed: 0 } from process 1791 (EMPFileProvider) because accessing preferences outside an application's container requires user-preference-read or file-read-data sandbox access
fault	16:26:01.347939-0700	EMPFileProvider	Couldn't read values in CFPrefsPlistSource<0x7f83db7104d0> (Domain: group.com.mydomain.MyApp, User: kCFPreferencesCurrentUser, ByHost: No, Container: (null), Contents Need Refresh: Yes): accessing preferences outside an application's container requires user-preference-read or file-read-data sandbox access
error	16:26:01.349210-0700	kernel	1 duplicate report for System Policy: EMPFileProvider(1791) deny(1) file-read-data /Users/mikec/Library/Group Containers/group.com.mydomain.MyApp/Library/Preferences/group.com.mydomain.MyApp.plist
error	16:26:01.351654-0700	cfprefsd	rejecting read of { group.com.mydomain.MyApp, mikec, kCFPreferencesAnyHost, /Users/mikec/Library/Group Containers/group.com.mydomain.MyApp/Library/Preferences/group.com.mydomain.MyApp.plist, managed: 0 } from process 1791 (EMPFileProvider) because accessing preferences outside an application's container requires user-preference-read or file-read-data sandbox access
fault	16:26:01.352056-0700	EMPFileProvider	Couldn't read values in CFPrefsPlistSource<0x7f83db7104d0> (Domain: group.com.mydomain.MyApp, User: kCFPreferencesCurrentUser, ByHost: No, Container: (null), Contents Need Refresh: Yes): accessing preferences outside an application's container requires user-preference-read or file-read-data sandbox access
error	16:26:01.357264-0700	kernel	1 duplicate report for System Policy: EMPFileProvider(1791) deny(1) file-write-create /Users/mikec/Library/Group Containers/group.com.mydomain.MyApp/Library/Logs/cloud1791_2.log
error	16:26:01.357269-0700	kernel	System Policy: EMPFileProvider(1791) deny(1) file-write-create /Users/mikec/Library/Group Containers/group.com.mydomain.MyApp/Library/Logs/cloud1791_3.log
default	16:26:01.360168-0700	EMPFileProvider	Extension `/Applications/EMPSecure.app/Contents/PlugIns/EMPFileProvider.appex/Contents/MacOS/EMPFileProvider` of type: `1` launched.

For the record, here are the Entitlements in the embedded provision profile in the main app:

<key>Entitlements</key>
<dict>
			
			<key>com.apple.security.application-groups</key>
	<array>
			<string>group.XXXXXXXXXX.com.mydomain.MyApp</string>
			<string>group.com.mydomain.MyApp</string>
			<string>XXXXXXXXXX.*</string>
	</array>
			
			<key>com.apple.application-identifier</key>
	<string>XXXXXXXXXX.com.mydomain.MyApp</string>
			
			<key>keychain-access-groups</key>
	<array>
			<string>XXXXXXXXXX.*</string>
	</array>
			
			<key>com.apple.developer.team-identifier</key>
	<string>XXXXXXXXXX</string>
	
</dict>

And here are the ones in the FileProvider extension:

<key>Entitlements</key>
<dict>
			
			<key>com.apple.developer.networking.networkextension</key>
	<array>
			<string>packet-tunnel-provider-systemextension</string>
			<string>app-proxy-provider-systemextension</string>
			<string>content-filter-provider-systemextension</string>
			<string>dns-proxy-systemextension</string>
			<string>dns-settings</string>
			<string>relay</string>
			<string>url-filter-provider</string>
			<string>hotspot-provider</string>
	</array>
			
			<key>com.apple.security.application-groups</key>
	<array>
			<string>group.XXXXXXXXXX.com.mydomain.MyApp</string>
			<string>group.com.mydomain.MyApp</string>
			<string>XXXXXXXXXX.*</string>
	</array>
			
			<key>com.apple.application-identifier</key>
	<string>XXXXXXXXXX.com.mydomain.MyApp.EMPFileProvider</string>
			
			<key>keychain-access-groups</key>
	<array>
			<string>XXXXXXXXXX.*</string>
	</array>
			
			<key>com.apple.developer.team-identifier</key>
	<string>XXXXXXXXXX</string>
	
</dict>

Note that there are multiple app groups present -- I didn't remove the old one in Xcode when I added the new one! Surely a failure with that one couldn't block a perfectly valid app group alongside it?

Anything else I should look for in the console logs to check which app groups the extension actually has access to?

I’m not sure what’s going wrong here but I have a couple of diagnostics tests for you to run. Both require you to get your file provider started. Once you have that:

• Use launchctl procinfo to check whether the process has the entitlements validated flag set. App Groups: macOS vs iOS: Working Towards Harmony has an explanation of how to do this.

• Use codesign to check the entitlements of the running process:

  % codesign -d --entitlements - PID
  

  where PID is the process ID.

What do you see?

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Thanks, Quinn! The first problem is that the "entitlements validated" flag is not set -- here's the relevant text:

code signing info = valid
	refuse invalid pages
	kill on invalid pages
	require enforcement
	allowed mach-o
	platform dyld

And the codesign result shows that the new app group is not present, just the old one:

Executable=/Applications/MyApp.app/Contents/PlugIns/EMPFileProvider.appex/Contents/MacOS/EMPFileProvider
[Dict]
	[Key] com.apple.security.app-sandbox
	[Value]
		[Bool] true
	[Key] com.apple.security.network.client
	[Value]
		[Bool] true
	[Key] keychain-access-groups
	[Value]
		[Array]
			[String] XXXXXXXXXX.com.mydomain.MyApp.Shared
	[Key] com.apple.security.application-groups
	[Value]
		[Array]
			[String] group.com.mydomain.MyApp
	[Key] com.apple.application-identifier
	[Value]
		[String] $(AppIdentifierPrefix)$(PRODUCT_BUNDLE_IDENTIFIER)

...which is odd because the embedded.provisionprofile inside the .appex lists all three:

				<key>com.apple.security.application-groups</key>
		<array>
				<string>group.XXXXXXXXXX.com.mydomain.MyApp</string>
				<string>group.com.mydomain.MyApp</string>
				<string>XXXXXXXXXX.*</string>
		</array>

This looks like there's something freaky going on in the signing of the app -- like it's somehow signing against an outdated version of the entitlements file somewhere? I'll try a complete rebuild and get back to you ASAP.

The first problem is that the "entitlements validated" flag is set.

OK. That explains the runtime behaviour you’re seeing, because:

• The system won’t grant you access to the container because it can’t trust your entitlements.
• And it won’t display the standard UI because it knows that file providers run in the background.

And the codesign result shows that the new app group is not present

And that’d do it.

As to how it got built this, it’s hard to say without digging deeper into your build process.

I'll try a complete rebuild and get back to you ASAP.

That’s always a good start. And if that doesn’t help, you can look at the build transcript to see how entitlement values are flowing from your .entitlements file to the codesign command.

Oh, and another option is to create a test project from one of the built-in templates, add an appex target to that, and then get the same app group working in that context. If you start out with a different bundle ID, that’ll tell you whether there’s something generally broken with this setup (which I doubt, but it’s always good to rule out such things). And once you get that working, change the bundle IDs to match your main project. If things continue to work, you know that the bundle IDs and their profiles are set correctly, which puts the focus on the setup of your existing project.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Right. I've done a clean build (with only the one, new app group), the entitlements file is correct, and it runs with the correct entitlements on both my dev machine and a Ventura one. Curiously, though, the "entitlements validated" flag is still not showing as set. I'm waiting to hear back on the results from the Sequoia machine, but I expect that means it's still going to fail.

So in the meantime -- could this be an issue with notarization rather than signing? I've just discovered that because we're distributing the app in-house rather than through the App Store, the installer package for these internal releases hasn't been going through notarytool. Is that a deal-breaker under the newer OS versions?

Right -- think I've had a breakthrough: the entitlements file for the appex shows its app ID as follows:

[Key] com.apple.application-identifier
[Value]
	[String] $(AppIdentifierPrefix)$(PRODUCT_BUNDLE_IDENTIFIER)

while the main app shows a proper value:

[Key] com.apple.application-identifier
[Value]
	[String] XXXXXXXXXX.com.mydomain.MyApp

For some reason, the Xcode variables weren't being substituted in the FileProvider!

The surprise here is simply that the app worked for so long without the entitlements being validated...

Anyway. I've confirmed that the app now presents the "entitlements validated" flag correctly on our older machines; once I hear back from the employee running this on Sequoia that it works now, I'll accept the answer here. Thanks!

It sounds like you’re making great progress.

The surprise here is simply that the app worked

Such is the nature of the Mac. On iOS the trusted execution system would simply block your code from running in this case. That’s much easier to debug. The Mac, with its legacy stretching back 40+ years, has to be more flexible. That has its pros and cons, and you’ve just hit one of the cons )-:

once I hear back from the employee running this on Sequoia …

*cross fingers*

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"