Hi,
I’m implementing a macOS DNS Proxy as a system extension and running into a persistent activation error:
OSSystemExtensionErrorDomain error 9 (validationFailed) with the message: extension category returned error
This happens both on an MDM‑managed Mac and on a completely clean Mac (no MDM, fresh install).
Setup macOS: 15.x (clean machine, no MDM)
Xcode: 16.x
Team ID: AAAAAAA111 (test)
Host app bundle ID: com.example.agent.NetShieldProxy
DNS Proxy system extension bundle ID: com.example.agent.NetShieldProxy.dnsProxy
The DNS Proxy is implemented as a NetworkExtension system extension, not an app extension.
Host app entitlements From codesign -d --entitlements :- /Applications/NetShieldProxy.app:
xml <dict> <key>com.apple.application-identifier</key> <string>AAAAAAA111.com.example.agent.NetShieldProxy</string>
<key>com.apple.developer.system-extension.install</key>
<true/>
<key>com.apple.developer.team-identifier</key>
<string>AAAAAAA111</string>
<key>com.apple.security.app-sandbox</key>
<true/>
<key>com.apple.security.application-groups</key>
<array>
<string>group.com.example.NetShieldmac</string>
</array>
<key>com.apple.security.files.user-selected.read-only</key>
<true/>
</dict> DNS Proxy system extension entitlements From codesign -d --entitlements :- "/Applications/NetShieldProxy.app/Contents/Library/SystemExtensions/com.example.agent.NetShieldProxy.dnsProxy.systemextension/Contents/MacOS/com.example.agent.NetShieldProxy.dnsProxy":
xml <dict> <key>com.apple.application-identifier</key> <string>AAAAAAA111.com.example.agent.NetShieldProxy.dnsProxy</string>
<key>com.apple.developer.networking.networkextension</key>
<array>
<string>dns-proxy-systemextension</string>
</array>
<key>com.apple.developer.team-identifier</key>
<string>AAAAAAA111</string>
<key>com.apple.security.application-groups</key>
<array>
<string>group.com.example.NetShieldmac</string>
<string>group.example.NetShieldmac</string>
<string>group.example.agent.enterprise.macos</string>
<string>group.example.com.NetShieldmac</string>
</array>
</dict> Both host and extension binaries are signed with the same Developer ID Application certificate for Team AAAAAAA111, and codesign -dv --verbose=4 shows matching TeamIdentifier and Authority for both.
DNS Proxy system extension Info.plist On the clean Mac, from:
bash plutil -p "/Applications/NetShieldProxy.app/Contents/Library/SystemExtensions/com.example.agent.NetShieldProxy.dnsProxy.systemextension/Contents/Info.plist" I get:
json { "CFBundleExecutable" => "com.example.agent.NetShieldProxy.dnsProxy", "CFBundleIdentifier" => "com.example.agent.NetShieldProxy.dnsProxy", "CFBundleName" => "com.example.agent.NetShieldProxy.dnsProxy", "CFBundlePackageType" => "SYSX", "CFBundleShortVersionString" => "1.0.1.8", "CFBundleSupportedPlatforms" => [ "MacOSX" ], "CFBundleVersion" => "0.1.1", "LSMinimumSystemVersion" => "13.5", "NSExtension" => { "NSExtensionPointIdentifier" => "com.apple.dns-proxy", "NSExtensionPrincipalClass" => "com_example_agent_NetShieldProxy_dnsProxy.DNSProxyProvider" }, "NSSystemExtensionUsageDescription" => "SYSTEM_EXTENSION_USAGE_DESCRIPTION" } The DNSProxyProvider class inherits from NEDNSProxyProvider and is built in the system extension target.
Activation code In the host app, I use:
swift import SystemExtensions
final class SystemExtensionActivator: NSObject, OSSystemExtensionRequestDelegate { private let extensionIdentifier = "com.example.agent.NetShieldProxy.dnsProxy"
func activate(completion: @escaping (Bool) -> Void) {
let request = OSSystemExtensionRequest.activationRequest(
forExtensionWithIdentifier: extensionIdentifier,
queue: .main
)
request.delegate = self
OSSystemExtensionManager.shared.submitRequest(request)
}
func request(_ request: OSSystemExtensionRequest,
didFailWithError error: Error) {
let nsError = error as NSError
print("Activation failed:", nsError)
}
func request(_ request: OSSystemExtensionRequest,
didFinishWithResult result: OSSystemExtensionRequest.Result) {
print("Result:", result.rawValue)
}
} Runtime behavior on a clean Mac (no MDM) config.plist is created under /Library/Application Support/NetShield (via a root shell script).
A daemon runs, contacts our backend, and writes /Library/Application Support/NetShield/state.plist with a valid dnsToken and other fields.
The app NetShieldProxy.app is installed via a notarized, stapled Developer ID .pkg.
The extension bundle is present at: /Applications/NetShieldProxy.app/Contents/Library/SystemExtensions/com.example.agent.NetShieldProxy.dnsProxy.systemextension.
When I press Activate DNS Proxy in the UI, I see in the unified log:
text NetShieldProxy: [com.example.agent:SystemExtensionActivator] Requesting activation for system extension: com.example.agent.NetShieldProxy.dnsProxy NetShieldProxy: [com.example.agent:SystemExtensionActivator] SystemExtensionActivator - activation failed: extension category returned error (domain=OSSystemExtensionErrorDomain code=9) NetShieldProxy: [com.example.agent:SystemExtensionActivator] SystemExtensionActivator - OSSystemExtensionError code enum: 9 NetShieldProxy: [com.example.agent:SystemExtensionActivator] SystemExtensionActivator - validationFailed And:
bash systemextensionsctl list
-> 0 extension(s)
There is no prompt in Privacy & Security on this clean Mac.
Question Given:
The extension is packaged as a system extension (CFBundlePackageType = SYSX) with NSExtensionPointIdentifier = "com.apple.dns-proxy".
Host and extension share the same Team ID and Developer ID Application cert.
Entitlements on the target machine match the provisioning profile and Apple’s docs for DNS Proxy system extensions (dns-proxy-systemextension).
This is happening on a clean Mac with no MDM profiles at all.
What are the likely reasons for OSSystemExtensionErrorDomain error 9 (validationFailed) with "extension category returned error" in this DNS Proxy system extension scenario?
Is there any additional configuration required for DNS Proxy system extensions (beyond entitlements and Info.plist) that could trigger this category-level validation failure?
Any guidance or examples of a working DNS Proxy system extension configuration (host entitlements + extension Info.plist + entitlements) would be greatly appreciated.
Thanks!
You post is quite hard to read. I recommend that you review the advice in Quinn’s Top Ten DevForums Tips, and specifically the stuff about code blocks in tip 5.
What are the likely reasons for … extension category returned error
I discuss the concept of the extension category in this post. The rest of that thread should have some useful hints and tips.
Taking a step back, you’re clearly testing the direct distribution case here, that is, where the sysex uses Developer ID signing. I’d like to clarify the context for that:
- Do you already have this working with Apple Development signing? That is, this code is working at your desk but fails when you deploy it using Developer ID signing?
- Or is this the first time you’ve tested your code?
If it’s the latter, I recommend that you stop using Developer ID and instead switch to Apple Development. That’s a good idea in general — see The Care and Feeding of Developer ID for more on that — but in the case of an NE sysex it actively makes your day-to-day development easier. I explain this process in some detail in Debugging a Network Extension Provider.
Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"
