-34018 A required entitlement isn't present. from command line swift program and more

[Just when you thought that the keychain API couldn’t get any weirder…]

First up, some background info:

• TN3137 On Mac keychain APIs and implementations explains the difference between the data protection and file-based keychain.

• SecItem: Fundamentals and SecItem: Pitfalls and Best Practices provide lots of important backstory, including more details about the keychain shim on macOS.

• Sharing access to keychain items among a collection of apps explains how the data protection keychain uses entitlements for access control.

• TN3125 Inside Code Signing: Provisioning Profiles talks about restricted entitlements and provisioning profiles.

• SecItem attributes for keys has some info about the semantics of each of the label-ish keychain attributes.

You’ve described two issues here:

• When you add a certificate to the keychain with a custom label, you can’t then find that certificate using that label.

• When you add a keychain to the keychain from your command-line tool, that fails with -34018.

I’ll explain the cause of each in turn, and then offer advice as to how to proceed.

The first issue is one of the many incompatibilities caused by the keychain shim on macOS. When you add an object reference to the keychain using kSecValueRef, the system populates the keychain attributes from the attributes associated with that object. That raises the question of what happens when you also specify attributes via the dictionary. For example, if the object has an internal label and you specify a label yourself using kSecAttrLabel, what do you get?

Annoyingly, the results differ between the file-based keychain, through the shim, and the data protection keychain:

• The shim uses the label attached to the certificate object.

• The data protection keychain uses the label you supply via kSecAttrLabel.

)-:

Hence the behaviour you’re seeing. The certificate ends up in the file-based keychain with a kSecAttrLabel value of testdeletejune27.invisinet.com, so you can’t find it when you query for a certificate with the kSecAttrLabel of com.example.mycert.

The second issues seems to be related to a recent change in SecItemAdd when dealing with keys. In general, SecItemAdd no macOS goes through the shim and thus targets the file-based keychain by default. However, it seems that on modern systems (I’m testing on macOS 14.5) it’s targeting the data protection keychain. So, your key ends up in the data protection keychain, which is all well and good until you try to do this from a command-line tool.

The data protection keychain relies on keychain access groups, and the use of keychain access groups is mediated by entitlements. Those are restricted entitlements, which means that they must be authorised by a provisioning profile. That’s problematic for a command-line tool because there’s no place to store that profile (r. 125850707)

So, when you call SecItemAdd it tries to add the key to the data protection keychain, which fails because you don’t have the entitlements necessary to establish a default keychain access group, and hence the -34018 error, aka errSecMissingEntitlement.

So, what should you do? First up, stop using kSecAttrLabel. Regardless of what else is going on here, that attribute is meant for a user-visible label, not for you the app developer. The correct option is kSecAttrApplicationTag.

Now, that attribute is only supported on keys, not certificates. What to do with certificates is tricky. Most folks who import a certificate and a private key are trying to form a digital identity. Is that what you’re doing here? If so, you don’t need to label the certificate because you can search for a digital identity and then identify both components using the label from the private key component.

If you’re not trying to form a digital identity then things get trickier. In that case, please post back with more details about how you’re tracking certificates.

Regarding the errSecMissingEntitlement error, you need to make a decision about whether you want to target the data protection keychain or the file-based keychain. IMO it’s better to use the former rather than the latter. Quoting TN3137:

The file-based keychain is on the road to deprecation.

If you want to stick with the file-base keychain, the simple fix is to pass false to the kSecUseDataProtectionKeychain on SecItemAdd.

If you want to move forward to the data protection keychain, you need to sign your command-line tool with the necessary entitlements (specifically, the App ID entitlement). This is possible, but a little painful. I wrote up my recommendations in Signing a daemon with a restricted entitlement.

*phew*

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

First of all, thank you for the superbly detailed response! Quinn always delivers.

So, it sounds like the data protection keychain is the way to go and more portable between macOS and iOS and we are implementing on both. I created a test .app to call my code, added appropriate entitlement, and your suggestions work so far, I can add a key with a tag, find it, etc.

For tracking identities we just generate a GUID to store in core data with some other app specific items for each identity.

When we run our app of course we would like to find the cert/key pair using that ID.

Here is what I plan on doing.

1. Add the key with a tag (GUID)
2. Add the certificate
3. When I need to grab the key/cert I will

a) find the key using that tag b) find the cert associated with that key

Of course we would also like to delete the identities when we no longer need them.

I'm not clear on your statement: "identify both components using the label from the private key component."

You mentioned I should stop using labels, so I'm not sure what "label" means in this context.

How do I go about finding the cert once i have the key?

Is there an easy way to identify the key associated with a certificate in the keychain access app?

Is there any advantage to getting the certs to show up in the MyCertificates section I see in keychain app or to have our own keychain?

alf

I'm not sure what "label" means in this context.

Yeah, sorry, I was using that in the general sense, not in the sense of kSecAttrLabel.

How do I go about finding the cert once i have the key?

That ties into my question about about digital identities. Most folks who have a private key and a certificate are actually working with a digital identity. That is, the private key matches the public key in the certificate. Is that the case here? Or are the key and the certificate unrelated?

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Sorry, yes, I am trying to create/manage identities.

On a side note I am concerned about the one to may thing with keys and certificates and how to manage that possibility. Multiple certs for a key may (hopefully) be an error condition for us but I still have to handle it in the code.

Back to the main problem. Using the data protection keychain I can add, find, delete keys no problem using kSecAttrApplicationTag. I was also able to add the cert previously discussed without specifying any tag or label but am not able to find it with the label testdeletejune27.invisinet.com, so maybe that label isn't assigned with the data security keychain? In addition, it seems that certs in this keychain aren't visible with the keychain access app or apparently security find-certificate -a, so it's disconcerting not having another way to verify the state of things. Also, now that my cert is added I have no way of deleting it and I know it's there as I get an already exists error if I try to add it again.

I think I understand now that I need to find the cert from the key and I have an example of that via the AI bots, so can move in that direction. Once I can find the cert, I am still not sure how to delete it, since SecItemDelete needs a query that works.

This example I found seems like it may be a good way to manage these identities. Is this the right approach? what about the one to many issue? I'm not sure we will have control as far as how the customer will generate their certs.:

  var items = [String: Any]()
  
  items[kSecClass as String] = kSecClassIdentity
  items[kSecAttrApplicationTag as String] = tag.data(using: .utf8)
  items[kSecImportExportPassphrase as String] = kCFNull // Optional passphrase
  items[kSecValueRef as String] = SecPKCS12Import(certificateData: certificateData as CFData, keyData: privateKeyData as CFData, options: [:])
  
  let status = SecItemAdd(items as CFDictionary, nil)
  return status
}

func findIdentity(forKeyTag tag: String) -> SecIdentity? {
  var query = [String: Any]()
  query[kSecClass as String] = kSecClassIdentity
  query[kSecAttrApplicationTag as String] = tag.data(using: .utf8)
  query[kSecReturnRef as String] = kCFBooleanTrue!
  query[kSecUseDataProtectionKeychain as String] = true
  
  var result: AnyObject?
  let status = SecItemCopyMatching(query as CFDictionary, &result)
  
  if status == errSecSuccess, let itemRef = result as? SecCFTypeRef {
    return itemRef as? SecIdentity
  } else {
    // Handle error (e.g., no item found)
    print("Error searching keychain: \(status)")
    return nil
  }
}

func deleteIdentity(forKeyTag tag: String) -> OSStatus {
  var query = [String: Any]()
  query[kSecClass as String] = kSecClassIdentity
  query[kSecAttrApplicationTag as String] = tag.data(using: .utf8)
  
  let status = SecItemDelete(query as CFDictionary)
  return status
}

Sorry, missed the first line of that first function: func addIdentityToKeychain(certificateData: Data, privateKeyData: Data, tag: String) -> OSStatus {

Also, would this code work the same on iOS?

I also now realize I didn't notice that example wasn't using the data protection keychain

When I was able to add a key to the data protection keychain, I then tried the following code to look for all certs, but I think it didn't like the kSecUseDataProtectionKeychain attribute there, as I got:

findCertificatesFromKey(key:): 358, certStatus: -25303 The specified attribute does not exist.

code:

func findCertificatesFromKey(key: SecKey) -> [SecCertificate]? {
	let certQuery: [String: Any] = [
		kSecUseDataProtectionKeychain as String: true,
		kSecClass as String: kSecClassCertificate,
		kSecAttrKeyType as String: kSecAttrKeyTypeRSA,  // or other key type as required
		kSecReturnRef as String: true,
		kSecMatchLimit as String: kSecMatchLimitAll
	]

	var certItems: CFTypeRef?
	let certStatus = SecItemCopyMatching(certQuery as CFDictionary, &certItems)

	if certStatus == errSecSuccess {
		if let certArray = certItems as? [SecCertificate] {
			for certificate in certArray {
				// Here you can check if the certificate is associated with the key
				print("Certificate found: \(certificate)")
				// You can use SecCertificateCopyValues to get more info about the certificate
			}
			return certArray
		} else {
			print("No certificates found")
		}
	} else {
		print("\(#function): \(#line), certStatus: \(certStatus) \(SecCopyErrorMessageString(certStatus, nil) as String? ?? "Unknown error")")
	}
	return nil
}

I am trying to create/manage identities.

Thanks for confirming that.

it seems that certs in this keychain aren't visible with the keychain access app or apparently security find-certificate

Yep. Both of those are a common source of confusion, which I call out in the User interface section of TN3137.

I usually work around this by adding a small test UI to my app that lets me do keychain-related stuff, like dump all the items in my keychain access group, delete those items, and so on.

In terms of how you handle multiple certificates for the same key, the trick here is to work in ‘identity space’. If there are multiple certificates for the same key, you’ll get an identity for each one. It’s trivial to get the certificate and key from each identity (SecIdentityCopyCertificate and SecIdentityCopyPrivateKey). You can then check for key equality using CFEquals [1].

Alternatively, if you want to prevent this from happening then you can do that on the import side. When you import your PKCS#12, you get an identity. Add its private key to the keychain first. If it’s a dup, that add will fail, and you know you shouldn’t be adding this digital identity.

-25303 The specified attribute does not exist.

Right. Look at the parameter block you’re passing in there:

let certQuery: [String: Any] = [
    kSecUseDataProtectionKeychain as String: true,
    kSecClass as String: kSecClassCertificate,
    kSecAttrKeyType as String: kSecAttrKeyTypeRSA,
    …
]

You’re querying for a certificate (kSecClass is kSecClassCertificate) and the certificate ‘table’ only has certificate ‘columns’. Thus, you can’t query for a key ‘column’ (kSecAttrKeyType).

Earlier I mentioned my two SecItem posts:

• SecItem: Fundamentals

• SecItem: Pitfalls and Best Practices

I recommend that you go back and have a read of those. The first explains how you can view the keychain as an SQL database, with a table for each item class (kSecClass), columns in that table representing the various attributes of such items, and a row in that table for each item. If you keep that model in your head it’s much easier to understand what’s going on.

Oh, and the second has a bunch of info about digital identity formation. It also explains how to avoid all those annoying as String casts (-:

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] I can’t remember whether the Swift == operator will route through to that. You’ll have to test it.