Original Problem
We use codesign and notarytool in a scripted environment to build and distribute binaries daily. We also do manual builds by logging into the build server using SSH. This has been working for many years, but after updating to a new "Developer ID Application" certificate, codesign was failing with errSecInternalComponent and the console logs showed errSecInteractionNotAllowed.
Summary of Resolution
Attempting to fix the problem resulted in multiple copies of the same Certificate which were NOT shown by Keychain Access. I had to run security delete-identity multiple times to clear out the redundant Identities and then imported the certificate using the security CLI tool.
Details
I originally followed these instructions for requesting and installing a new certificate:
https://developer.apple.com/help/account/certificates/create-developer-id-certificates/
Tip: Use the security tool intead
These instructions fail to mention two critical points: 1) they assume the machine you generate the request on is the same machine you will be using to perform signatures, and 2) KeyChain Access does not allow you to set permissions for applications like codesign. I made the mistake of following the instructions on my workstation, and then tried to import the certificate to the build machine by double clicking on the .cer file.
When that did not work, I followed various forum suggestions and eventually realized I need to export the private key as a .p12 file from the workstation, and import it into the build machine.
Tip: The term "Certificate" often refers to a public certificate by itself, while "Identity" to refers to the combination of a public certificate and private key.
At this point, I could use codesign, but only within Terminal.app while logged into the build machine's console. I tried various security commands to reimport the Identity, set a key partition list, and unlock the keychain, but none of them allowed codesign to work from within SSH or cron scripts.
Eventually I stumbled upon this:
sudo security find-identity -v
Password:
1) 3C255…1560 "Developer ID Application: Data Expedition, Inc. (VK…8X)"
2) 3C255…1560 "Developer ID Application: Data Expedition, Inc. (VK…8X)"
3) 3C255…1560 "Developer ID Application: Data Expedition, Inc. (VK…8X)"
4) EA377…96DD "Developer ID Application: Data Expedition, Inc. (VK…8X)"
5) 3C255…1560 "Developer ID Application: Data Expedition, Inc. (VK…8X)"
6) 3C255…1560 "Developer ID Application: Data Expedition, Inc. (VK…8X)"
7) 3C255…1560 "Developer ID Application: Data Expedition, Inc. (VK…8X)"
8) 3C255…1560 "Developer ID Application: Data Expedition, Inc. (VK…8X)"
9) 3C255…1560 "Developer ID Application: Data Expedition, Inc. (VK…8X)"
10) 3C255…1560 "Developer ID Application: Data Expedition, Inc. (VK…8X)"
10 valid identities found
Keychain Access only showed one copy of the Identity in each keychain, but with security I could see there were actually 9.
Tip: Keychain Access does not accurately display keychain contents. If it shows no contents at all, type a letter in the search box. Identities are distinguished from lone Certificates by a drop-down caret to the left of the certificate name. Clicking that shows the key.
To fix the redundant Identities, I had to run this command four times to delete the nine copies:
security delete-identity -Z 3C255…1560
I repeated this until the identity (I used the SHA1 hash of the certificate) no longer showed up in security find-identity -v.
I then re-imported the certificate and key using security import, which is what I should have done from the begininng.
The Correct Way
Here are the commands I used to get things going after I deleted all the problem certificates:
security import mycertificate.cer -k /Library/Keychains/System.keychain -T /usr/bin/codesign
This next command I ran in Terminal.app on the console so it could display a password prompt:
security import ImportThisKey.p12 -k /Library/Keychains/System.keychain -T /usr/bin/codesign
After this, I used security find-identity -v to verify that there was only one copy of the Identity. I then verified that codesign could be used from SSH and cron-scripts even while logged out of the console.
I suspect that a lot of mysterious certificate problems might be caused by duplicate certificates, each with different permissions. As far as I can tell, there is no way to uniquely identify a certificate/identity or the permissions attached to them. The system just searches based on hash, or team-id, or other non-unique property and seems to just arbitrarily pick one.
I hope this helps someone else stuck with errSecInternalComponent errors!