When creating a permanent key with SecAccessControlCreateWithFlags, which keychain is it saved in?

Answered #2 myself -- No because there is an ID appended which seems to be based on your cert/codesigning. Reference information here and here

Still wondering about #1 though

Your reference to keychain files suggest that you’re thinking about this in terms of the legacy file-based keychain on macOS. Before we go further, I recommend that you read TN3137 On Mac keychain APIs and implementations, which provides a lot of the background to this issue.

where (which keychain, which file) do these keys actually get saved?

SE-protected keys are only supported in the data protection keychain.

I realize there in a keychain somewhere, however they are invisible to the user via Keychain Access and the security CLI.

I’m going to quote from TN3137 here:

Keychain Access displays all keychain items in file-based keychains but only password items in the data protection keychain.

and:

The keychain support in the security command-line tool is primarily focused on the file-based keychain.

Where do these keys actually live … ?

In the data protection keychain.

what mechanism/rules are hiding them from the user?

When Apple updated Keychain Access to show the data protection, we made an policy decision keychain to only show password items. For better or worse. If you disagree with that choice, feel free to file a bug against Keychain Access. And please post your bug number, just for the record.

As a user, how can I inspect this claim if the key metadata isn't available to me?

There’s no good way to do this. Just like an iOS (-:

If you’re feeling particularly motivated you can poke around within the data protection keychain SQLite database in ~/Library/Keychains, but that’s not something we support.

Can any app that knows the kSecAttrApplicationTag use the key …

No. Access to data protection keychain items is determined by the item’s keychain access group. The ability to use a keychain access group is gated by entitlements, as discussed in Sharing Access to Keychain Items Among a Collection of Apps. These are restricted entitlements, as defined in TN3125 Inside Code Signing: Provisioning Profiles, and so must be authorised by a provisioning profile. Apple’s policies for creating profiles mean that, at most, only programs from the same team can share keychain access groups.

This assumes you have SIP enabled. Disabling SIP lets you set boot arguments that nobble a lot of this security. Don’t disable SIP on machines that you care about.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Thank you for the thorough, excellent response @eskimo.

I do think the policy decision that was made actually hurts security. I get it, it's a lot of "stuff" that most users wouldn't have the first clue how to understand. But it's extremely troubling that there is no practical way to audit how programs are using keychain and what ACLs they have put on keys.

Don't even get me started on the fact that SecKeys can be stored as key objects, but CryptoKit made the decision to YOLO it and have devs store them on their own as Generic Password :(

there is no practical way to audit how programs are using keychain and what ACLs they have put on keys.

You started this thread asking about APIs and now you’ve switch to auditing user behaviour. These are two different things, and it’s not uncommon for Apple to provide management tools that do things that you can’t do using APIs. If your goal is to audit user behaviour, I encourage you to file an enhancement request with the specific details of that goal.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

@eskimo, I am referring to application behaviour. How an app makes keys, how many keys, what ACLs are on them. That is not user behaviour, unless you count the developer as the "user" in this case--but I think in the traditional sense the "user" is an end user of the phone/laptop. I'm referring to potential insecure use or misuse of API, knowing that it's impractical for the end user to ever understand what's happening.

I filed a request, but it's a bit like shouting into the void right? I hope I am wrong.

Sure, but you’re still referring to someone in a system admin role auditing the behaviour of code that they don’t control (apps from other third-parties as used by their users).

I filed a request

What was the bug number?

but it's a bit like shouting into the void right?

It depends. When filing enhancement requests specifically, my view is that your bug report is the first step in convincing Apple that adding the feature you want is in their best interest.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"