This is a continuation of from the old forums: https://devforums.apple.com/thread/246122
Calling SecItemCopyMatching will sometimes return an OSStatus of -34018 (errSecMissingEntitlement). This seems to happen when the system is running low on memory. This has not been fixed in iOS9. I've of course filed radars about this and I would encourage others to do the same while iOS 9 is under development.
I can confirm I have another case of this bug in official releases running iOS 9.2 on more than one device. In my case that scares a lot due to we trust on the KeyStore very sensitive and crucial information for the app.
As a software engineer, It strongly draws my attention that this bug is still alive after some years bearing in mind its importance.
Same here. iPhone 6, iOS 9.2 (13C75), using Alamofire and AlamofireImage. I’m working in Swift, with a Keychain struct I created myself.
After three or four retrievals (it’s pretty consistent), SecItemCopyMatching returns -34018. (See Keychain.password(), below; the error hits at the call to SecItemCopyMatching().) Resetting the password during the run of the app does not help; killing and restarting it clears the error for another three or four cycles, which I don’t have to say is unacceptable. The stored password is still there; Keychain.password() returns it as expected. The bug manifests even if no other SecItem* call is made.
(I see I don’t release returnPointer in that func; I’ll correct that in my own copy. I don’t think it should affect the bug, it’s just an 8-byte leak. I’m leaving the fix out of this listing to preserve my example.)
I use the password by embedding a hash in the body of my transactions. (I use POSTs or PUTs, even in cases that would ordinarily use GETs.) I don’t claim this is what an experienced developer would do; I’m just describing the circumstances. All transactions are performed through Alamofire; this sometimes entails a loop of transactions such as the upload of a few images; the password is retrieved only once for each series.
Quinn’s previous suggestion that this may be an entitlements or signing issue doesn’t seem to hold up here:
I’m appending my Keychain struct for thoroughness. I DO NOT recommend it for use in others’ work. I have not qualified it as a product. Developers must be particularly on their guard about security practices, and I have no illusions about mine.
//
// Keychain.swift
// Incident
//
// Created by Fritz Anderson on 11/28/15.
// Copyright © 2015 The University of Chicago. All rights reserved.
//
import Foundation
import Security
import CommonCrypto
func hashPassword(password: String) -> NSData {
let hashLength = 256/8
let stringData = password.dataUsingEncoding(NSUTF8StringEncoding)!
let digestBytes = UnsafeMutablePointer.alloc(hashLength)
CC_SHA256(stringData.bytes, UInt32(stringData.length), digestBytes)
return NSData(bytes: digestBytes, length: hashLength)
}
extension NSData {
var hexString: String {
let contents = UnsafeBufferPointer(
start: UnsafePointer(bytes),
count: length)
return contents.reduce("") {
$0 + String($1, radix: 16)
}
}
}
func credentialsDictionary(user: String? = nil,
host: String? = nil) throws -> [String: AnyObject]
{
let userName = user ?? (DefaultsKeys.AccountName.value() as! String)
var retval: [String: AnyObject] = ["account" : userName]
let hostName = host ?? Globals.HostName.value!
var chain = Keychain(user: userName, host: hostName)
if let password = try chain.password() {
let hash = hashPassword(password).hexString
retval["password"] = hash
}
else {
throw KeychainError.NoPasswordDefined(user: userName, host: hostName)
}
return retval
}
func credentialsJSONData(user: String? = nil,
host: String? = nil) throws -> NSData
{
let dict = try credentialsDictionary(user, host: host)
return try NSJSONSerialization.dataWithJSONObject(dict, options: [])
}
public enum KeychainError : ErrorType {
case Duplicate
case OperationOnMissing
case NotAuthorized
case Parameter
case Generic(osError: Int)
case NoPasswordDefined(user: String, host: String)
/// The `KeychainError` case matching an `OSStatus` value.
///
/// - parameters:
/// - status: the `OSStatus` to match
///
/// - returns: Optional `KeychainError`: `nil` if the code was `noErr`; `.Generic` if it was not covered by any of the other cases; or one of those cases if it matches a recognized code. See the source for this `func` for details.
public static func fromOSStatus(status: OSStatus) -> KeychainError? {
switch status {
case noErr: return nil
case errSecDuplicateItem: return .Duplicate
case errSecItemNotFound: return .OperationOnMissing
case errSecAuthFailed: return .NotAuthorized
case -50: return .Parameter
default: return .Generic(osError: Int(status))
}
}
/// A description of the represented error, tagged with an optional context string.
///
/// The optional context string is prepended to the returned description. Example, without a context:
///
/// "There was no such entry."
///
/// If the context is "changing the password":
///
/// "While changing the password: There was no such entry."
///
/// Notice that the context string is wrapped in "While _ : ". Adjust your phrasing accordingly.
///
/// - parameters:
/// - attempting: `nil` (the default) if there is to be no tag; otherwise a string to insert in the description.
///
/// - returns: A description of the error, including the context string if one was supplied.
public func explain(attempting: String? = nil) -> String {
let circumstance: String
if let attempting = attempting {
circumstance = "While \(attempting): "
}
else { circumstance = "" }
let expansion: String
switch self {
case .Duplicate: expansion = "Another entry was already there."
case .OperationOnMissing: expansion = "There was no such entry."
case .NotAuthorized: expansion = "The operation was not authorized."
case .Parameter: expansion = "An internal error (paramErr) occurred.\n\n"
+ "Please report this to me@example.com"
case let .Generic(osError): expansion = "An unexpected error occurred (\(osError))."
case let .NoPasswordDefined(user: user, host: host):
expansion = "No password was defined for \(user) on \(host)."
}
return circumstance + expansion
}
}
/**
Convenient access to the system keychain.
`struct Keychain` simplifies CRUD operations on the system
keychain by wrapping every search and initialization parameter
internally, where client code can’t see.
There is no custom initializer; call `Keychain(user:,host:)`
The client initializes the struct with username and host.
After that,
* `password()` retrieves the password, if the matching
record is in the keychain.
* `setPassword(_:)` creates or updates the keychain record.
* `delete()` removes the record.
Getting or setting the password is always a mutation, because the underlying Security Framework calls set `osStatus`.
- throws: Any of a number of `KeychainError`s.
The `enum` specifies the likeliest errors. Any other is thrown
as `KeychainError.Generic`; examine `osStatus` for the code.
*/
struct Keychain {
// MARK: Publicly-accessible data
/// The login ID for the user, such as `criedel`.
let user: String
/// The name of the host for the user’s account, such as `www.example.com`.
let host: String
/// The error value (or `noErr`) from the last Keychain call.
var osStatus: OSStatus = noErr
init(user: String, host: String) {
self.user = user
self.host = host
}
// MARK: Getter/setter for password
/// The password for the user + host as stored in the keychain.
///
/// The absence of the user + host, or a password for the combination,
/// is one of the expected outcomes; the function will return `nil`
/// in that case.
///
/// This `func` takes care of the **Retrieve** part of the CRUD pattern.
///
/// - returns: the password if the user + host is registered, and a password is set; otherwise `nil`
///
/// - throws: Any of the `KeychainError`s for a misconfigured or forbidden access.
mutating func password() throws -> String? {
let returnPointer = UnsafeMutablePointer.alloc(1)
returnPointer.initialize(nil)
var terms = searchPattern()
terms[kSecReturnData] = true
osStatus = SecItemCopyMatching(terms, returnPointer)
switch osStatus {
case noErr:
if let password = (returnPointer.memory as? NSData),
retval = String(data: password, encoding: NSUTF8StringEncoding)
{ return retval }
else { return nil }
case errSecItemNotFound:
return nil
default:
throw KeychainError.fromOSStatus(osStatus)!
}
}
/// Set a (new) password for the (new) user + host.
///
/// * If the combination is registered, but the password is different, the record is updated.
/// * If they are not registered, the record is created, including the desired password.
///
/// This `func` takes care of both the **Create** and **Update** parts of the CRUD pattern.
///
/// - throws: Any of the `KeychainError` errors thrown by the underlying implementation.
///
/// - parameters:
/// - newPassword: The password to set for the user + host. The `func` will not validate it.
mutating func setPassword(newPassword: String) throws {
if let old = try? password(),
oldPassword = old
{
if newPassword == oldPassword {
// The password is already set. Nothing to do.
return
}
else {
// Item exists but must be updated
try updatePasswordInKeychain(newPassword)
}
}
else {
// nil oldPassword, meaning the item must be created.
try createKeychainItemWithPassword(newPassword)
}
}
/// Remove the user + host combination from the keychain.
///
/// This `func` takes care of the **Delete** part of the CRUD pattern.
///
/// - throws: Any of the `KeychainError` errors thrown by the underlying implementation.
mutating func delete() throws {
osStatus = SecItemDelete(searchPattern())
if let error = KeychainError.fromOSStatus(osStatus) {
throw error
}
}
// MARK: Convenience keychain access
/// A minimal dictionary of search criteria for the user + host
private func searchPattern() -> [NSString: AnyObject] {
let retval: [NSString: AnyObject] = [
// Entry class
kSecClass : kSecClassInternetPassword,
kSecAttrProtocol : kSecAttrProtocolHTTPS,
kSecAttrAuthenticationType
: kSecAttrAuthenticationTypeHTTPBasic,
kSecAttrAccount : user,
kSecAttrServer : host
]
return retval
}
/// Create a keychain item for the user + host, including password.
///
/// It is assumed that no item matching the user + host is in the keychain.
///
/// - parameters:
/// - password: The password to store in the item.
///
/// - throws: Any of the `KeychainError` errors, most likely `.Duplicate`.
private mutating func createKeychainItemWithPassword(password: String) throws {
let passwordData = password.dataUsingEncoding(NSUTF8StringEncoding)!
var values = searchPattern()
values[kSecValueData] = passwordData
// Prevent synchronization with other devices:
values[kSecAttrSynchronizable] = false
// The device has to have been unlocked since last restart, and will not be transferred to another device (as from backup)
values[kSecAttrAccessible] = kSecAttrAccessibleAfterFirstUnlockThisDeviceOnly
osStatus = SecItemAdd(values, nil)
if let error = KeychainError.fromOSStatus(osStatus) {
throw error
}
}
/// Replace the password for an existing keychain item for user + host.
///
/// It is assumed that an item matching the user + host is in the keychain.
///
/// - parameters:
/// - password: The password to store in the item.
///
/// - throws: Any of the `KeychainError` errors, most likely `.OperationOnMissing`.
private mutating func updatePasswordInKeychain(password: String) throws {
let passwordData = password.dataUsingEncoding(NSUTF8StringEncoding)!
let updateData: [NSString: AnyObject] = [ kSecValueData: passwordData,
// Prevent synchronization with other devices:
kSecAttrSynchronizable: false,
// The device has to have been unlocked since last restart, and will not be transferred to another device (as from backup)
kSecAttrAccessible: kSecAttrAccessibleAfterFirstUnlockThisDeviceOnly
]
osStatus = SecItemUpdate(searchPattern(), updateData)
if let error = KeychainError.fromOSStatus(osStatus) {
throw error
}
}
}
Well, we are into the new year, engineers and managers at Apple are refreshed from a holiday break, so let's hope we hear something soon.
I don't know if this might be helpfull, but I have a mixed objc (started with objc) and swift app. I use SSKeychain (written in ObjC) to store and read info from the keychain.
The part in objc works fine: I can retrieve stored info.
When I try to get them using swift I obtain that error, the other weird issue is that on simulator and with ad-hoc builds everything seems to work.
I get this error only if I run the app on device (9.2) while debugging.
Other curious stuff is that after a restart I can retrieve stored info at the same line of code where I get this error.
Pretty annoing.
Has anyone seen keychain error -34018 with iOS 9.2.1?
Yes, unfortunately we are still experiencing this problem in iOS 9.2.1. Very frustrating and disappointing to be honest.
Quinn, You said the only fix you are aware of was to "restart the process." What process? securityd? It woudl be useful to know what to look for in the Activity Monitor Instrument.
And as a user application, we can't restart a system daemon like securityd, can we?
Duncan Champney
WareTo
You said the only fix you are aware of was to "restart the process." What process?
The app itself.
Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware
let myEmail = "eskimo" + "1" + "@apple.com"I'm going completely mad at the moment 😟
How can it be this annoying?
In my case it's even weirder! I have a lot of logs concerning this issue. I have logs whith memory pressure being really high (marked with a nice "Received memory warning." message from os) and the access still works, and cases with a vast free memory and still get this error!!! I guess this being related to memory pressure might be a complete hoax.
On the other hand, I always face this problem on consecutive runs, i.e. My first run will always work (regardless of memory pressure) second or third work 50% and after that it just fails. And to add some more interesting observations, I don't need to actually free any memory or restart the device, just waiting on it for a couple of minutes will do!
Somebody kill me now. I don't want to raise a child in a world were I can't rely on apple to fix an error so wide spread!
On my tomb stone you'll read: "There was a keychain, but there were no access"
As per some other developers' speculations, most of these problems are arisen from a situation where Keychain has been accessed too often! In my case I did access three time in a row and this happened. Because my use of Keychain was only related to some public/private key form of access, I was able to completely resolve my problem by caching the keys!
It seems rather redundant to say that even if there is a security measurement to prevent some attacks on the Keychain, it is resulting in way too much false positives here and needs to be solved somehow.
For further info I strongly suggest everybody to follow this SO question: http://stackoverflow.com/questions/20344255/secitemadd-and-secitemcopymatching-returns-error-code-34018-errsecmissingentit
Any news regarding this one?
You uploaded this thread in Jul 2015, we are now in Feb 2016. You claimed "We have finally been able to reproduce the -34018 error on iOS 8.3. This is the first step in identifying the root cause and then coming up with a fix."
So, thousands of engineers and... nothing???
I've managed to recreate this bug in a slight different way. I can access the keychain as many times as I want in a short space of time, however as soon as I launch the camera through an image picker view controller, then I immediately start getting the -34018 on all future reads and writes.
Just to add a little bit to this thread.
iOS 9.2.1 has this issue
Reproducibility
- I can reproduce it more often with a debugger attached (currently with about 25% probability). However, sometimes I see it even without a debugger (much, much rarer).
- Usually failures are clustered. Sometimes I am getting this problem several times in the row and after I don't see it for a while.
- I have about ~200MB of free memory when I reproduce this issue. So, it could be one of the variables, but it's not only thing which casues it. One of the thread mentioned that securityd got killed because of memory pressure. I don't think it's simple like that (there were other people who mentioned like me that they saw the problem without memory pressure).
- I don't think that signing has any influence on this thing. The same app (with the same signature and the same profiles installed on the device) will sometimes throw this error and sometime work fine.
- I am seeing this problem on main thread. So, I don't think that a problem is running on non main thread (I saw solution recommending to do dispatch to run on the main thread)
- I don't think that multiple or frequent access to Keychain has anything to do with it. I did a test and run a loop with 1000 access in it and it went well (if the problem was # or frequence it would fail). However, I see this error most of the time literally on the first attempt to work with a keychain.
- People were reporting that it happens only when an app switches from a background or go through a deeplinking. In my cases, it happens with a just common usage (an app being started by a debugger).
Thoughts
I hope that Apple engineers will figure it out. It's kind of strange to not being able to rely on Keychain.
Just a shot in the dark. Frankly, taking into account all of these factors which kind of influence, but aren't exact immediate cause, my guess would be that it's either tricky race condition or a memory corruption somewhere in keychain related process.
I think the only temporary workaround is to not use Keychain (with all security implications of this decision).
Our app was just launched and this is our top crash. By a very very large margin.
I can easily reproduce as well.
in my applicationDidBecomeActive I call SecItemCopyMatching. I open bunch of apps like Safari and open tabs loading various pages, then I resume my app. After a few attempts like this I get the -34018 error.
I work on a security app, so it's pretty important that the Keychain works properly. Please fix this Apple.
Was just able to reproduce and caught the console logs. When the error hits, I see this same entries over and over again in the console log:
Mar 9 13:06:41 Jeejah securityd[94] <Error>: secTaskDiagnoseEntitlements MISSING keychain entitlements: raw entitlement values: <CFBasicHash 0x14fe2ebc0 [0x19fc27b68]>{type = mutable dict, count = 0,
entries =>
}
Mar 9 13:06:41 Jeejah securityd[94] <Error>: secTaskDiagnoseEntitlements MISSING keychain entitlements: original ag: (null)
Mar 9 13:06:41 Jeejah securityd[94] <Error>: secTaskDiagnoseEntitlements MISSING keychain entitlements: newly parsed ag: (null)
Mar 9 13:06:41 Jeejah securityd[94] <Error>: __security_simulatecrash Simulating crash, reason: keychain entitlement(s) missing, code=53c00007
Mar 9 13:06:41 Jeejah ReportCrash[372] <Error>: Simultaneous simulate crash requests for pid 94
Mar 9 13:06:41 Jeejah securityd[94] <Error>: securityd_xpc_dictionary_handler KP Prevention[371] copy_matching Error Domain=NSOSStatusErrorDomain Code=-34018 "client has neither application-identifier nor keychain-access-groups entitlements" UserInfo={NSDescription=client has neither application-identifier nor keychain-access-groups entitlements}
I'm posting the full console log to the bug report we filed. And in case you are wondering, yes the app DOES have an application-identifier.
BTW - I can tell from conversations I have with management on this issue that not all are convinced that it is an Apple defect. So, to paraphrase what somebody else has said in this thread, Apple is making our life harder by being so quite on this issue.
Congrats Apple! The original thread for this bug is now more than 1 and a half years old (18 Sept 2014).
😢
Pretty insane that Apple hasn't addressed this yet. Anyone have any advice for storing private/important data (e.g., user token) without keychain?
What we did is use User Defaults while encrypting the desired values in AES256 and build+obfuscate the key in runtime. I know its a massive degradation to the security of the app but we had no other option. It was using this bypass or killing the app and telling the user to clean his memory and open it again.
I suggest opening a bug in the bug reporter and start praying Apple will solve it until 2019 christmas.
I'll hook in to this thread in the hopes that iOS 9.3 release will bring any changes...
Hi eskimo,
Is this issue solved permanently in iOS9.3? (keychain OSStatus -34018)
OK, here’s the latest. This is a complex problem with multiple possible causes:
Some instances of the problem are caused by incorrect app signing. You can easily distinguish this case because the problem is 100% reproducible.
Some instances of the problem are caused by a bug in how iOS supports app development (r. 23,991,853). Debugging this was complicated by the fact that another bug in the OS (r. 23,770,418) masked its effect, meaning the problem only cropped up when the device was under memory pressure.
We believe these problems were resolved in iOS 9.3.
We suspect that there may be yet more causes of this problem.
So, if you see this problem on a user device (one that hasn’t been talked to by Xcode) that’s running iOS 9.3 or later, please do file a bug report about it. Try to include the device system log in your bug report (I realise that can be tricky when dealing with customer devices; one option is to ask the customer to install Apple Configurator, which lets them view the system log). And if you do file a bug, please post your bug number, just for the record.
On behalf of Apple I’d like to thank everyone for their efforts in helping to track down this rather horrid issue.
Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware
let myEmail = "eskimo" + "1" + "@apple.com"Quinn,
Thank you for the update. I have one follow-up and one request:
Follow-ups: You mentioned getting a system log on a 9.3 device "that hasn't been talked to by Xcode" - do you mean that once the device has been connected to Xcode, that the system log is not helpful? We have encountered this bug on an app signed with an Enterprise certificate (app installed over the air, not while tethered to XCode) - but the device itself has been connected to XCode for debugging; is the device system log still useful in this case?
Request: I know that Apple's policy is to not comment on defects, and it is for a very good reason; but it is a 'policy' and not a fundamental law and therefore exceptions can be discussed and granted. Due to the longevity of this defect, and the seemingly random nature of which apps or devices it occurs on, makes this a strong candidate. Especially since one app will experience this issue while another won't leaves very smart people to believe that there must be something wrong in the code my code. While I can only speak for myself, I have wasted a great deal of energy writing and testing code that attempts to work around this defect; I suspect others on this forum also have had such an experience. Please, on behalf of those who are wasting their energies on this defect instead of implementing other iOS features or other improvements on their app, request an exception from management and get all known details out in the open. Thank you.
Andrew
… do you mean that once the device has been connected to Xcode, that the system log is not helpful?
When you connect a device to a Mac with Xcode running, it puts the device in ‘developer mode’. My suggestion is that you try to reproduce this on a device where that’s not happened.
Note that restarting the device clears this state, so when I said “talked to by Xcode” I should have said “talked to by Xcode since it was last restarted”.
Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware
let myEmail = "eskimo" + "1" + "@apple.com"EDIT: still happening to us
Quinn this is excellent news!
Thank you for your efforts on keeping us updated. You have been our only beacon of hope in this dark hour for Apple 🙂
Love,
The whole team at Easter Egg!