Technical Note TN2232

HTTPS Server Trust Evaluation

HTTPS is a cornerstone of Internet security. When making a connection over HTTPS, the client must evaluate whether to trust the server. If this trust evaluation fails, the client refuses to connect. This can happen for a variety of reasons, some benign—the server might be using a self-signed certificate, an intermediate certificate is missing, and so on—and some malicious—the server is an impostor, looking to steal the user's data. This document describes the reasons why server trust evaluation can fail, and how this problem can be resolved while not compromising the user's security.

While the focus of this document is on HTTPS, it also covers TLS, which is the underlying security protocol used by HTTPS, and TLS's older cousin, SSL.

You should read this document if you're creating an iOS or Mac OS X program that uses HTTPS, TLS or SSL to talk to a server securely, and you need to resolve a server trust evaluation failure or enforce a stricter form of server trust evaluation.

Introduction
Understanding Server Trust Evaluation Failures
Basic Trust Customization
Trust Customization for Specific APIs
Resolving Specific Server Trust Evaluation Failures
Enforcing Stricter Server Trust Evaluation
Hints and Tips
Appendix A: Common Server Trust Evaluation Errors
Appendix B: Glossary
Document Revision History

Introduction

Your first encounter with HTTPS server trust evaluation is likely to be an error like the following:

Domain=NSURLErrorDomain Code=-1202 "The certificate for this server is invalid. You might be connecting to a server that is pretending to be “example.com” which could put your confidential information at risk." UserInfo=0x14a730 {NSErrorFailingURLStringKey=https://example.com/, NSLocalizedRecoverySuggestion=Would you like to connect to the server anyway?, NSErrorFailingURLKey=https://example.com/, NSLocalizedDescription=The certificate for this server is invalid. You might be connecting to a server that is pretending to be “example.com” which could put your confidential information at risk., NSUnderlyingError=0x14a6c0 "The certificate for this server is invalid. You might be connecting to a server that is pretending to be “example.com” which could put your confidential information at risk.", NSURLErrorFailingURLPeerTrustErrorKey=<SecTrustRef: 0x14ec00>}

In this case error -1202 in the NSURLErrorDomain domain is NSURLErrorServerCertificateUntrusted, which means that server trust evaluation has failed. You might also receive a variety of other errors; Appendix A: Common Server Trust Evaluation Errors lists the most common ones.

If you receive one of these errors and search the 'net for help, you might find advice like this:

To get around this problem simply disable the certificate checks.

Or, worse yet, like this:

To get around this problem disable the certificate checks by calling +[NSURLRequest setAllowsAnyHTTPSCertificate:forHost:].

Blindly following this advice is a serious mistake. HTTPS (actually, the underlying TLS protocol) offers two important security guarantees, and if you disable server trust evaluation you totally invalidate one of these guarantees.

In most cases the best way to resolve a server trust evalution failure is to fix the server. This has two benefits: it offers the best security and it reduces the amount of code you have to write. The remainder of this technote describes how you can diagnose server trust evaluation failures and, if it's not possible to fix the server, how you can customize server trust evaluation to allow your connection to proceed without completely undermining the user's security.

TLS Security Guarantees

HTTPS is defined by RFC 2818 but, for the most part, it consists of a simple composition of two existing protocols:

HTTPS inherits its security guarantees from TLS. By default these are:

  • on-the-wire privacy — This guarantees that the data is secure from a passive attacker (someone who can see but can't modify the packets exchanged between the client and server).

  • client-authenticates-server authentication — This protects the client from an active attacker (someone who can both see and modify the packets exchanged between the client and the server). Most importantly, it protects the client from various forms of man-in-the-middle attack, including impostor servers (servers who pretend to be the real server and thus acquire confidential information from the client).

TLS server trust evaluation is the part of TLS that implements client-authenticates-server authentication. If you disable it, you lose this second, extremely important security guarantee.

Resolving Server Trust Evaluation Failures

If disabling server trust evaluation completely is a mistake, what should you do? Your first step should be to diagnose the problem. To do this, read Understanding Server Trust Evaluation Failures. Once you understand the problem you'll likely find that the easiest solution is to fix the server. This reduces the amount of code you have to write and guarantees the user's security.

If it's not possible to fix the server, you will need to customize the server trust evaluation to allow the connection to proceed without completely undermining the user's security. Basic Trust Customization explains the general process for customizing server trust evaluation; it's followed by Trust Customization for Specific APIs which explains the process for various commonly-used APIs. Finally, Resolving Specific Server Trust Evaluation Failures is a discussion of how to work around specific server trust evaluation problems.

You can also customize server trust evaluation for the opposite reason, that is, to make the connection more secure. See Enforcing Stricter Server Trust Evaluation for a discussion of this.

Understanding Server Trust Evaluation Failures

When you connect to a server using TLS, it gives you the certificate of the server and guarantees that the server holds the private key that matches the public key embedded in that certificate. This is the first step in establishing a secure connection. The second step, which is just as critical, is for you to examine the certificate you got from the server to decide whether it matches the server you are expecting to talk to. This process is known as server trust evaluation, and this section describes the basic techniques involved.

Trust Evaluation Fundamentals

TLS server trust evaluation consists of two basic steps:

  1. basic X.509 certificate trust evaluation

  2. TLS-specific additions

To understand the first step, you need to know a little bit about X.509 certificates. An X.509 certificate has five critical attributes:

If the digital signature is valid then the certificate is a guarantee from the issuer that the subject holds the private key associated with the public key in the certificate. For example, the certificate for DevForums (https://devforums.apple.com/) is (at the time of writing) issued by Entrust, and by signing that certificate Entrust is guaranteeing that the maintainers of the DevForums server hold the private key associated with the public key in the certificate.

X.509 certificate trust evaluation is a recursive two-step process:

  1. check the validity of the certificate itself — This involves various things, but the two most important are a) verifying the digital signature, and b) checking that the verify date (typically the current date) is within the certificate's valid date range.

  2. check the validity of the issuer — This involves finding the issuer's certificate and (recursively) checking its validity.

Clearly this recursive process must terminate eventually. Trust evaluation can succeed in only one case: if it hits a trusted anchor. A trusted anchor is a certificate that the system trusts implicitly, typically because it's the root certificate of a well-known certificate authority that has been baked in to the system.

On the other hand, trust evaluation can fail for a variety of reasons:

  • if it hits an invalid certificate

  • if it can't find the certificate for an issuer

  • if it hits a self-signed certificate that is not a trusted anchor

If X.509 trust evaluation is successful, the system then applies additional TLS-specific checks. In practice this involves checking that the DNS name that you are attempting to connect to matches the DNS name in the certificate. There are, however, a few wrinkles:

  • Typically you would expect to find this DNS name in the subject's Common Name field, but it can also be in a Subject Alternative Name extension. If a name is present in the Subject Alternative Name extension, it takes priority over the Common Name.

  • The Subject Alternative Name extension may also contain IP addresses, which the client consults if it connected via an IP address rather than a DNS name.

  • The DNS name in the certificate might be a wildcard name, for example, "*.apple.com".

  • The Extended Key Usage extension is expected to include the Server Authentication value.

If you're interested in the details of these TLS-specific checks, see RFC 2818.

Common Failures

With the above in mind, it's easy to understand the various server trust evaluation failures that you're likely to see. These include:

  • missing issuer certificate — For any given certificate (except the trusted anchor), the system must be able to locate the certificate of the issuer.

  • date problems — For any given certificate, the verify date must be within the certificate's valid date range.

  • self-signed certificate — For any given certificate, if the certificate is self-signed, it will cause evaluation to fail (unless it's a trusted anchor).

  • no trusted anchor — The system must be able to follow the path of issuer certificates leading to a trusted anchor.

  • DNS name mismatch — The DNS name that you're trying to connect to must match the name in the server certificate, as described in the previous section.

If, after investigating each of the points above, you still can't work out why server trust evaluation is failing, you might want to try the technique described in Investigating Hard-To-Debug Trust Evaluation Failures.

Debugging Tools

There are a variety of tools you can use to debug server trust evaluation problems, and this section discusses some of the more common ones.

Keychain Access

Keychain Access (in the Utilities folder) has a number of useful certificate debugging features:

  • It can import and export certificates and identities in a variety of formats.

  • If you double click on a certificate it will display a nice GUI certificate viewer.

  • The Certificate Assistant, which you access via Keychain Access, can evaluate trust on a specific certificate.

  • The Certificate Assistant can create self-signed certificates.

  • The Certificate Assistant can create a certificate authority which you can use to issue leaf and intermediate certificates for testing. You can learn more about this feature of Certificate Assistant below.

When playing around with Keychain Access it's a good idea to create a scratch keychain which you can use for testing credentials. This ensures that your testing credentials don't get mixed up with the important credentials you use day-to-day.

Security Tool

The security tool on OS X gives you access to a wide range of security functionality, not all of which is exposed via Keychain Access. For example, you can use it to:

  • dump the keychain in a text format

  • work with trust settings in detail

  • add certificates to and remove them from the keychain

For more information about the security tool, read its man page.

Safari on the Mac

If you visit a trusted HTTPS web site with Safari you can click the lock icon in the title bar to view the sequence of certificates leading from the server's certificate to a trusted root.

If you visit an untrusted HTTPS web site with Safari, it will display its "can't verify the identity of the website" sheet. The Show Certificate button will extend the sheet so that you can see the certificates being considered.

Both of these certificate viewers let you drag a certificate out to the desktop (start your drag on the large certificate icon), which is useful if you need to check the certificate with other tools (for example, Certificate Assistant's trust evaluation feature).

Interactive Testing

In some cases it's useful to connect to a server and issue it commands for testing purposes. For typical Internet protocols (HTTP, SMTP, NNTP, and so on) you can do this with the telnet tool. This does not work, however, if the protocol uses TLS. In that case your best option is the s_client subcommand of the openssl tool. Listing 1 shows how you can use this tool to manually get the contents of <https://www.apple.com> (remember that HTTPS uses port 443).

Listing 1  Using openssl s_client

$ openssl s_client -connect www.apple.com:443
CONNECTED(00000003)
[...]
GET / HTTP/1.1
Host: www.apple.com
 
HTTP/1.1 200 OK
Server: Apache/2.2.3 (Oracle)
Content-Length: 9464
Content-Type: text/html; charset=UTF-8
ntCoent-Length: 9516
Cache-Control: max-age=47
Expires: Mon, 25 Jun 2012 16:18:24 GMT
Date: Mon, 25 Jun 2012 16:17:37 GMT
Connection: keep-alive
 
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-US" lang="en-US">
[...]
</html>
closed
$

The s_client subcommand supports a number of useful debugging options. For example:

  • You can supply the -cert argument to have it respond to client certificate requests.

  • You can specify the -showcerts option to get the complete list of certificates provided by the server.

  • The -debug and -msg options enable low-level debugging features.

See the man page for more information about these options and more.

  • If you're investigating weird TLS behavior it's sometimes useful to get a 'second opinion' by testing with the OpenSSL TLS stack.

  • On the other hand some problems are specific to Secure Transport, so that a successful test with the s_client subcommand doesn't guarantee that your app's code will work.

Finally, the s_client subcommand with the -showcerts option is a good way to get a copy of the server's certificate chain. Listing 2 is an example of this.

Listing 2  Getting the full list of certificates

$ openssl s_client -showcerts -host www.apple.com -port 443
[...]
---
Certificate chain
 0 s:/C=US/L=Cupertino/O=Apple Inc./ST=CALIFORNIA/CN=www.apple.com
   i:/C=US/O=Akamai Technologies Inc/CN=Akamai Subordinate CA 3
-----BEGIN CERTIFICATE-----
MIIDJzCCApCgAwIBAgIOAQAAAAABNwl6mQWT01EwDQYJKoZIhvcNAQEFBQAwUTEL
MAkGA1UEBhMCVVMxIDAeBgNVBAoTF0FrYW1haSBUZWNobm9sb2dpZXMgSW5jMSAw
HgYDVQQDExdBa2FtYWkgU3Vib3JkaW5hdGUgQ0EgMzAeFw0xMjA1MDExNzUzNDNa
Fw0xMzA1MDExNzUzNDNaMGMxCzAJBgNVBAYTAlVTMRIwEAYDVQQHEwlDdXBlcnRp
bm8xEzARBgNVBAoTCkFwcGxlIEluYy4xEzARBgNVBAgTCkNBTElGT1JOSUExFjAU
BgNVBAMTDXd3dy5hcHBsZS5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK
AoIBAQCdJlExG8umAtxL9Df/10diaYoqFeeVDbU13cH0KNxq2vV0nSb3dROtUAig
/wXj6jFU16fhfFegjcYBkYL9qiUkIiROMNo9r1IzZX4Yv9HT30vBdRMZkuIdy9eP
m9nctNVYyBGRJapem1c+llxAJzToiuQofBY6L6K2dO7nuGm7AZ/PwcbyxZEWoh0E
2SKMbMD9vG0Dph+rZDcPxENFa401j95ZiyyDinHXPliPVGimQmLeaWXMOhJSGXcd
FidodxJCGPUIMuQnxipIH8EAiOg76aKsFvi/7ctoYtRGsXJZqRLnKJ2JVAq+tQxj
6so8vaO5I5wqjyCa+ECJJ0i8Vhp/AgMBAAGjbDBqMDkGA1UdHwQyMDAwLqAsoCqG
KGh0dHA6Ly9jcmwuZ2xvYmFsc2lnbi5uZXQvQWthbWFpU3ViMy5jcmwwHQYDVR0O
BBYEFK5QfqCwo5h4aWa7DmhIJdMZ50FjMA4GA1UdDwEB/wQEAwIFIDANBgkqhkiG
9w0BAQUFAAOBgQCMfDikw5AwrCCCkhcb+ak5oTRmhV88mL5Pk7SzVTbMdCoaktOD
+Bu7iX0OYsISOjYu0x2CzX2VQ5kP5NhA7fqXOiq4iG1G/Ae+xW01lUB1gJ7VUwoX
9LabdT6c812EOMpza4lrnLqnOsiSCDf1SWv0Lo+pMkZ9Ka9EbSd3DqUEHw==
-----END CERTIFICATE-----
 1 s:/C=US/O=Akamai Technologies Inc/CN=Akamai Subordinate CA 3
   i:/C=US/O=GTE Corporation/OU=GTE CyberTrust Solutions, Inc./CN=GTE CyberTrust Global Root
-----BEGIN CERTIFICATE-----
MIIDxzCCAzCgAwIBAgIEBAAEAzANBgkqhkiG9w0BAQUFADB1MQswCQYDVQQGEwJV
[...]
bZ14M6VWhKYouKEZnaAsSCe+XHsF0haUfOnxpj4p7CZj/DnGZVB8Uh92ORa0lyY5
q44d/bV6wDodO38=
-----END CERTIFICATE-----
---
[...]

You can examine one of these certificates by:

  1. copying the text (the -----BEGIN CERTIFICATE----- line through to the -----END CERTIFICATE----- line) into a text file with the .pem extension

  2. dragging that file into Keychain Access

  3. double clicking the newly imported certificate

This will open the standard certificate viewer which you can use to examine the certificate in detail.

Dumping ASN.1

Many security technologies, including certificates, use the ASN.1 binary data format. This format is not even remotely human readable, but there are tools that can help. Most notably the dumpasn1 command line tool can convert a file containing ASN.1 binary data to the ASN.1 text format, making it much easier to understand.

Basic Trust Customization

iOS and OS X support trust evaluation by way of trust objects (of type SecTrustRef). It's possible to create trust objects from scratch but, in the case of TLS server trust evaluation, it's typically the case that the API you're using already does some default server trust evaluation and you want to customize that. Thus the standard approach is:

  1. get the trust object

  2. evaluate it yourself, just to be sure that it's the cause of the failure

  3. if the trust evaluation succeeds, you may choose to enforce stricter trust evaluation rules, as discussed in Enforcing Stricter Server Trust Evaluation

  4. if the trust evaluation fails, apply your customizations to the trust object

  5. evaluate the trust again, and either allow or deny the connection based on this evaluation

The mechanism used to get the trust object is API specific. Trust Customization for Specific APIs discusses the common cases. However, once you've got the trust object, the process of evaluating and customizing it is the same for all APIs.

Listing 3 shows the basic technique for evaluating a trust object.

Listing 3  Evaluating a trust object

OSStatus            err;
BOOL                allowConnection;
SecTrustResultType  trustResult;
 
allowConnection = NO;
 
err = SecTrustEvaluate(trust, &trustResult);
if (err == noErr) {
    allowConnection = (trustResult == kSecTrustResultProceed) ||
                      (trustResult == kSecTrustResultUnspecified);
}

You can then customize the server trust evaluation as you see fit. For example, if you're talking to a server whose certificate was issued by a certificate authority that's not trusted by the system, you can call SecTrustSetAnchorCertificates to set the anchors for the trust object to be that certificate authority's root certificate. Listing 4 shows an example of this.

Listing 4  Using a custom anchor

OSStatus            err;
BOOL                allowConnection;
SecCertificateRef   customAnchor;
SecTrustResultType  trustResult;
 
allowConnection = NO;
 
customAnchor = ... the CA's root certificate ...;
 
err = SecTrustSetAnchorCertificates(
    trust,
    (__bridge CFArrayRef) [NSArray arrayWithObject:(__bridge id) customAnchor]
);
if (err == noErr) {
    err = SecTrustEvaluate(trust, &trustResult);
}
if (err == noErr) {
    allowConnection = (trustResult == kSecTrustResultProceed) ||
                      (trustResult == kSecTrustResultUnspecified);
}
code

Finally, there may be situations where the required customization cannot be done directly on the trust object. For example, if you want a trust object to consider an additional intermediate certificate, you can't add that certificate directly to the object (r. 16058372) . You can, however, re-create the trust object with the parameters you want and then evaluate that new trust object. Listing 5 shows an example of this.

Listing 5  Re-creating the trust object

OSStatus            err;
BOOL                allowConnection;
CFArrayRef          policies;
NSMutableArray *    certificates;
CFIndex             certCount;
CFIndex             certIndex;
SecCertificateRef   extraIntermediate;
SecTrustRef         newTrust;
SecTrustResultType  newTrustResult;
 
allowConnection = NO;
 
policies = NULL;
newTrust = NULL;
 
err = SecTrustCopyPolicies(trust, &policies);
if (err == errSecSuccess) {
    certificates = [NSMutableArray array];
 
    certCount = SecTrustGetCertificateCount(trust);
    for (certIndex = 0; certIndex < certCount; certIndex++) {
        SecCertificateRef   thisCertificate;
 
        thisCertificate = SecTrustGetCertificateAtIndex(trust, certIndex);
        [certificates addObject:(__bridge id)thisCertificate];
    }
 
    extraIntermediate = ... the extra intermediate certificate to use ...;
    [certificates addObject:(__bridge id)extraIntermediate];
 
    err = SecTrustCreateWithCertificates(
        (__bridge CFArrayRef) certificates,
        policies,
        &newTrust
    );
    if (err == noErr) {
        err = SecTrustEvaluate(newTrust, &newTrustResult);
    }
    if (err == noErr) {
        allowConnection = (newTrustResult == kSecTrustResultProceed) ||
                          (newTrustResult == kSecTrustResultUnspecified);
    }
}
 
if (newTrust != NULL) {
    CFRelease(newTrust);
}
if (policies != NULL) {
    CFRelease(policies);
}

Trust Customization for Specific APIs

Once you understand the basics of using a trust object, you have to know how to get such an object from the API you're using. The following sections cover a variety of APIs, from the very high-level (WebView and UIWebView) to the lowest level (Secure Transport).

WebView

OS X's WebView gives you access to NSURLConnection authentication challenges via its resourceLoadDelegate property. You can respond to authentication challenges in the same way that you would with NSURLConnection. Alas, it's not possible to access NSURLAuthenticationMethodServerTrust authentication challenges via this mechanism (r. 9475067) . You can work around this limitation using an NSURLProtocol subclass, as illustrated by Sample Code 'CustomHTTPProtocol'.

UIWebView

UIWebView does not provide any way for an app to customize its HTTPS server trust evaluations (r. 10131336) . You can work around this limitation using an NSURLProtocol subclass, as illustrated by Sample Code 'CustomHTTPProtocol'.

HTTP Live Streaming

HTTP Live Streaming's support for server trust evaluation depends on the type of resource being fetched, as shown in Table 1.

Table 1  Customizing HTTPS Server Trust Evaluation in HTTP Live Streaming

Resource Type

Extension

Can Customize Server Trust Evaluation?

media segment

.ts

no

index (playlist)

.m3u8

yes

key

n/a

yes

For general information about HTTP Live Streaming, see HTTP Live Streaming Overview. For specific information on how to get HTTP Live Streaming to call your code to fetch keys—at which point you can use whatever technique you want to access them, including NSURLSession with a customized HTTPS server trust evaluation—watch WWDC 2011 Session 408 HTTP Live Streaming Update.

Also, AV Foundation allows you to implement your own resource loading, at which point you can use any network API to load these resources (and thus customize HTTPS server trust evaluation using that API). Specifically, you can set a resource loader (AVAssetResourceLoader) on an asset (AVAsset) and then use its delegate to override the loading of non-media segment resources (see Table 1). Support for this technique was introduced in iOS 6.0 and OS X 10.9.

NSURLSession

NSURLSession allows you to customize HTTPS server trust evaluation by implementing the -URLSession:didReceiveChallenge:completionHandler: delegate method. To customize HTTPS server trust evaluation, look for a challenge whose protection space has an authentication method of NSURLAuthenticationMethodServerTrust. For those challenges, resolve them as described below. For other challenges, the ones that you don't care about, call the completion handler block with the NSURLSessionAuthChallengePerformDefaultHandling disposition and a NULL credential.

When dealing with the NSURLAuthenticationMethodServerTrust authentication challenge, you can get the trust object from the challenge's protection space by calling the -serverTrust method. After using the trust object to do your own custom HTTPS server trust evaluation, you must resolve the challenge in one of two ways:

  • If you want to deny the connection, call the completion handler block with the NSURLSessionAuthChallengeCancelAuthenticationChallenge disposition and a NULL credential.

  • If you want to allow the connection, create a credential from your trust object (using +[NSURLCredential credentialForTrust:]) and call the completion handler block with that credential and the NSURLSessionAuthChallengeUseCredential disposition.

NSURLConnection

NSURLConnection allows you to customize HTTPS server trust evaluation in much the same way as NSURLSession. There are two significant differences:

  • You implement a different delegate method (in this case -connection:willSendRequestForAuthenticationChallenge:).

  • When resolving a challenge, you must get the sender from the challenge (via the -sender method) and then call the appropriate method on that sender:

    • for challenges that you don't care about, call -performDefaultHandlingForAuthenticationChallenge:

    • if you want to deny a connection, call -cancelAuthenticationChallenge:

    • if you want to allow a connection, call -useCredential:forAuthenticationChallenge:, passing in a credential created with a trust object as described in NSURLSession

CFHTTPStream

You can customize the HTTPS server trust evaluation for a CFHTTPStream in the same way you do for a CFSocketStream (see the next section). There is, however, one serious gotcha: by the time you're in a position to get a trust object in order to apply your custom trust evaluation, the CFHTTPStream has already sent your HTTP request to the server. So if the HTTP request is potentially confidential, this technique is not appropriate. In that case you should either move up a layer, and use NSURLSession, or down a layer, and use CFSocketStream directly.

CFSocketStream

To customize the TLS server trust evaluation for a CFSocketStream, you should do the following:

  1. use the kCFStreamSSLValidatesCertificateChain entry of the kCFStreamPropertySSLSettings property to disable server trust evaluation completely

  2. once the stream has connected, but before you send any data or trust any data you receive, get the trust object from the stream via the kCFStreamPropertySSLPeerTrust property

  3. use that trust object to implement whatever custom server trust evaluation you desire

  4. either continue with the connection or close it, depending on your server trust evaluation

The primary gotcha here relates to step 2: when is the stream connected? It turns out that the trust object is not available at the time you get the stream open-completed event. To guarantee that the trust object is available, you must wait for the has-space-available event or the has-bytes-available event. See Table 2 for the details.

Table 2  Stream Events and Trust Object Availability

NSStream Event

CFStream Event

Trust Object?

NSStreamEventOpenCompleted

kCFStreamEventOpenCompleted

not available

NSStreamEventHasSpaceAvailable

kCFStreamEventCanAcceptBytes

available

NSStreamEventHasBytesAvailable

kCFStreamEventHasBytesAvailable

available

Secure Transport

Secure Transport is the lowest-level TLS implementation on both OS X and iOS and, as you might expect, it has good support for custom TLS server trust evaluation. The procedure is as follows:

  1. before starting the connection, call SSLSetSessionOption to set kSSLSessionOptionBreakOnServerAuth

  2. if necessary, as discussed below, call SSLSetEnableCertVerify to disable the default server trust evaluation

  3. run the Secure Transport handshake as per usual

  4. when SSLHandshake returns errSSLServerAuthCompleted, call SSLCopyPeerTrust to get a trust object for the connection

  5. use that trust object to implement whatever custom server trust evaluation you desire

  6. either continue the Secure Transport handshake or shut down the connection

Resolving Specific Server Trust Evaluation Failures

The previous sections discussed the type of server trust evaluation failures that might occur, how you use a trust object to implement your own custom server trust evaluation, and how you get a trust object from the most frequently use APIs. This section shows how all these pieces fit together: how you can use the techniques described earlier to resolve specific server trust evaluation failures while still maintaining some degree of security.

The bulk of this discussion assumes that you have encountered a server trust evaluation failure with one specific server, or a small, well-characterized group of servers. If you're creating a general-purpose program, one that might connect to a wide range of servers as specified by the user, you should skip forward to Trust Exceptions.

Before reading the rest of this section, there are two things to keep in mind:

Server Name Failures

If server trust evaluation fails because the server's DNS name does not match the DNS name in the certificate, you can work around the failure by overriding the DNS name in the trust object. To do this, call SecPolicyCreateSSL to create a new policy object with the correct server name and then call SecTrustSetPolicies to apply it to the trust object.

Missing Intermediate Certificates

There are a number of ways that you can resolve a server trust evaluation failure caused by a missing intermediate certificate:

  • The best approach is, as always, to fix the server. TLS requires that the server provide the client all certificates leading from the server certificate to the trusted anchor certificate, including any intermediate certificates (see Section 7.4.2 of RFC 5246). In many cases the absence of an intermediate certificate is just a minor oversight that the server admin can quickly correct.

  • On OS X, if your program (or the user) adds the intermediate certificate to any keychain in the keychain search list, the trust object will use it automatically.

  • On iOS, if your app adds the intermediate certificate to its keychain, the trust object will use it automatically.

  • If all else fails, your program can obtain the intermediate certificate (bundle it with the program or download it from the Internet) and then create a new trust object that includes the union of the certificates from the existing trust object and your extra intermediate certificate. See Listing 5 for an example of this.

Trusting One Specific Certificate

In some cases it's useful to treat a certificate as a simple identity token. For example, in a peer-to-peer program, X.509 trust evaluation is pointless because there is no central authority to issue certificates. However, you can still use TLS to securely communicate within that architecture. The procedure is as follows:

  1. get a copy of the remote peer's certificate via a trusted channel of your own devising; you could have the remote user email you their certificate, put it on a USB stick, or whatever

  2. get the server certificate from the trust object (pass an index of 0 to SecTrustGetCertificateAtIndex)

  3. get the data for that server certificate (SecCertificateCopyData)

  4. compare this to the certificate data you got in step 1; if they match, you're talking to the correct peer

Custom Certificate Authority

If your server's certificate is issued by a certificate authority that is not trusted by the system by default, you can resolve the resulting server trust evaluation failure by including the certificate authority's root certificate in your program. The procedure is as follows:

  1. include a copy of the certificate authority's root certificate in your program

  2. once you have the trust object, create a certificate object from the certificate data (SecCertificateCreateWithData) and then set that certificate as a trusted anchor for the trust object (SecTrustSetAnchorCertificates)

  3. SecTrustSetAnchorCertificates sets a flag that prevents the trust object from trusting any other anchors; if you also want to trust the default system anchors, call SecTrustSetAnchorCertificatesOnly to clear that flag

  4. evaluate the trust object as per usual

Self-Signed Certificates

Dealing with self-signed certificates is problematic because there are a variety of different reasons why people use a self-signed certificate. This section covers the most common cases.

Development

During development a self-signed certificate makes it easier to set up an TLS-based server for testing. This is a perfectly reasonable use of self-signed certificates, and a perfectly reasonable excuse for completely disabling TLS server trust evaluation.

While a self-signed certificate is a reasonable approach during development, there is a better way: create your own certificate authority (see below) and have it issue a certificate for your test server. You can then either a) hard-wired your certificate authority's root certificate into your app (as explained in Custom Certificate Authority), or b) have each of your testers install your certificate authority's root certificate via the standard system user interface (Safari, Mail and configuration profiles on iOS, Keychain Access on OS X). This approach has the advantage that you don't need to disable server trust evaluation during development, which means that you can't forget to re-enable it when you ship a production build.

  • Certificate Assistant — This app is built in to OS X and supports a reasonably nice user interface for managing your own certificate authority; Technical Note TN2326, 'Creating Certificates for TLS Testing' has detailed instructions on how to use it.

  • OpenSSL — The OpenSSL command line utility lets you create and manage your own certificate authority. This is rather complex (see the man page for the details) but there are lots of tutorials out there on the Internet.

Business Reasons

Some organizations use a self-signed certificate for production infrastructure because of business reasons. Perhaps the organization is unwilling to buy a certificate from a trusted certificate authority (although such certificates are not expensive by any means). Or perhaps the organization can acquire such a certificate, but there is a lot of red tape involved.

The obvious solution here is to resolve the business issue and move on. However, that's not always possible. In that case a self-signed certificate is an option, but it may not be the best option. In most cases it's better to use your own certificate authority, either purely for development purposes (see Development) or also in production as well (see the note above). Using your own certificate authority has a number of advantages over a self-signed certificate:

  • server changes — If the server changes in such a way that breaks server trust evaluation, your certificate authority can issue a new certificate that describes the new state of the server and clients will automatically trust it. For example, if you have to change the DNS name of the server, the new certificate can contain the new DNS name.

  • reissue — If the server's certificate expires, your certificate authority can reissue it.

  • revocation — Your certificate authority can support certificate revocation, either via OCSP or a CRL.

Third Party Server

The previous sections have discussed the reasons why your organization should avoid self-signed certificates. However, what happens if your program communicates with a server out of your control, and it has a self-signed certificate? The best solution here is to work with the server vendor to get them to move to a certificate issued by a trusted certificate authority. If that's not possible, the next best option is to embed the server's certificate in your app and then use the approach described in Trusting One Specific Certificate to trust just that certificate.

Trust Exceptions

The bulk of this technote assumes that you're trying to resolve a server trust evaluation failure with a specific server. But what happens if you're building a general purpose application, one that can connect to a wide variety of servers? The canonical example of this is a web browser, but there many others (email programs, RSS readers, and so on). In that case you can't apply a specific solution; you need a solution that works in general.

The best general-purpose approach is to do nothing. The default server trust evaluation has two critical advantages:

  • it's easy to implement

  • it's secure

The last point is critical: if there is a server trust evaluation failure and you provide the user with a way to bypass the security, the user will almost always choose whatever option that works, no matter how insecure it is. From a security perspective it's better to just fail, and then have the user pressure the server admin into fixing their server.

If you choose to ignore this advice (and you won't be alone; many popular general-purpose apps present the user with security questions, despite the fact that they don't have the skills to answer meaningfully), you can still take steps to minimize the risk.

The overall strategy here is:

  1. try to connect to the server

  2. if that fails with a server trust evaluation failure, inform the user about the problem (see Displaying Certificates and Displaying Trust Results for details)

  3. if the user decides to connect anyway, remember that decision and continue with the connection

  4. later on, if you encounter the same server trust evaluation failure, ignore the problem and connect anyway

The tricky part is step 4. How can you tell that this is the same failure? If you get this wrong, you risk compromising the user's security above and beyond what they've agreed to. Consider the following sequence:

  1. the user connects to a server with an expired certificate

  2. your program detects this and asks the user to confirm that they really want to connect

  3. the user agrees, so your program remembers that decision and connects to the server

  4. later on the user moves to a hostile network, one that includes an active attacker that implements an impostor server; they go to connect to the server again

  5. your program connects to the impostor server; it detects the server trust evaluation failure, which is good, but it sees that the user has agreed to connect to the server despite such problems, and connects anyway

The problem here is that your program has now connected to an impostor server, even though the user only agreed to ignore an expired certificate.

You can address this issue using trust exceptions. When the user agrees to connect to a server your program can ask the trust object for a trust exception (SecTrustCopyExceptions). This records the information necessary to get around the current trust evaluation failure. In the future, when your program sees a trust evaluation failure for the same server, it can apply the exception to the trust object (SecTrustSetExceptions). If that resolves the server trust evaluation failure, it can safely continue with the connection. If not, it must ask the user to agree to an additional trust exception.

Enforcing Stricter Server Trust Evaluation

Customizing server trust evaluation is not just about working around problems; you can also use it to make security tighter. If you're working on a high-security program, you might want to go beyond the default server trust evaluation and add some more checks of your own.

For example, you might want to not only check that the server's certificate was issued by a trusted certificate authority, but that it was issued by a specific certificate authority (a technique known as certificate authority pinning). It's easy to accomplish this using a trust object. The procedure is as follows:

  1. include a copy of the certificate authority's root certificate in your program

  2. once you have the trust object, create a certificate object from the certificate data (SecCertificateCreateWithData) and then set that certificate as the only trusted anchor for the trust object (SecTrustSetAnchorCertificates)

  3. evaluate the trust object; if the evaluation succeeds, you know that the server's certificate is valid and was issued by the certificate authority that you specified

Certificate authority pinning is just one example of how you can enforce stricter server trust evaluation. There are many other checks that you might consider. For example:

Hints and Tips

This section describes some general hints and tips when dealing with customizing server trust evaluation.

SecTrustEvaluate Can Block

There are two situations where SecTrustEvaluate may need to access the network in order to complete its job:

  • when downloading intermediate certificates

  • when determining whether a certificate has been revoked (via a OCSP or CRL)

These network operations have relatively short timeouts, but they still make SecTrustEvaluate unsuitable for use on the main thread, especially on iOS. If you need to call SecTrustEvaluate from the main thread, you have three options:

  • you can use SecTrustEvaluateAsync (introduced in OS X 10.7 and iOS 7.0)

  • you can use SecTrustSetNetworkFetchAllowed to disable network access for your trust object (introduced in OS X 10.9 and iOS 7.0)

  • you can use a concurrency primitive (for example, GCD, an NSOperation, or a thread) to run SecTrustEvaluate on a secondary thread

Investigating Hard-To-Debug Trust Evaluation Failures

Sometimes it's hard to work out exactly why trust evaluation is failing. Your first step should always be to work through the points described in Common Failures. If you still can't figure out the problem, there are a few more tricks you can try:

  • print the result of SecTrustCopyResult

  • print the result of SecTrustCopyProperties

  • examine the trust exception data (as returned by SecTrustCopyExceptions)

The first two points need no further explanation. The last point is a bit tricky. If you look inside the data, you'll see that it's actually a binary property list. If you save the data to a file with the .plist extension and then open it with Xcode (or your favorite property list editor), you may find useful hints as to why the trust evaluation failed.

Displaying Certificates

In some circumstances it may be useful to display certificates to a user. For example, an advanced user might want to manually confirm the identity of the server that your app is communicating with. OS X has high-level APIs for this:

  • SFCertificateView is an NSView subclass that displays a certificate

  • SFCertificatePanel is a panel that displays one or more certificates; it can be used independently or as a sheet

In general you should use these high-level APIs but, if you want to display your own user interface, you can use low-level APIs (specifically, SecCertificateCopyValues) to extract details from the certificate and build your own UI from that.

iOS does not have a high-level API for displaying certificates. If you want to do this, you will have to build your own user interface. Doing that is non-trivial because iOS has only limited APIs for getting information from a certificate (r. 11004183) ; in fact, the only thing you can do is get a user-visible name for the certificate by calling SecCertificateCopySubjectSummary. Beyond that, you will need your own code to parse the certificate.

Displaying Trust Results

The advice given throughout this document is that you should not ask users security questions that they are not qualified to answer. If you ignore that advice then you'll probably want a way to display the results of a failed trust evaluation to the user. OS X has a high-level API for this, SFCertificateTrustPanel.

There is no equivalent high-level API on iOS, but both platforms allow you to get detailed information about trust evaluation failures. You can call SecTrustCopyResult, which returns information about the overall trust evaluation, and SecTrustCopyProperties, which returns information about each certificate in the path leading from the server certificate to the trusted anchor. You can use this information to populate your own trust results UI.

Appendix A: Common Server Trust Evaluation Errors

Table 3 lists some errors you're likely to get back in the face of server trust evaluation failures.

Table 3  Common Server Trust Evaluation Errors

Error Domain

Error Code

Error Code Value

NSURLErrorDomain

NSURLErrorSecureConnectionFailed

-1200

NSURLErrorDomain

NSURLErrorServerCertificateHasBadDate

-1201

NSURLErrorDomain

NSURLErrorServerCertificateUntrusted

-1202

NSURLErrorDomain

NSURLErrorServerCertificateHasUnknownRoot

-1203

NSURLErrorDomain

NSURLErrorServerCertificateNotYetValid

-1204

kCFErrorDomainCFNetwork

kCFURLErrorSecureConnectionFailed

-1200

kCFErrorDomainCFNetwork

kCFURLErrorServerCertificateHasBadDate

-1201

kCFErrorDomainCFNetwork

kCFURLErrorServerCertificateUntrusted

-1202

kCFErrorDomainCFNetwork

kCFURLErrorServerCertificateHasUnknownRoot

-1203

kCFErrorDomainCFNetwork

kCFURLErrorServerCertificateNotYetValid

-1204

NSOSStatusErrorDomain

errSSLXCertChainInvalid

-9807

NSOSStatusErrorDomain

errSSLUnknownRootCert

-9812

NSOSStatusErrorDomain

errSSLNoRootCert

-9813

NSOSStatusErrorDomain

errSSLCertExpired

-9814

NSOSStatusErrorDomain

errSSLCertNotYetValid

-9815

NSOSStatusErrorDomain

errSSLHostNameMismatch

-9843

Appendix B: Glossary

If you're not familiar with TLS, and specifically X.509 public key infrastructure, many of the terms used in this technote may be daunting. This glossary explains these terms and their specific meaning in this context.



Document Revision History


DateNotes
2014-07-29

Corrected the guidance in the "HTTP Live Streaming" section (r. 16538960). Fixed the column ordering in the table in Appendix A (r. 16057325).

2014-03-13

Updated for Security API changes in iOS 7 and OS X 10.9. Added the NSURLSession section. Added three news hints and tips. Updated to reference CustomHTTPProtocol sample code and TN2326. Expanded the "Enforcing Stricter Server Trust Evaluation" section. Other minor editorial changes.

2013-02-07

Clarified the nature of +setAllowsAnyHTTPSCertificate:forHost:.

2012-10-10

New document that describes how you can resolve HTTPS server trust evaluation, for example, to work with a server with a missing intermediate certificate.