TLS Security Guarantees

HTTPS is defined by RFC 2818 but, for the most part, it consists of a simple composition of two existing protocols:

• HTTP (as defined by RFC 2616 Hypertext Transfer Protocol -- HTTP/1.1 and others)

• TLS (as defined by RFC 5246 The Transport Layer Security (TLS) Protocol Version 1.2 and others), or the older SSL protocol

HTTPS inherits its security guarantees from TLS. By default these are:

• on-the-wire privacy — This guarantees that the data is secure from a passive attacker (someone who can see but can't modify the packets exchanged between the client and server).

• client-authenticates-server authentication — This protects the client from an active attacker (someone who can both see and modify the packets exchanged between the client and the server). Most importantly, it protects the client from various forms of man-in-the-middle attack, including impostor servers (servers who pretend to be the real server and thus acquire confidential information from the client).

TLS server trust evaluation is the part of TLS that implements client-authenticates-server authentication. If you disable it, you lose this second, extremely important security guarantee.

Resolving Server Trust Evaluation Failures

If disabling server trust evaluation completely is a mistake, what should you do? Your first step should be to diagnose the problem. To do this, read Understanding Server Trust Evaluation Failures. Once you understand the problem you'll likely find that the easiest solution is to fix the server. This reduces the amount of code you have to write and guarantees the user's security.

If it's not possible to fix the server, you will need to customize the server trust evaluation to allow the connection to proceed without completely undermining the user's security. Basic Trust Customization explains the general process for customizing server trust evaluation; it's followed by Trust Customization for Specific APIs which explains the process for various commonly-used APIs. Finally, Resolving Specific Server Trust Evaluation Failures is a discussion of how to work around specific server trust evaluation problems.

You can also customize server trust evaluation for the opposite reason, that is, to make the connection more secure. See Enforcing Stricter Server Trust Evaluation for a discussion of this.

Trust Evaluation Fundamentals

TLS server trust evaluation consists of two basic steps:

1. basic X.509 certificate trust evaluation

2. TLS-specific additions

To understand the first step, you need to know a little bit about X.509 certificates. An X.509 certificate has five critical attributes:

• information about the subject

• information about the issuer

• information about the certificate itself (for example, the valid date range, which is the range of dates for which the certificate is valid)

• a public key

• a digital signature

If the digital signature is valid then the certificate is a guarantee from the issuer that the subject holds the private key associated with the public key in the certificate. For example, the certificate for DevForums (https://devforums.apple.com/) is (at the time of writing) issued by Entrust, and by signing that certificate Entrust is guaranteeing that the maintainers of the DevForums server hold the private key associated with the public key in the certificate.

X.509 certificate trust evaluation is a recursive two-step process:

1. check the validity of the certificate itself — This involves various things, but the two most important are a) verifying the digital signature, and b) checking that the verify date (typically the current date) is within the certificate's valid date range.

2. check the validity of the issuer — This involves finding the issuer's certificate and (recursively) checking its validity.

Clearly this recursive process must terminate eventually. Trust evaluation can succeed in only one case: if it hits a trusted anchor. A trusted anchor is a certificate that the system trusts implicitly, typically because it's the root certificate of a well-known certificate authority that has been baked in to the system.

On the other hand, trust evaluation can fail for a variety of reasons:

• if it hits an invalid certificate

• if it can't find the certificate for an issuer

• if it hits a self-signed certificate that is not a trusted anchor

If X.509 trust evaluation is successful, the system then applies additional TLS-specific checks. In practice this involves checking that the DNS name that you are attempting to connect to matches the DNS name in the certificate. There are, however, a few wrinkles:

• Typically you would expect to find this DNS name in the subject's Common Name field, but it can also be in a Subject Alternative Name extension. If a name is present in the Subject Alternative Name extension, it takes priority over the Common Name.

• The Subject Alternative Name extension may also contain IP addresses, which the client consults if it connected via an IP address rather than a DNS name.

• The DNS name in the certificate might be a wildcard name, for example, "*.apple.com".

• The Extended Key Usage extension is expected to include the Server Authentication value.

If you're interested in the details of these TLS-specific checks, see RFC 2818.

Common Failures

With the above in mind, it's easy to understand the various server trust evaluation failures that you're likely to see. These include:

• missing issuer certificate — For any given certificate (except the trusted anchor), the system must be able to locate the certificate of the issuer.

• date problems — For any given certificate, the verify date must be within the certificate's valid date range.

• self-signed certificate — For any given certificate, if the certificate is self-signed, it will cause evaluation to fail (unless it's a trusted anchor).

• no trusted anchor — The system must be able to follow the path of issuer certificates leading to a trusted anchor.

• DNS name mismatch — The DNS name that you're trying to connect to must match the name in the server certificate, as described in the previous section.

If, after investigating each of the points above, you still can't work out why server trust evaluation is failing, you might want to try the technique described in Investigating Hard-To-Debug Trust Evaluation Failures.

Debugging Tools

There are a variety of tools you can use to debug server trust evaluation problems, and this section discusses some of the more common ones.

$ openssl s_client -connect www.apple.com:443

CONNECTED(00000003)

[...]

GET / HTTP/1.1

Host: www.apple.com

HTTP/1.1 200 OK

Server: Apache/2.2.3 (Oracle)

Content-Length: 9464

Content-Type: text/html; charset=UTF-8

ntCoent-Length: 9516

Cache-Control: max-age=47

Expires: Mon, 25 Jun 2012 16:18:24 GMT

Date: Mon, 25 Jun 2012 16:17:37 GMT

Connection: keep-alive

<!DOCTYPE html>

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-US" lang="en-US">

[...]

</html>

closed

$

$ openssl s_client -showcerts -host www.apple.com -port 443

[...]

---

Certificate chain

 0 s:/C=US/L=Cupertino/O=Apple Inc./ST=CALIFORNIA/CN=www.apple.com

   i:/C=US/O=Akamai Technologies Inc/CN=Akamai Subordinate CA 3

-----BEGIN CERTIFICATE-----

MIIDJzCCApCgAwIBAgIOAQAAAAABNwl6mQWT01EwDQYJKoZIhvcNAQEFBQAwUTEL

MAkGA1UEBhMCVVMxIDAeBgNVBAoTF0FrYW1haSBUZWNobm9sb2dpZXMgSW5jMSAw

HgYDVQQDExdBa2FtYWkgU3Vib3JkaW5hdGUgQ0EgMzAeFw0xMjA1MDExNzUzNDNa

Fw0xMzA1MDExNzUzNDNaMGMxCzAJBgNVBAYTAlVTMRIwEAYDVQQHEwlDdXBlcnRp

bm8xEzARBgNVBAoTCkFwcGxlIEluYy4xEzARBgNVBAgTCkNBTElGT1JOSUExFjAU

BgNVBAMTDXd3dy5hcHBsZS5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK

AoIBAQCdJlExG8umAtxL9Df/10diaYoqFeeVDbU13cH0KNxq2vV0nSb3dROtUAig

/wXj6jFU16fhfFegjcYBkYL9qiUkIiROMNo9r1IzZX4Yv9HT30vBdRMZkuIdy9eP

m9nctNVYyBGRJapem1c+llxAJzToiuQofBY6L6K2dO7nuGm7AZ/PwcbyxZEWoh0E

2SKMbMD9vG0Dph+rZDcPxENFa401j95ZiyyDinHXPliPVGimQmLeaWXMOhJSGXcd

FidodxJCGPUIMuQnxipIH8EAiOg76aKsFvi/7ctoYtRGsXJZqRLnKJ2JVAq+tQxj

6so8vaO5I5wqjyCa+ECJJ0i8Vhp/AgMBAAGjbDBqMDkGA1UdHwQyMDAwLqAsoCqG

KGh0dHA6Ly9jcmwuZ2xvYmFsc2lnbi5uZXQvQWthbWFpU3ViMy5jcmwwHQYDVR0O

BBYEFK5QfqCwo5h4aWa7DmhIJdMZ50FjMA4GA1UdDwEB/wQEAwIFIDANBgkqhkiG

9w0BAQUFAAOBgQCMfDikw5AwrCCCkhcb+ak5oTRmhV88mL5Pk7SzVTbMdCoaktOD

+Bu7iX0OYsISOjYu0x2CzX2VQ5kP5NhA7fqXOiq4iG1G/Ae+xW01lUB1gJ7VUwoX

9LabdT6c812EOMpza4lrnLqnOsiSCDf1SWv0Lo+pMkZ9Ka9EbSd3DqUEHw==

-----END CERTIFICATE-----

 1 s:/C=US/O=Akamai Technologies Inc/CN=Akamai Subordinate CA 3

   i:/C=US/O=GTE Corporation/OU=GTE CyberTrust Solutions, Inc./CN=GTE CyberTrust Global Root

-----BEGIN CERTIFICATE-----

MIIDxzCCAzCgAwIBAgIEBAAEAzANBgkqhkiG9w0BAQUFADB1MQswCQYDVQQGEwJV

[...]

bZ14M6VWhKYouKEZnaAsSCe+XHsF0haUfOnxpj4p7CZj/DnGZVB8Uh92ORa0lyY5

q44d/bV6wDodO38=

-----END CERTIFICATE-----

---

[...]

WebView

OS X's WebView gives you access to NSURLConnection authentication challenges via its resourceLoadDelegate property. You can respond to authentication challenges in the same way that you would with NSURLConnection. Alas, it's not possible to access NSURLAuthenticationMethodServerTrust authentication challenges via this mechanism (r. 9475067) . You can work around this limitation using an NSURLProtocol subclass, as illustrated by Sample Code 'CustomHTTPProtocol'.

UIWebView

UIWebView does not provide any way for an app to customize its HTTPS server trust evaluations (r. 10131336) . You can work around this limitation using an NSURLProtocol subclass, as illustrated by Sample Code 'CustomHTTPProtocol'.

HTTP Live Streaming

HTTP Live Streaming's support for server trust evaluation depends on the type of resource being fetched, as shown in Table 1.

For general information about HTTP Live Streaming, see HTTP Live Streaming Overview. For specific information on how to get HTTP Live Streaming to call your code to fetch keys—at which point you can use whatever technique you want to access them, including NSURLSession with a customized HTTPS server trust evaluation—watch WWDC 2011 Session 408 HTTP Live Streaming Update.

Also, AV Foundation allows you to implement your own resource loading, at which point you can use any network API to load these resources (and thus customize HTTPS server trust evaluation using that API). Specifically, you can set a resource loader (AVAssetResourceLoader) on an asset (AVAsset) and then use its delegate to override the loading of non-media segment resources (see Table 1). Support for this technique was introduced in iOS 6.0 and OS X 10.9.

NSURLSession

NSURLSession allows you to customize HTTPS server trust evaluation by implementing the -URLSession:didReceiveChallenge:completionHandler: delegate method. To customize HTTPS server trust evaluation, look for a challenge whose protection space has an authentication method of NSURLAuthenticationMethodServerTrust. For those challenges, resolve them as described below. For other challenges, the ones that you don't care about, call the completion handler block with the NSURLSessionAuthChallengePerformDefaultHandling disposition and a NULL credential.

When dealing with the NSURLAuthenticationMethodServerTrust authentication challenge, you can get the trust object from the challenge's protection space by calling the -serverTrust method. After using the trust object to do your own custom HTTPS server trust evaluation, you must resolve the challenge in one of two ways:

• If you want to deny the connection, call the completion handler block with the NSURLSessionAuthChallengeCancelAuthenticationChallenge disposition and a NULL credential.

• If you want to allow the connection, create a credential from your trust object (using +[NSURLCredential credentialForTrust:]) and call the completion handler block with that credential and the NSURLSessionAuthChallengeUseCredential disposition.

NSURLConnection

NSURLConnection allows you to customize HTTPS server trust evaluation in much the same way as NSURLSession. There are two significant differences:

• You implement a different delegate method (in this case -connection:willSendRequestForAuthenticationChallenge:).

• When resolving a challenge, you must get the sender from the challenge (via the -sender method) and then call the appropriate method on that sender:

  • for challenges that you don't care about, call -performDefaultHandlingForAuthenticationChallenge:

  • if you want to deny a connection, call -cancelAuthenticationChallenge:

  • if you want to allow a connection, call -useCredential:forAuthenticationChallenge:, passing in a credential created with a trust object as described in NSURLSession

CFHTTPStream

You can customize the HTTPS server trust evaluation for a CFHTTPStream in the same way you do for a CFSocketStream (see the next section). There is, however, one serious gotcha: by the time you're in a position to get a trust object in order to apply your custom trust evaluation, the CFHTTPStream has already sent your HTTP request to the server. So if the HTTP request is potentially confidential, this technique is not appropriate. In that case you should either move up a layer, and use NSURLSession, or down a layer, and use CFSocketStream directly.

CFSocketStream

To customize the TLS server trust evaluation for a CFSocketStream, you should do the following:

1. use the kCFStreamSSLValidatesCertificateChain entry of the kCFStreamPropertySSLSettings property to disable server trust evaluation completely

2. once the stream has connected, but before you send any data or trust any data you receive, get the trust object from the stream via the kCFStreamPropertySSLPeerTrust property

3. use that trust object to implement whatever custom server trust evaluation you desire

4. either continue with the connection or close it, depending on your server trust evaluation

The primary gotcha here relates to step 2: when is the stream connected? It turns out that the trust object is not available at the time you get the stream open-completed event. To guarantee that the trust object is available, you must wait for the has-space-available event or the has-bytes-available event. See Table 2 for the details.

Secure Transport

Secure Transport is the lowest-level TLS implementation on both OS X and iOS and, as you might expect, it has good support for custom TLS server trust evaluation. The procedure is as follows:

1. before starting the connection, call SSLSetSessionOption to set kSSLSessionOptionBreakOnServerAuth

2. if necessary, as discussed below, call SSLSetEnableCertVerify to disable the default server trust evaluation

3. run the Secure Transport handshake as per usual

4. when SSLHandshake returns errSSLServerAuthCompleted, call SSLCopyPeerTrust to get a trust object for the connection

5. use that trust object to implement whatever custom server trust evaluation you desire

6. either continue the Secure Transport handshake or shut down the connection

Server Name Failures

If server trust evaluation fails because the server's DNS name does not match the DNS name in the certificate, you can work around the failure by overriding the DNS name in the trust object. To do this, call SecPolicyCreateSSL to create a new policy object with the correct server name and then call SecTrustSetPolicies to apply it to the trust object.

Missing Intermediate Certificates

There are a number of ways that you can resolve a server trust evaluation failure caused by a missing intermediate certificate:

• The best approach is, as always, to fix the server. TLS requires that the server provide the client all certificates leading from the server certificate to the trusted anchor certificate, including any intermediate certificates (see Section 7.4.2 of RFC 5246). In many cases the absence of an intermediate certificate is just a minor oversight that the server admin can quickly correct.

• On OS X, if your program (or the user) adds the intermediate certificate to any keychain in the keychain search list, the trust object will use it automatically.

• On iOS, if your app adds the intermediate certificate to its keychain, the trust object will use it automatically.

• If all else fails, your program can obtain the intermediate certificate (bundle it with the program or download it from the Internet) and then create a new trust object that includes the union of the certificates from the existing trust object and your extra intermediate certificate. See Listing 5 for an example of this.

Trusting One Specific Certificate

In some cases it's useful to treat a certificate as a simple identity token. For example, in a peer-to-peer program, X.509 trust evaluation is pointless because there is no central authority to issue certificates. However, you can still use TLS to securely communicate within that architecture. The procedure is as follows:

1. get a copy of the remote peer's certificate via a trusted channel of your own devising; you could have the remote user email you their certificate, put it on a USB stick, or whatever

2. get the server certificate from the trust object (pass an index of 0 to SecTrustGetCertificateAtIndex)

3. get the data for that server certificate (SecCertificateCopyData)

4. compare this to the certificate data you got in step 1; if they match, you're talking to the correct peer

Custom Certificate Authority

If your server's certificate is issued by a certificate authority that is not trusted by the system by default, you can resolve the resulting server trust evaluation failure by including the certificate authority's root certificate in your program. The procedure is as follows:

1. include a copy of the certificate authority's root certificate in your program

2. once you have the trust object, create a certificate object from the certificate data (SecCertificateCreateWithData) and then set that certificate as a trusted anchor for the trust object (SecTrustSetAnchorCertificates)

3. SecTrustSetAnchorCertificates sets a flag that prevents the trust object from trusting any other anchors; if you also want to trust the default system anchors, call SecTrustSetAnchorCertificatesOnly to clear that flag

4. evaluate the trust object as per usual

Self-Signed Certificates

Dealing with self-signed certificates is problematic because there are a variety of different reasons why people use a self-signed certificate. This section covers the most common cases.

Trust Exceptions

The bulk of this technote assumes that you're trying to resolve a server trust evaluation failure with a specific server. But what happens if you're building a general purpose application, one that can connect to a wide variety of servers? The canonical example of this is a web browser, but there many others (email programs, RSS readers, and so on). In that case you can't apply a specific solution; you need a solution that works in general.

The best general-purpose approach is to do nothing. The default server trust evaluation has two critical advantages:

• it's easy to implement

• it's secure

The last point is critical: if there is a server trust evaluation failure and you provide the user with a way to bypass the security, the user will almost always choose whatever option that works, no matter how insecure it is. From a security perspective it's better to just fail, and then have the user pressure the server admin into fixing their server.

If you choose to ignore this advice (and you won't be alone; many popular general-purpose apps present the user with security questions, despite the fact that they don't have the skills to answer meaningfully), you can still take steps to minimize the risk.

The overall strategy here is:

1. try to connect to the server

2. if that fails with a server trust evaluation failure, inform the user about the problem (see Displaying Certificates and Displaying Trust Results for details)

3. if the user decides to connect anyway, remember that decision and continue with the connection

4. later on, if you encounter the same server trust evaluation failure, ignore the problem and connect anyway

The tricky part is step 4. How can you tell that this is the same failure? If you get this wrong, you risk compromising the user's security above and beyond what they've agreed to. Consider the following sequence:

1. the user connects to a server with an expired certificate

2. your program detects this and asks the user to confirm that they really want to connect

3. the user agrees, so your program remembers that decision and connects to the server

4. later on the user moves to a hostile network, one that includes an active attacker that implements an impostor server; they go to connect to the server again

5. your program connects to the impostor server; it detects the server trust evaluation failure, which is good, but it sees that the user has agreed to connect to the server despite such problems, and connects anyway

The problem here is that your program has now connected to an impostor server, even though the user only agreed to ignore an expired certificate.

You can address this issue using trust exceptions. When the user agrees to connect to a server your program can ask the trust object for a trust exception (SecTrustCopyExceptions). This records the information necessary to get around the current trust evaluation failure. In the future, when your program sees a trust evaluation failure for the same server, it can apply the exception to the trust object (SecTrustSetExceptions). If that resolves the server trust evaluation failure, it can safely continue with the connection. If not, it must ask the user to agree to an additional trust exception.

SecTrustEvaluate Can Block

There are two situations where SecTrustEvaluate may need to access the network in order to complete its job:

• when downloading intermediate certificates

• when determining whether a certificate has been revoked (via a OCSP or CRL)

These network operations have relatively short timeouts, but they still make SecTrustEvaluate unsuitable for use on the main thread, especially on iOS. If you need to call SecTrustEvaluate from the main thread, you have three options:

• you can use SecTrustEvaluateAsync (introduced in OS X 10.7 and iOS 7.0)

• you can use SecTrustSetNetworkFetchAllowed to disable network access for your trust object (introduced in OS X 10.9 and iOS 7.0)

• you can use a concurrency primitive (for example, GCD, an NSOperation, or a thread) to run SecTrustEvaluate on a secondary thread

Investigating Hard-To-Debug Trust Evaluation Failures

Sometimes it's hard to work out exactly why trust evaluation is failing. Your first step should always be to work through the points described in Common Failures. If you still can't figure out the problem, there are a few more tricks you can try:

• print the result of SecTrustCopyResult

• print the result of SecTrustCopyProperties

• examine the trust exception data (as returned by SecTrustCopyExceptions)

The first two points need no further explanation. The last point is a bit tricky. If you look inside the data, you'll see that it's actually a binary property list. If you save the data to a file with the .plist extension and then open it with Xcode (or your favorite property list editor), you may find useful hints as to why the trust evaluation failed.

Displaying Certificates

In some circumstances it may be useful to display certificates to a user. For example, an advanced user might want to manually confirm the identity of the server that your app is communicating with. OS X has high-level APIs for this:

• SFCertificateView is an NSView subclass that displays a certificate

• SFCertificatePanel is a panel that displays one or more certificates; it can be used independently or as a sheet

In general you should use these high-level APIs but, if you want to display your own user interface, you can use low-level APIs (specifically, SecCertificateCopyValues) to extract details from the certificate and build your own UI from that.

iOS does not have a high-level API for displaying certificates. If you want to do this, you will have to build your own user interface. Doing that is non-trivial because iOS has only limited APIs for getting information from a certificate (r. 11004183) ; in fact, the only thing you can do is get a user-visible name for the certificate by calling SecCertificateCopySubjectSummary. Beyond that, you will need your own code to parse the certificate.

Displaying Trust Results

The advice given throughout this document is that you should not ask users security questions that they are not qualified to answer. If you ignore that advice then you'll probably want a way to display the results of a failed trust evaluation to the user. OS X has a high-level API for this, SFCertificateTrustPanel.

There is no equivalent high-level API on iOS, but both platforms allow you to get detailed information about trust evaluation failures. You can call SecTrustCopyResult, which returns information about the overall trust evaluation, and SecTrustCopyProperties, which returns information about each certificate in the path leading from the server certificate to the trusted anchor. You can use this information to populate your own trust results UI.