Strange behavior with NSURLSession and ATS on macOS

Question

Created Aug ’16

Replies 5

Boosts 0

Views 1.1k

Participants 2

Yesterday we ran into a strange behavior with ATS and NSURLSession. We have a macOS application which connects to an internal network webservices over HTTPS using NSURLSession. Recently we have disabled ATS entirely via NSAllowsArbitraryLoads to "true" in Info.plist file.

<key>NSAppTransportSecurity</key>

<dict>

<key>NSAllowsArbitraryLoads</key>

<true/>

</dict>

Now when users of this application trying to connect the webservice are getting connection errors. Now the strange part is not all of the users are seeing this error. They have different versions of 10.11.x. A user with 10.11.6 sees a problem and another user with same OS version doesn't.

I am baffeled as to what external behavior/parameters controlling the NSURLSession besides ATS and why this behavior changes per machines? I suspected HSTS so I deleted the HSTS.plist file but no use (not sure if that is the right file, but it is used by Safari). I have verified ATS to be working by verifying the the ciphers offered by the clients in Client Hello. We delegate the trust evaluation and override the system defaults procedure with our own trusted anchors and it passes without any issue.

NSURLSession failed with error: Error Domain=NSURLErrorDomain Code=-1200 "An SSL error has occurred and a secure connection to the server cannot be made." UserInfo={NSURLErrorFailingURLPeerTrustErrorKey=<SecTrust 0x7fa4c4d15ea0 [0x7fff7a2b4440]>, NSLocalizedRecoverySuggestion=Would you like to connect to the server anyway?, _kCFStreamErrorDomainKey=3, _kCFStreamErrorCodeKey=-9802, NSErrorPeerCertificateChainKey=(
    "<SecCertificate 0x7fa4c4d214d0 [0x7fff7a2b4440]>"
), NSUnderlyingError=0x7fa4c2420b80 {Error Domain=kCFErrorDomainCFNetwork Code=-1200 "(null)" UserInfo={_kCFNetworkCFStreamSSLErrorOriginalValue=-9802, kCFStreamPropertySSLPeerCertificates=(
    "<SecCertificate 0x7fa4c4d214d0 [0x7fff7a2b4440]>"
), _kCFStreamPropertySSLClientCertificateState=0, kCFStreamPropertySSLPeerTrust=<SecTrust 0x7fa4c4d15ea0 [0x7fff7a2b4440]>, _kCFStreamPropertySSLClientCertificates=(
    "<SecIdentity 0x7fa4c4d14c90 [0x7fff7a2b4440]>",
    "<SecCertificate 0x7fa4c2425660 [0x7fff7a2b4440]>"
), _kCFStreamErrorDomainKey=3, _kCFStreamErrorCodeKey=-9802}}, NSErrorClientCertificateChainKey=(
    "<SecIdentity 0x7fa4c4d14c90 [0x7fff7a2b4440]>",
    "<SecCertificate 0x7fa4c2425660 [0x7fff7a2b4440]>"
), NSLocalizedDescription=An SSL error has occurred and a secure connection to the server cannot be made.,

I have verified the server to be ATS complaint by running "nscurl --ats-diagnostics", TLSTool and also attempted to debug the logs by setting CFNETWORK_DIAGNOSTICS but not much fruitful came out of it. Now, I am ran out of ideas, so need help.

Thanks in advance.

Boost

Answer 1

DTS Engineer OP

Apple

Aug ’16

Is this app specific? If you put a small test app on the device, does it show the problem? Try it with both a sandboxed and non-sandboxed test app, because the App Sandbox isolates you from many system-wide settings.

Are there any redirects involved? Use

-URLSession:task:willPerformHTTPRedirection:newRequest:completionHandler:

to check for that in your small test app.

Is this user specific? If you create a temporary test user on an affected machine, does the problem still occur?

Grab a packet trace of the problem. Exactly where in the TLS handshake do things fail?

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

0

Answer 2

invincible OP

Aug ’16

Thanks for your reply.

Is this app specific? If you put a small test app on the device, does it show the problem? Try it with both a sandboxed and non-sandboxed test app, because the App Sandbox isolates you from many system-wide settings.

Well it looks like. I don't see any issue while connecting to HTTPS host via browser. I have already tested with another test application but it shows the same problem. I am planning to write another utility which will just do NSURLSession and connect to the host.

Another thing to note here is if I provide an IP address of the host (in my actual app) instead of FQDN, the connection is successful.

The current application is non-sandboxed, so I'll sandbox it and try today.

Are there any redirects involved? Use

-URLSession:task:willPerformHTTPRedirection:newRequest:completionHandler:
to check for that in your small test app.

There are NO redirects involved.

Is this user specific? If you create a temporary test user on an affected machine, does the problem still occur?

Grab a packet trace of the problem. Exactly where in the TLS handshake do things fail?

I haven't tried with different users on same Mac, may be will try that. I have already captured packets with original application. Is there anyway to upload it here or need to put it in Dropbox or some other public share? Meanwhile, I put together a summary below.

Client Server

--- TCP SYN ---->

<-- SYN ACK ----

---- ACK ---------->

-----Client Hello---->

(Cipher Suite: TLS_EMPTY_RENEGOTIATION_INFO_SCSV (0x00ff))

(Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (0xc02c)

.... 20 more

<---Server Hello-----

(negotiated cipher: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030))

<----Certificate and Certificate Request----

----- TCP ACK ---->

----- FIN ACK -------> (This is where client rejects)

<---- FIN ACK -----

-------- ACK-------->

0

Answer 3

DTS Engineer OP

Apple

Aug ’16

Is there anyway to upload it here or need to put it in Dropbox or some other public share?

DevForums has no file sharing feature. Normally I ask folks to upload files to their sharing site of choice but in this case I don’t think that’s necessary. Your summary is sufficient. If it comes to looking at packet traces in detail I’ll have you open a DTS tech support incident so I can dedicate more time to this.

I don't see any issue while connecting to HTTPS host via browser.

OK.

I am planning to write another utility which will just do NSURLSession and connect to the host.

Yep. That’d be really useful test.

The current application is non-sandboxed, so I'll sandbox it and try today.

OK, although it might be easier to run the sandboxed vs non-sandboxed test with your the test app you’re planning rather than with your app as a whole.

There are NO redirects involved.

OK.

I haven't tried with different users on same Mac, may be will try that.

Please do; it’ll help you determine the scope of the issue.

Based on your packet trace summary it seems like server trust evaluation is failing. This can happen for a bunch of different reasons. Once you determine the scope of the issue, that’ll suggest where to look for the problem.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

0

Answer 4

invincible OP

Aug ’16

Here are some updates on the issue -

With further debugging, we found that clearing the Safari browsing history fixes the problem 😮. We ran this workaround on most of the user's machines who have reported this problem and it appears to be fixed. We had a chance to consistently reproduce this problem on the machine where it has never been seen anytime in the past.

When user connects to the URL via Safari, it often is presented with certificate warning dialog box to continue or cancel connection and in this case user cancels the connection instead of going ahead. At this time, running original application connecting to same URL causes connection failure. And cleaning up the browser cache makes everything work again!

Now I am more confused as why would Safari and NSURLSession sharing the Session/URL attributes? And is this an expected behavior or a bug? Is it safe to say that we have found the root cause of this issue? Are there any parameters on NSURLSession that I should have configured to avoid this behavior, Apple docs doesn't say anything?

Thanks you.

0

Answer 5

DTS Engineer OP

Apple

Aug ’16

With further debugging, we found that clearing the Safari browsing history fixes the problem.

Yeah, this is one of the reasons I was asking about sandboxing and redirects. In a non-sandboxed Mac app, NSURL{Session,Connection} share a lot of state with Safari. This is inherited from the original NSURLConnection architecture, where such state sharing was a desired behaviour (oh how the world has changed since 2002!).

If you sandbox your Mac app then all of that state sharing gets disabled.

Some of that state sharing—namely the three standard state holders, NSHTTPCookieStorage, NSURLCredentialStorage, and NSURLCache—can be disabled via the NSURLSessionConfiguration, but I’m not sure if that covers all the cases.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

0