"sandbox.itunes.apple.com/verifyReceipt": Connection reset

From a pure networking perspective, resetting the connection implies that something took

place in the transport of the connection that either the client or the server did not like. Thus resulting in the connection reset. Do you have any more logs on this incident that you can share? Make sure to pull out IPs or sensitive information.

Matt Eaton

DTS Engineering, CoreOS

meaton3 at apple.com

No, I don't, all I have are logs concerning "connection reset" and sometimes "connect time out".

But I'd like to notice that it mostly occurs in iPhone 8 and more rarely in 10, 11.

Our whole payment procedure is the next:

iPhone side - buy subscribsion, after payment went success, send receipt to the backend for further handling.

Backend side - get receipt data and go to "https://buy.itunes.apple.com/verifyReceipt"/"https://sandbox.itunes.apple.com/verifyReceipt" to verify that receipt and then mark user as premium.

Actually it happens durring testing with test accounts.

Do you have any thoughts on why it can happen?

No, I'm not sure why this is happening. To try and track this down further, it sounds like you are performing server side receipt validation though; if so, that is good news because you should be able to look at your access logs or error logs for more information. If you turn enable verbose logging on your server does this provide any further insight into what is happening?

Matt Eaton

DTS Engineering, CoreOS

meaton3 at apple.com

Hey Matt,

I am also hitting this issue when performing server side receipt validation.

{
  "message": "read ECONNRESET",
  "name": "Error",
  "stack": "Error: read ECONNRESET\n    at TLSWrap.onStreamRead (internal/stream_base_commons.js:111:27)",
  "config": {
    "url": "https://sandbox.itunes.apple.com/verifyReceipt",
    "method": "post",
    "data": "{\"receipt-data\":\"<MY RECEIPT DATA>",\"password\":\"<MY APP STORE SHARED SECRET>>\",\"exclude-old-transactions\":true}",
    "headers": {
      "Accept": "application/json, text/plain, */*",
      "Content-Type": "application/json;charset=utf-8",
      "User-Agent": "axios/0.19.2",
      "Content-Length": 7449
    },
    "transformRequest": [
      null
    ],
    "transformResponse": [
      null
    ],
    "timeout": 0,
    "xsrfCookieName": "XSRF-TOKEN",
    "xsrfHeaderName": "X-XSRF-TOKEN",
    "maxContentLength": -1
  },
  "code": "ECONNRESET"
}

I'm encountering this when I attempt to verify a sandbox receipt from my test server.

I only attempt to connect to the sandbox endpoint for verifyReceipt if and only if I get a successful connection to the production AND it returns status 21007 as a response status.

To me, this implies that there's an issue connecting to https://sandbox.itunes.apple.com/verifyReceipt

However, I'm not sure what other information I can provide to help debug here.

Happy to help or find more information, just let me know!

-Clark

For what it's worth, I haven't changed any code and it's working now.

EDIT: this issue is happening again

This is one of error logs I get

```

2020-04-03 01:02:36.287 ERROR 1 --- [pool-2-thread-1] o.s.s.s.TaskUtils$LoggingErrorHandler : Unexpected error occurred in scheduled task.

org.springframework.web.client.ResourceAccessException: I/O error on POST request for "https://sandbox.itunes.apple.com/verifyReceipt":Connection reset; nested exception is java.net.SocketException: Connection reset

at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:557)

at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:502)

at org.springframework.web.client.RestTemplate.postForEntity(RestTemplate.java:357)

at com.stark.paytrack.apple.SpringHttpValidationStrategy.validate(SpringHttpValidationStrategy.java:66)

at com.stark.paytrack.apple.SpringHttpValidationStrategy.validate(SpringHttpValidationStrategy.java:37)

at com.stark.paytrack.subscription.AppleSubscriptionCreator.create(AppleSubscriptionCreator.java:70)

at com.stark.paytrack.subscription.PlatformSubscriptionCreator.create(PlatformSubscriptionCreator.java:54)

at com.stark.paytrack.subscription.DefaultRecurringSubscriptionUpdater.update(DefaultRecurringSubscriptionUpdater.java:60)

at com.stark.paytrack.subscription.DefaultExpiredSubscriptionHandler.cleanupSubscription(DefaultExpiredSubscriptionHandler.java:61)

at com.stark.paytrack.subscription.DefaultExpiredSubscriptionHandler.cleanupSubscriptions(DefaultExpiredSubscriptionHandler.java:103)

at com.stark.paytrack.subscription.SubscriptionMonitor.checkForExpiredSubs(SubscriptionMonitor.java:42)

at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:607)

at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65)

at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:473)

at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)

at java.lang.Thread.run(Thread.java:748)

Caused by: java.net.SocketException: Connection reset

at java.net.SocketInputStream.read(SocketInputStream.java:197)

at java.net.SocketInputStream.read(SocketInputStream.java:122)

at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)

at sun.security.ssl.InputRecord.read(InputRecord.java:480)

at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:936)

at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1324)

at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1351)

at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1335)

at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)

at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)

at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:167)

at org.springframework.http.client.SimpleBufferingClientHttpRequest.executeInternal(SimpleBufferingClientHttpRequest.java:78)

at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48)

at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:52)

at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:541)

... 22 common frames omitted

```

but I'm not sure it'll help, to that URL I send `receipt-data` and `password`.

I remembered that it started happening approximatelly from the beginning of this year, I didn't change a code in this period.

Thanks for posting this information, this does provide more insight into what is happening. Take a look at the logs here:

  at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketException: Connection reset
  at java.net.SocketInputStream.read(SocketInputStream.java:197)
  at java.net.SocketInputStream.read(SocketInputStream.java:122)
  at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
  at sun.security.ssl.InputRecord.read(InputRecord.java:480)
  at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:936)
  at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1324)
  at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1351)
  at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1335)
  at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)

What looks like is happening here is the connection has gone up and the TLS handshake is starting. When you (the client) attempts to read what it presumes to be the ServerHello response, it's actually a reset from the server. This reset most likely is in response to your ClientHello that contains your clients support ciphers and key information. Take a look at your client certificate here and make sure that everything is valid.

Since this is happening in only 80% of the cases, do you have multiple clients performing this validation? Your actual server access logs might be able to give you more information here too.

Matt Eaton

DTS Engineering, CoreOS

meaton3 at apple.com

`Since this is happening in only 80% of the cases, do you have multiple clients performing this validation?` - no, on our test environment we have only one client for receipt validation. On production we have multiple ones, but it's not happenning there, it happens only on sandbox and this is the issue for us because we can not understand what actually the problem is and we can not release new versions of appllications for iPhone 8 and others.

And I'd like to notice that we get the "Connection reset" error only when trying to connect to https://sandbox.itunes.apple.com/verifyReceipt and not to https://buy.itunes.apple.com/verifyReceipt, this URL always responses with expected codes like 21007 or 21008 that says us that we are using test accounts.

Thanks.

Based on the information provided, my recommendation would be to take a look at your TLS / Certificate configuration in your test environment.

Matt Eaton

DTS Engineering, CoreOS

meaton3 at apple.com

Even when I do request with curl's help like this:

curl -X POST --data "password=" --data "receipt-data=" https://sandbox.itunes.apple.com/verifyReceipt

from the same instance, first time I can get "curl: (35) TCP connection reset by peer" and second time I can get response 200.

There can be 3 success requests then 1 or more errors in a row, that's why I don't think that this is the problem with the TLS certificates, because is certificates are wrong I will get "curl: (35) LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to sandbox.itunes.apple.com:443" error.

Thanks!

To help you nail this down you can use the -v flag on your curl command to see the exchange between both sides. Also, you could take a look at the packet level exchange using tcpdump / Wireshark or OpenSSL's s_client command.

Matt Eaton

DTS Engineering, CoreOS

meaton3 at apple.com

here is the response with flag -v

*   Trying 17.154.66.159...
* TCP_NODELAY set
* Connected to sandbox.itunes.apple.com (17.154.66.159) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* NSS error -5961 (PR_CONNECT_RESET_ERROR)
* TCP connection reset by peer
* stopped the pause stream!
* Closing connection 0
curl: (35) TCP connection reset by peer

and the next one

*   Trying 17.154.66.159...
* TCP_NODELAY set
* Connected to sandbox.itunes.apple.com (17.154.66.159) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* ALPN, server accepted to use http/1.1
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
* subject: CN=buy.itunes.apple.com,OU=GNCS Traffic Management,O=Apple Inc.,L=Cupertino,ST=California,C=US,serialNumber=C0806592,incorporationState=California,incorporationCountry=US,businessCategory=Private Organization
* start date: Mar 11 00:00:00 2020 GMT
* expire date: Mar 12 12:00:00 2021 GMT
* common name: buy.itunes.apple.com
* issuer: CN=DigiCert SHA2 Extended Validation Server CA-3,OU=www.digicert.com,O="DigiCert, Inc.",C=US
> POST /verifyReceipt HTTP/1.1
> Host: sandbox.itunes.apple.com
> User-Agent: curl/7.53.1
> Accept: */*
> Content-Length: 18903
> Content-Type: application/x-www-form-urlencoded
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 200 Apple WebObjects
< x-apple-jingle-correlation-key: V6NXK6GWZQRO4HE64QEUTSBLNY
< x-apple-request-uuid: af9b7578-d6cc-22ee-1c9e-e40949c82b6e
< pod: 100
< x-apple-translated-wo-url: /WebObjects/MZFinance.woa/wa/verifyReceipt
< apple-tk: false
< edge-control: no-store
< edge-control: cache-maxage=0
< x-apple-lokamai-no-cache: true
< cache-control: private
< cache-control: no-cache
< cache-control: no-store
< cache-control: no-transform
< cache-control: must-revalidate
< cache-control: max-age=0
< itspod: 100
< x-webobjects-loadaverage: 10
< apple-seq: 0
< apple-originating-system: MZFinance
< strict-transport-security: max-age=31536000
< x-frame-options: SAMEORIGIN
< x-apple-orig-url: https://sandbox.itunes.apple.com/WebObjects/MZFinance.woa/wa/verifyReceipt
< x-apple-application-site: SB
< date: Tue, 07 Apr 2020 12:52:50 GMT
< set-cookie: itspod=100; version="1"; expires=Thu, 07-May-2020 12:52:50 GMT; path=/; domain=.apple.com
< set-cookie: mzf_in=990214; version="1"; path=/WebObjects; domain=.apple.com; secure; HttpOnly
< set-cookie: mzf_dr=0; version="1"; expires=Thu, 01-Jan-1970 00:00:00 GMT; path=/WebObjects; domain=.apple.com
< apple-timing-app: 4 ms
< expires: Tue, 07 Apr 2020 12:52:50 GMT
< x-apple-application-instance: 990214
< content-length: 16
<
* Connection #0 to host sandbox.itunes.apple.com left intact

Thank you for posting these logs. First, I will say that you are correct. Something is odd here that you are receiving inconsistent results when testing :sandbox.itunes.apple.com. This usually means there is some sort of network configuration issue in the path your are traveling to the peer or on your client end. Luckily, adding the verbose flag does point out where the break down is. Take a look at these lines in the failing case:

* Initializing NSS with certpath: sql:/etc/pki/nssdb 
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt 
  CApath: none 
* NSS error -5961 (PR_CONNECT_RESET_ERROR) 
* TCP connection reset by peer

This means you are starting the secure handshake and the peer (Apple Servers) reset your connect due to NSS error -5961 (PR_CONNECT_RESET_ERROR). Dig into this error to find a resolution on your server side.

Matt Eaton

DTS Engineering, CoreOS

meaton3 at apple.com