Sandbox testing broken in iOS 11

Question

Created Dec ’17

Replies 3

Boosts 0

Views 1.8k

Participants 2

We've had several people test this on dozens of iOS 11 devices and consistently get the same results - sandbox testing for auto-renewing IAP subscriptions does not work correctly in iOS 11.

These are the official directions from Apple. Here I have modified them for TestFlight:

1. Create a sandbox user from iTunes Connect

2. Launch TestFlight and download the app that needs testing

3. Go into Settings and log out of the App Store

4. Launch the app and try to make an in-app purchase

5. When prompted, log in with the sandbox account from step 1

6. When finished testing, go into Settings and log out of the App Store. Then log back in to your primary Apple ID

Step 5 does not work. When the user reaches step 5, there are two possible outcomes:

A. iOS simply asks them to enter the password for their primary Apple ID, not giving them the option to log in using a sandbox account

B. iOS asks them to log in to the App Store and gives them the opportunity to enter the credentials for the sandbox account. However, after they do this, another dialog pops up asking them to enter the password for their primary Apple ID.

In either case, sandbox subscription purchases are applied to the user's regular Apple ID, not a sandbox tester ID. Because of this, we are unable to do any sandbox testing.

Boost

Answer 1

PBK OP

Dec ’17

You wrote "these are the oficial directions from Apple" connected to a link and then you wrote in your post something different from what is in the link. The link refers to testing devices that are running code from Xcode on your Mac. See that #3 in those directions in the link - they say "connect to your Mac".

So the question is, how do you test with TestFlight? You use the user's regular iTunes Account username. You see, when a user runs an app from TestFlight, TestFlight automatically creates a sandbox account with the same username/password as their real account.

0

Answer 2

kevin_f OP

Dec ’17

Thanks for the tip, PBK. I did not think the testing process would be much different when using TestFlight, but it sounds like I was mistaken. From doing a bit more research using the information you've provided, it sounds like Apple has not documented the automatic sandbox feature, but that it works as you've described.

This seems to me like a crippling flaw in the testing process. Our QA team has always tested through TestFlight. Some of them don't even own Macs, and I don't think it's feasible to have them all do tethered testing through Xcode. When each tester is limited to a single automatic sandbox account, the amount of testing they can do is very limited.

0

Answer 3

PBK OP

Dec ’17

External testers are meant to be actual user-type people. So there is an upside to having them use their 'peudo-real' iTunes Account. Albeit, they get to 'test' the IAPs only once per user.

0