Beta Testing Your iOS App

You’ve created your iOS app in Xcode and optionally added app services. You’ve tested your app on iOS Simulator and on your own devices. It’s time for beta testing. In this phase, you distribute the app to a wider audience—to give the app a “real-world” test and, in some cases, to offer users a preview of your next version.

There are three options to distribute your app for beta testing. Two prerelease options are managed through iTunes Connect. You upload your app to iTunes Connect using Xcode and then sign in to iTunes Connect to invite testers to download your app. Optionally, distribute your app to 25 of your iTunes Connect users (with either Technical, Admin, or Legal roles) or 1000 users specifying just email addresses. Distributing your app to users requires approval from App Review.

To distribute your prerelease build using iTunes Connect:

  1. Create an iTunes Connect app record.

  2. Update the build string.

  3. Archive and validate your app.

  4. Upload your app to iTunes Connect.

  5. Manage testing in iTunes Connect.

  6. Solicit crash reports and feedback from testers.

The third option is to distribute your app using an ad hoc provisioning profile that allows your app to run on up to 100 registered devices. The steps to use an ad hoc provisioning profile, described in Distributing Your Beta App Using Ad Hoc Provisioning, are slightly different than the steps to upload your app to iTunes Connect. You can register up to 100 devices, but this limit includes devices you use for development and beta testing. If you are a member of the iOS Developer Enterprise Program, you don’t have access to iTunes Connect, so use this method for beta testing your apps.

Creating Your App Record in iTunes Connect

If you’re beta testing a final candidate for a release, be sure to validate the app before distributing it to testers. The validation tests are performed by iTunes Connect, which checks whether your Xcode project is configured correctly for the store. For example, it reports a problem if you’re missing required app icons. You must have an iTunes Connect app record to validate and upload your app to iTunes Connect. To create your app record, read Creating an App Record before continuing.

Updating the Build String

If you update the build string, as described in Setting the Version Number and Build String, iTunes recognizes that the build string changed and properly syncs the new version of your app to test devices.

Archiving and Validating Your App

You create an archive of your app regardless of the type of distribution method you select. Xcode archives allow you to build your app and store it, along with critical debugging information, in a bundle that’s managed by Xcode. For example, if you distribute a build of your app to testers, archiving the debugging information makes it easier to interpret crash reports that they send you later. After your app is released, you use the same debugging information to decipher crash reports that you download from iTunes Connect.

With archives, you can upload the same build you distribute for testing that you later submit to the store. It’s important to test the exact build that’s a candidate for release. Differences between Xcode build settings can cause bugs that don’t appear during testing because the binary being tested was built differently. By having Xcode make an archive of your app, you can be sure you’re testing the exact same build of your app that you submit to the store.

Follow these steps to archive and validate your app:

  1. Review the Archive scheme settings.

  2. Create an archive of your app.

  3. Validate the archive.

Reviewing the Archive Scheme Settings

The first time you make an archive, review the Archive scheme settings to ensure that you don’t archive a debug version of your app.

To review the Archive scheme

  1. In the Xcode project editor, choose Product > Scheme > Edit Scheme to open the scheme editor.

  2. Click Archive in the column on the left.

    ../Art/6_archiveschemesettings_2x.png
  3. From the Build Configuration pop-up menu, choose Release, and click Close.

Creating an Archive

Next, create an archive of your app. Xcode stores this archive in the Archives organizer.

To create an archive

  1. In the Xcode project editor, choose iOS Device or your device name from the Scheme toolbar menu.

    You can’t create an archive of a simulator build. If an iOS device is connected to your Mac, the device name appears in the Scheme toolbar menu. When you disconnect the iOS device, the menu item changes to iOS Device.

  2. Choose Product > Archive.

    The Archives organizer appears and displays the new archive.

Xcode runs preliminary validation tests on the archive and may display a validate warning in the project editor. For example, if you are not using asset catalogs and don’t set required app icons, as described in Setting Individual App Icon Files, an Info.plist warning message appears. If you see this warning, fix the issue and create the archive again.

Running iTunes Connect Validation Tests

Because iTunes Connect validates your archive when you upload it to the store, it’s best to validate your beta version now to discover issues early.

To validate an archive

  1. In the Archives organizer, select the archive.

  2. Click the Validate button.

    ../Art/6_ios_archive_organizer_2x.png
  3. In the dialog that appears, choose a team from the pop-up menu and click Choose.

    If necessary, Xcode creates a distribution certificate and distribution provisioning profile for you. The distribution provisioning profile name beings with the text XC:.

  4. In the dialog that appears, review the app, its entitlements, and provisioning profile, and click Validate.

    Xcode uploads the archive to iTunes Connect, and iTunes Connect runs validation tests. If a dialog appears stating that no application record can be found, click Done, create an app record in iTunes Connect, and repeat these steps.

    ../Art/6_ios_review_dist_profile_2x.png
  5. Review validation issues that are found, if any, and click Done.

Fix any validation issues, create a new archive, and validate it again. Validation errors don’t prevent you from uploading your app for beta testing.

Distributing Your Prerelease Build Using iTunes Connect

Use Xcode to upload your app to iTunes Connect, and then use iTunes Connect to distribute it for beta testing or to submit it to the store.

To upload your app to iTunes Connect

  1. In the Archives organizer, select the archive.

    ../Art/6_ios_archive_organizer_2x.png
  2. Click the Submit button.

    iTunes Connect validates your archive when you upload it to the store.

  3. In the dialog that appears, choose a team from the pop-up menu and click Choose.

    If necessary, Xcode creates a distribution certificate and distribution provisioning profile for you. The name of the distribution provisioning profile begins with the text XC:.

  4. In the dialog that appears, review the app, its entitlements, and provisioning profile, and click Submit.

    Xcode uploads the archive to iTunes Connect and iTunes Connect runs validation tests. If a dialog appears stating that no application record can be found, click Done, create an app record in iTunes Connect, and repeat these steps.

  5. If issues are found, click Done and fix them before continuing.

    For a complete list of issues you should fix before distributing your app, read App Store Review Guidelines for iOS Apps. For more information on the app review process, go to App Review.

  6. If no issues are found, click Submit to upload your app.

After you upload the app, use iTunes Connect to select beta distribution options, described in TestFlight Beta Testing (Optional) in iTunes Connect Developer Guide. You specify whether you want to distribute your app to 1000 beta testers or just to selected iTunes Connect users.

Distributing Your App Using Ad Hoc Provisioning

Alternatively, distribute the app for testing on registered devices using an ad hoc provisioning profile. Using this method, testers don’t need to be team members or iTunes Connect users to run the app, but their devices need to be registered in Member Center. You can register up to 100 devices per year that your team can use for development and testing. Therefore, choose this method if you can use a portion of these devices for testing and can collect device IDs from testers. Also choose this method if you’re not ready to create an app record in iTunes Connect. You don’t need to validate or upload your app to iTunes Connect to distribute it using an ad hoc provisioning profile.

The steps are similar to iTunes Connect beta testing, except that you register test devices, and instead of uploading your app to iTunes Connect, you export an archive as an iOS App Store Package and distribute it for testing:

  1. Optionally, create an iTunes Connect app record.

  2. Register all test devices.

  3. Update the build string.

  4. Archive and validate your app.

  5. Export the archive as an iOS App Store Package.

  6. Install the app on test devices.

  7. Solicit crash reports from testers.

To update the build string, read Updating the Build String, and to archive and optionally validate your app, read Archiving and Validating Your App. This section contains the steps that are different for ad hoc distribution.

About Ad Hoc Provisioning Profiles

An ad hoc provisioning profile is a distribution provisioning profile for iOS apps that allows your app to be installed on designated devices and to use app services without the assistance of Xcode. It’s one of the two types of distribution provisioning profiles that you can create for iOS apps. (You use the other type of distribution provisioning profile later to submit your app to the store.) An ad hoc provisioning profile ensures that test versions of your app aren’t copied and distributed without your knowledge.

When you’re ready to distribute your app to testers, you create an ad hoc provisioning profile specifying an App ID that matches one or more of your apps, a set of test devices, and a single distribution certificate.

../Art/ad_hoc_provisioning_2x.png

Each iOS device in an ad hoc provisioning profile is identified by its unique device ID (UDID). The devices you register and add to a provisioning profile are stored by Member Center. Each individual or company can register up to 100 devices per membership year for development and testing.

Sign the iOS App Store Package containing your app by using the distribution certificate specified in the ad hoc provisioning profile, and distribute iOS App Store Package to testers.

Registering Test Devices

Register one or more test devices before you create an ad hoc provisioning profile. To register test devices, collect device IDs from testers and add them to Member Center.

Testers can get their device ID using iTunes. (They don’t need to install Xcode to do this.) Send the instructions in Locating iOS Device IDs Using iTunes to testers and ask them to send their device IDs to you, or follow these steps to collect your own device IDs.

In Member Center, register one or more devices, as described in Registering Devices Using Member Center.

Creating an iOS App Store Package for Ad Hoc Deployment

Because testers don’t have Xcode to run your app, you create an iOS App Store Package that they can then use to install your app on their device. Use Xcode to create an archive, and from the archive generate an iOS App Store Package (a file with a .ipa filename extension).

To create an iOS App Store Package for ad hoc deployment

  1. In the Archives organizer, select the archive.

    ../Art/6_ios_archive_organizer_2x.png
  2. Optionally, click the Validate button.

    If you’re close to submitting your app to the store, validate it now and fix any problems before distributing it for final testing. For a complete list of issues that should be fixed before you distribute your app, read App Store Review Guidelines for iOS Apps. For more information on the app review process, go to App Review. Click the Next button to continue.

  3. Otherwise, click the Export button, select “Save for Ad Hoc Deployment,” and click Next.

    ../Art/6_ios_createappstorepackage_1_2x.png
  4. In the dialog that appears, choose a team from the pop-up menu and click Choose.

    If necessary, Xcode creates a distribution certificate and an ad hoc provisioning profile for you. The name of the ad hoc provisioning profile begins with the text XC Ad Hoc:.

    ../Art/6_ios_export_choose_team_2x.png
  5. In the dialog that appears, review the app, its entitlements, and the provisioning profile, and click Export.

  6. Enter a filename and location for the iOS App Store Package file, and click Export.

    The file has an .ipa extension.

Installing Your App on Test Devices

Before you distribute your app to testers, follow the same steps that testers use to install and run the app on their devices. Use iTunes to install the app on a nondevelopment device. iOS extracts the embedded ad hoc provisioning profile in your app and installs it on the device for you. Then test your app on the device.

To install the app on a device

  1. Connect the testing device to a Mac running iTunes.

    If possible, don’t use a Mac that you use for development.

  2. Double-click the iOS App Store Package file that you created earlier (the file with the .ipa extension).

  3. In iTunes, click the device button in the upper-right corner of the window.

  4. Click the Apps button.

    The app appears in the iTunes Apps list.

    ../Art/5_ios_installapp_1_2x.png
  5. Under Apps, choose “Sort by Name” or “Sort by Kind” from the pop-up menu.

    An Install or Remove button appears adjacent to the app.

  6. If an Install button appears, click it.

    The button text changes to Will Install.

  7. Click the Apply button or the Sync button in the lower-right corner to sync the device.

    The app is uploaded to the device so that the user can start testing.

Finally, send the iOS App Store Package file to testers, along with the app installation instructions and the crash report instructions, as described in Soliciting Crash Reports from Testers.

Distributing Your App Using the Xcode Service

After you perform the steps in this chapter to create an ad hoc provisioning profile and export your app for beta testing, you can optionally set up an Xcode service to distribute your app. The Xcode service, available in OS X Server, automates the integration process of building, analyzing, testing, and archiving your app. The Xcode service also hosts a website that facilitates the distribution of product builds and archives to testers and other team members. See Xcode Continuous Integration Guide for more information about using the Xcode service to distribute versions of your app to testers.

Copying App Sandbox Data

To further diagnose an issue with your app, you can view, download, and replace the contents of an app container on an iOS device, as described in Viewing, Downloading, and Replacing App Containers on iOS Devices.

Soliciting Crash Reports from Testers

When an app crashes on a device, iOS creates a record of that event. The next time the tester connects the iOS device to iTunes, iTunes downloads those records (known as crash logs) to the tester’s Mac. Testers should send these crash logs to you along with any bug reports. Later, after your app is released, you can also retrieve crash reports of your live app from iTunes Connect.

Tell testers how to retrieve crash reports from their devices and send them to you.

To send crash reports from a Mac

  1. Connect the testing device to a Mac running iTunes.

    iTunes downloads the crash reports to your Mac.

  2. In the Finder, choose Go > Go to Folder.

  3. Enter ~/Library/Logs/CrashReporter/MobileDevice.

  4. Open the folder identified by your device’s name.

  5. Select the crash logs named after the app you’re testing.

  6. Choose Finder > Services > Mail > Send File.

  7. In the New Message window, enter the developer’s address in the To field and appropriate text in the Subject field.

  8. Choose Message > Send.

  9. To avoid sending duplicate reports later, delete the crash reports you’ve sent.

To send crash reports from Windows

  1. In the Windows search field, enter the crash log directory for your operating system, replacing <user_name> with your Windows user name.

    • For crash log storage on Windows, type:

      C:\Users\<user_name>\AppData\Roaming\Apple computer\Logs\CrashReporter/MobileDevice

    • For crash log storage on Windows XP, type:

      C:\Documents and Settings\<user_name>\Application Data\Apple computer\Logs\CrashReporter

  2. Open the folder named after your device’s name, and send the crash logs for the app you’re testing in an email message using the subject-text format <app_name> crash logs from <your_name> (for example, MyTestApp crash logs from Anna Haro) to the app’s developer.

To learn how to interpret the reports that you receive from testers, read Analyzing Crash Reports.

Ad Hoc Provisioning Profiles in Depth

You signed the iOS App Store Package containing your app using the distribution certificate specified in the ad hoc provisioning profile. The ad hoc provisioning profile was included in the app bundle when you built and archived the app. Then you installed the app on a test device. The app successfully launches if the app’s bundle ID matches the App ID, the signature matches the distribution certificate, and the device is in the device list of the ad hoc provisioning profile.

../Art/AdHocProfileLaunch_2x.png

Recap

In this chapter, you learned how to distribute your iOS app for beta testing using three different methods. You learned how to upload an archive of your app to iTunes Connect for distribution to iTunes Connect users or a group of users with email addresses. You also learned how to distribute your app on designated test devices using ad hoc provisioning profiles. You also received instructions to send to ad hoc testers to install the beta version of your app and send crash reports to you.