Beta Testing Your iOS App

You’ve created your iOS app in Xcode and added key technologies and services. You’ve tested your app on iOS Simulator and your own devices. It’s time for beta testing. In this phase, you distribute the app to a wider audience—to give the app a “real-world” test and, in some cases, to offer testers a preview of your next version.

To distribute your app for beta testing:

  1. Optionally, create an iTunes Connect app record.

  2. Register all test devices.

  3. Create a distribution certificate.

  4. Create an ad hoc provisioning profile.

  5. Update the build string.

  6. Archive and validate your app.

  7. Create an iOS App Store Package.

  8. Install the app on test devices.

  9. Distribute your app to testers.

  10. Solicit crash reports from testers.

About Ad Hoc Provisioning Profiles

An ad hoc provisioning profile is a distribution provisioning profile for iOS apps that allows your app to be installed on designated devices and use key technologies and services without the assistance of Xcode. It’s one of the two types of distribution provisioning profiles you can create for iOS apps. (You use the other type of distribution provisioning profile later to submit your app to the store.) An ad hoc provisioning profile ensures that test versions of your app aren’t copied and distributed without your knowledge.

When you’re ready to distribute your app to testers, you create an ad hoc provisioning profile specifying an App ID that matches one or more of your apps, a set of test devices, and a single distribution certificate.

../Art/ad_hoc_provisioning_2x.png

Each iOS device in an ad hoc provisioning profile is identified by its unique device ID (UDID). The devices you register and add to a provisioning profile are stored by Member Center. Each individual or company can register up to 100 devices per membership year for development and testing.

You sign the iOS App Store Package containing your app using the distribution certificate specified in the ad hoc provisioning profile and distribute iOS App Store Package to testers.

Creating Your App Record in iTunes Connect

If you’re beta testing a final candidate for a release, be sure to validate the app before distributing it to testers. The validation tests are performed by iTunes Connect, which checks whether your Xcode project is configured correctly for the store. For example, it reports a problem if you’re missing required app icons. Your iTunes Connect app record needs to be in the “Waiting for Upload” or later state to validate the archive. To create your app record and change it to the “Waiting for Upload” state, read “Creating an App Record” before continuing.

Registering Test Devices

You must have one or more registered devices to create an ad hoc provisioning profile. To register test devices, collect device IDs from testers and add them to Member Center.

Testers can get their device ID using iTunes. (They don’t need to install Xcode to do this.) Send the instructions in “Locating iOS Device IDs Using iTunes” to testers and ask them to send their device IDs to you, or follow these steps to collect your own device IDs.

In Member Center, register one or more devices, as described in “Registering Devices Using Member Center.”

Creating Distribution Certificates

You must create a store distribution certificate before you can create an ad hoc provisioning profile. If you refresh provisioning profiles, as described in “Refreshing Provisioning Profiles in Xcode,” Xcode may display a dialog asking if it should request a distribution certificate on your behalf. If you don’t click Request when this dialog appears, you can request specific types of certificates using Accounts preferences in Xcode, as described in “Requesting Signing Identities.” For iOS apps, request an iOS Distribution certificate. For Mac apps, request both a Mac App Distribution and a Mac Installer Distribution certificate.

Creating Ad Hoc Provisioning Profiles

Now you’re ready to create an ad hoc provisioning profile, which allows testers to run your app on their device without needing Xcode. To create an ad hoc provisioning profile, you select an App ID, a single distribution certificate, and multiple test devices.

bullet
To create an ad hoc provisioning profile
  1. In Certificates, Identifiers & Profiles, select Provisioning Profiles.

  2. Click the Add button (+) in the upper-right corner.

  3. Select Ad Hoc as the distribution method, and click Continue.

    ../Art/6_ios_createdistributionprofile_2_2x.png
  4. Choose the App ID you used for development, that matches your bundle ID, from the App ID pop-up menu, and click Continue.

    If you used a team provisioning profile during development and the menu contains only the Xcode iOS Wildcard App ID, select it. If the menu contains another Xcode-managed explicit App ID (it begins with “Xcode” and contains your bundle ID), select that App ID. If you created your own App ID, select that one.

  5. Select the distribution certificate you want to use, and click Continue.

  6. Select the devices you want to use for testing, and click Continue.

  7. Enter a profile name, and click Generate.

    ../Art/6_ios_createdistributionprofile_3_2x.png

    Wait while Member Center generates the provisioning profile.

  8. Click Done.

In Xcode, refresh the provisioning profiles to download the ad hoc provisioning profile, as described in “Refreshing Provisioning Profiles in Xcode.” The ad hoc provisioning profile should now appear in the Provisioning Profiles table on the view details dialog in Accounts preferences.

Updating the Build String

If you update the build string, as described in “Setting the Version Number and Build String,” iTunes recognizes that the build string changed and properly syncs the new iOS App Store Package to test devices.

Archiving and Validating Your App

You create an archive of your app regardless of the type of distribution method you select. Xcode archives allow you to build your app and store it, along with critical debugging information, in a bundle that’s managed by Xcode. For example, if you distribute a build of your app to testers, archiving the debugging information makes it easier to interpret crash reports that they send you later. After your app is released, you use the same debugging information to decipher crash reports that you download from iTunes Connect.

With archives, you can distribute the same build to the store that you distribute for testing. It’s important to test the exact build that’s a candidate for release. Differences between Xcode build settings can cause bugs that don’t appear during testing, because the binary being tested was built differently. By having Xcode make an archive of your app, you can be sure you’re testing the exact same build of your app that you submit to the store.

Follow these steps to archive and validate your app:

  1. Review the Archive scheme settings.

  2. Create an archive of your app.

  3. Validate the archive.

Reviewing the Archive Scheme Settings

The first time you make an archive, review the Archive scheme settings to ensure that you don’t archive a debug version of your app.

bullet
To review the Archive scheme
  1. In the Xcode project editor, choose Product > Scheme > Edit Scheme to open the scheme editor.

  2. Click Archive in the column on the left.

    ../Art/6_archiveschemesettings_2x.png
  3. Choose Release from the Build Configuration pop-up menu, and click OK.

Creating an Archive

Next, create an archive of your app. Xcode stores this archive in the Archives organizer.

bullet
To create an archive
  1. In the Xcode project editor, choose iOS Device or your device name from the Scheme toolbar menu.

    You can’t create an archive of a simulator build. If an iOS device is connected to your Mac, the device name appears in the Scheme toolbar menu. When you disconnect the iOS device, the menu item changes to iOS Device.

  2. Choose Product > Archive.

    The Archives organizer appears and displays the new archive.

Xcode runs validation tests on the archive and may display a validate warning in the project editor. For example, if you don’t set required app icons, as described in “Setting Individual App Icon Files,” an Info.plist warning message appears. If this happens, fix the issue and create the archive again.

Running iTunes Connect Validation Tests

Optionally, upload your archive to iTunes Connect to run validation tests.

Since iTunes Connect will validate your archive when you submit it to the store, it’s best to validate your beta version now to discover issues early. The app record in iTunes Connect must be in the “Waiting for Upload” or later state for you to validate your app, as described in “Changing the Status to Enable Uploading.” If you’re not ready to create your app record, you can skip this validation step.

bullet
To validate an archive
  1. In the Archives organizer, select the archive.

  2. Click the Validate button.

    ../Art/6_ios_archive_organizer_2x.png
  3. Enter your iTunes Connect credentials, and click Next.

    If a dialog appears stating that no application record can be found, create an app record in iTunes Connect before continuing.

  4. Select the ad hoc provisioning profile you created in a previous step, and click Validate.

    iTunes Connect runs validation tests.

  5. Review validation issues found, if any, and click Finish.

Fix any validation issues, create a new archive, and validate it again. Validation errors don’t prevent you from distributing your app for beta testing.

Troubleshooting

If Xcode doesn’t find signing identities, a dialog stating “No identities are available for signing” appears. Verify that you have a distribution certificate and an ad hoc provisioning profile before continuing.

If your ad hoc provisioning profile doesn’t appear in the Provisioning Profile pop-up menu when you create the iOS App Store Package, refresh the profiles in Xcode, as described in “Refreshing Provisioning Profiles in Xcode.”

Creating an iOS App Store Package

Because testers don’t have Xcode to run your app, you want to create an iOS App Store Package that they use to install your app on their device. You use Xcode to create an archive and generate an iOS App Store Package (a file with a .ipa filename extension) from the archive.

bullet
To create an iOS App Store Package for testing on devices
  1. In the Archives organizer, select the archive.

    ../Art/6_ios_archive_organizer_2x.png
  2. Optionally, click the Validate button.

    If you’re close to submitting your app to the store, validate it now and fix any problems before distributing it for final testing. For a complete list of issues you should fix before distributing your app, read App Store Review Guidelines for iOS Apps.

  3. Click the Distribute button.

  4. Select “Save for Enterprise or Ad Hoc Deployment,” and click Next.

    ../Art/6_ios_createappstorepackage_1_2x.png
  5. Choose your ad hoc provisioning profile from the Provisioning Profile pop-up menu, and click Export.

    ../Art/6_ios_createappstorepackage_2_2x.png
  6. Enter a filename and location for the iOS App Store Package file, and click Save.

    The file has an .ipa extension.

Troubleshooting

If Xcode doesn’t find signing identities, a dialog stating “No identities are available for signing” appears. Verify that you have a distribution certificate and an ad hoc provisioning profile before continuing.

If your ad hoc provisioning profile doesn’t appear in the Provisioning Profile pop-up menu when you create the iOS App Store Package, refresh the profiles in Xcode, as described in “Refreshing Provisioning Profiles in Xcode.”

Installing Your App on Test Devices

Before you distribute your app to testers, follow the steps that testers use to install and run the app on their devices. Use iTunes to install the app on a nondevelopment device. iOS extracts the embedded ad hoc provisioning profile in your app and installs it on the device for you. Then test your app on the device.

Follow these steps to install the app on a testing device.

bullet
To install the app on a device
  1. Connect the testing device to a Mac running iTunes.

    If possible, don’t use a Mac that you use for development.

  2. Double-click the iOS App Store Package file you created earlier (the file with the .ipa extension).

  3. In iTunes, click the device button in the upper-right corner of the window.

  4. Select the Apps button.

    The app appears in the iTunes app list.

    ../Art/5_ios_installapp_1_2x.png
  5. Under Apps, choose “Sort by Name” or “Sort by Kind” from the pop-up menu.

    An Install or Remove button appears adjacent to the app.

  6. If an Install button appears, click it.

    The button text changes to Will Install.

  7. Click the Apply button or the Sync button in the lower-left corner to sync the device.

    The app is uploaded to the device so that the user can start testing.

Finally, send the iOS App Store Package file to testers along with the app installation instructions and the crash report instructions, as described in “Soliciting Crash Reports from Testers.”

Soliciting Crash Reports from Testers

When an app crashes on a device, iOS creates a record of that event. The next time the tester connects the iOS device to iTunes, iTunes downloads those records (known as crash logs) to the tester’s Mac. Testers should send these crash logs to you along with any bug reports. Later, after your app is released, you can also retrieve crash reports of your live app from iTunes Connect.

Tell testers how to retrieve crash reports from their devices and send them to you.

bullet
To send crash reports from a Mac
  1. Connect the testing device to a Mac running iTunes.

    iTunes downloads the crash reports to your Mac.

  2. In the Finder, choose Go > Go to Folder.

  3. Enter ~/Library/Logs/CrashReporter/MobileDevice.

  4. Open the folder identified by your device’s name.

  5. Select the crash logs named after the app you’re testing.

  6. Choose Finder > Services > Mail > Send File.

  7. In the New Message window, enter the developer’s address in the To field and appropriate text in the Subject field.

  8. Choose Message > Send.

  9. To avoid sending duplicate reports later, delete the crash reports you sent.

bullet
To send crash reports from Windows
  1. Enter the crash log directory for your operating system in the Windows search field, replacing <user_name> with your Windows user name.

    • For crash log storage on Windows, type:

      C:\Users\<user_name>\AppData\Roaming\Apple computer\Logs\CrashReporter/MobileDevice

    • For crash log storage on Windows XP, type:

      C:\Documents and Settings\<user_name>\Application Data\Apple computer\Logs\CrashReporter

  2. Open the folder named after your device’s name and send the crash logs for the app you’re testing in an email message using the subject-text format <app_name> crash logs from <your_name> (for example, MyTestApp crash logs from Anna Haro) to the app’s developer.

To learn how to interpret the reports when you receive them from testers, read “Analyzing Crash Reports.”

Distributing Your App Using the Xcode Service

After you perform the steps in this chapter to create an ad hoc provisioning profile and distribute your app for beta testing, you can optionally setup an Xcode service to distribute your app. The Xcode service, available in OS X Server, automates the integration process of building, analyzing, testing, and archiving your app. The Xcode service also hosts a website that facilitates the distribution of product builds and archives to testers and other team members. See Xcode Continuous Integration Guide for more information about using the Xcode service to distribute versions of your app to testers.

Copying App Sandbox Data

To further diagnose an issue with your app, you can copy app data from an app sandbox on an iOS device to your file system. You can also copy app data from your file system to an app sandbox on an iOS device.

Ad Hoc Provisioning Profiles in Depth

You signed the iOS App Store Package containing your app using the distribution certificate specified in the ad hoc provisioning profile. The ad hoc provisioning profile was included in the app bundle when you built and archived the app. Then you installed the app on a test device. The app successfully launches if the app’s bundle ID matches the App ID, the signature matches the distribution certificate, and the device is in the device list of the ad hoc provisioning profile.

../Art/AdHocProfileLaunch_2x.png

Recap

In this chapter, you learned how to distribute your iOS app for beta testing on designated test devices. You also received instructions to send to testers to install the beta version of your app and send crash reports to you.