Become a Simulator expert

Hello and welcome to WWDC.

Hello everyone. Thank you for watching our presentation on becoming a Simulator expert. I'm Ryan Bowring an engineer on the Devices in Simulators team here at Apple. Today we are going to talk about how the Simulator can help you turbo charge the developer experience and create great apps. So let's start with a brief overview before we dive in. Just what is Simulator and what can it do. Well Simulator lets you test your apps for iOS, iPadOS, TVOS, and watchOS on your Mac. No physical device required. You can prototype your apps faster and test them on multiple kinds of simulated devices. And it exposes lots of device functionality.

Now Simulator is built right into Xcode and it starts automatically when you run or test your app without a physical device. So many of you have probably seen Simulator before and you can run the Simulator as a standalone tool and make use of its powerful features to help you out on your developer journey.

Here we have two simulators booted: an iPhone 11 and an 11-inch iPad pro both running iOS 14. in the title you can see both the device name and the version of the OS that is running and the toolbar buttons let you access some of the most widely used features of our platforms. Let's start with screenshots. In Xcode 12 we've significantly improved the experience of taking screenshots in Simulator. Click on the screenshot button in the toolbar and an image of the device will fly out to the side here. By control- clicking on the screenshot thumbnail you can save it to one of several different locations, copy it to the clipboard, or open it in various different apps.

You can drag it onto other apps to insert it. And by clicking on the screenshot button repeatedly you can take multiple screenshots one after another.

It's a great way to take screenshots for your apps' Appstore page. If you don't interact with the screenshot then it will be saved to the desktop by default.

You can also drag files onto the Simulator to cause certain events to happen, such as adding an image to the photo library, triggering a push notification, or adding a certificate. For instance we can drag an APNS file onto the Simulator window here to cause a push notification to appear. We'll talk more about support for push notifications in a few minutes. Finally in Xcode 12 it's now possible to bring Simulator into full screen mode, both on its own, and tiled with another app. For example, you can have the Simulator and Xcode side-by-side in full screen mode. Let's take a moment to talk about how Simulator can allow you to test your app support for a certain platform features. iPadOS supports mouse pointer interactions, meaning that if a user connects a mouse or a trackpad to their iPad, they can get full pointer and gesture support. Watch how we can use the Simulator to test the support with a click of a pointer button in the toolbar, you could enter into pointer capture mode on an iPad Simulator running a supported version of iPadOS. Notice that once pointer capture mode is enabled your macs mouse or trackpad now controls the pointer and the iPad Simulator, so all mouse and trackpad gestures are processed by iPadOS. It behaves just like as if it were connected to a physical iPad. For example you can open up the Maps app. We can do a pinch gesture to zoom in and zoom out.

All keyboard shortcuts are also captured, so if we want to go back to the home screen we can press command-H. To switch apps, we can press command tab or do a three finger swipe up on the trackpad. And, of course we can do a two fingered drag to scroll. To stop capturing and return control to the host Mac, simply press escape. It's also possible to capture just the keyboard but not the pointer. This makes all keyboard inputs stay with the simulator, but leaves your mouse free to use with a Mac. Also in Xcode 12, Simulator now keeps track of which simulators are currently capturing. When a simulator window loses focus such as if another window opens up then capturing is automatically stopped. But when you switch back to the Simulator capturing starts again. This is really handy when you are debugging in X code. Here I've set a breakpoint in our demo app source code. It will trigger when we load one of our drinks from the menu. When the breakpoint triggers your mouse pointer will automatically be released, allowing you to go into X code and inspect things. See? Our pointer is now free. Once you switch back to the Simulator, capturing starts right back up again, right where you left off. And of course you can use escape to stop capturing.

Now you may be wondering if you can change the shortcut to start capturing. For instance maybe you want to make the escape key do something in your app.

Well the answer is yes. It is customizable. Simulator has a preferences window which allows you to configure a number of important settings. At the top here, we can see the option for the shortcut to stop capturing. There are three supported options, escape which is the default, pressing both the left and the right command keys, or pressing control option. The preferences window also allows you to customize the lifetime of booted simulators. By default, when the simulator is launched, it boots the most recently use simulator. And the running simulator get shut down when you close their window or quit the Simulator application. Personally, I end up opening and closing the app frequently, so I've configured my preferences to leave simulators running after I quit the app. When I want to shut down a simulator, I simply close its window. To get this result I simply set the preference items accordingly.

Next we have the visual indicator options. Simulator can draw helpful widgets for you to show the position of the simulated fingers for drag and pinch gestures.

You can also choose whether or not you want device mask to appear in screenshots. If this is enabled then on certain devices such as the iPhone 11, the area with the sensor housing will be rendered as transparent. In addition to preferences which apply to all simulators, the menu bar contains a variety of options for each Simulator. The features menu contains a number of useful options to toggle features supported by our platforms. One such option is toggle appearance. This allows you to toggle between light mode and dark mode in iOS. I personally prefer dark mode. So let's set our simulator to use it by clicking on the menu item. You can also use the keyboard shortcut of course. Under the file menu you can see the option to start simulators.

To keep the clutter down in xCode's run destination menu, Simulator only creates a few devices by default, but you can create more. One feature we are introducing in Xcode twelve is the ability to create a new simulator right from within the app so you won't need to go into Xcode as much anymore.

You can bring up the new Simulator window by clicking on the file menu, and then on new simulator ... here. And as you can, see we can enter a name for the new simulator as well as choose the device type and the runtime. Let's create one now. I'll just give it a name and click on Create. And now you can see it listed under our list of simulators, right in the menu. Once the simulator is running you may want to adjust the window size. You can click and drag to resize the corners of the simulator. The window menu also contains several window scaling options that can adjust the window size. Let's go through these options in more detail. First the option for physical size. This makes the simulator window on screen exactly the same size as a physical device would be. This is done by taking the pixels per inch of your Mac's display. The pixels per inch of the device you are simulating and then calculating how big to make the window so that the content matches the size. This can be used to see exactly how your app will appear across multiple simulated devices with different screen sizes and pixel densities. Next I want to talk about point accurate. Point accurate mode sizes the window so that content appears the same size on devices with different scale factors. In other words it makes all points on all simulators the same on your screen regardless of the pixel density of the simulated device. So a device with a 3x display will appear the same size as a 2x display. This will result in the simulator screen content being downscaled if your Macs display has a lower pixel density than that of the simulated device. For example, in point accurate mode a simulated device with a 3x display will be downscaled if viewed on a Mac with a 2x or a 1x display, and a simulated device with a 2x display will be downscaled if viewed on a Mac with a 1x display.

If the simulated devices display and your Macs display have the same pixel density then no scaling needs to occur. It's the same as pixel accurate in this case. Finally let's talk about the pixel accurate display mode. In this mode, each pixel on the simulated device is mapped directly to one pixel on your Mac's display. This will cause simulators to appear larger on screen. If the Mac's display has a lower pixel density than the simulated device.

As an example if we have a simulated device with a 2x screen running on a Mac with a 1x screen then the simulator window will be double the size of the point accurate mode. Now I want to talk about ways you can leverage the command line to access all of the power Simulator offers. You can control Simulator from the command line using the simctl tool. Its name is a portmanteau of the words simulator and control. It is an amazing tool to help boost your productivity. First we're going to talk about privacy. At Apple we take our users privacy seriously. For so many people our devices are often an extension of our own personal lives. They contain so much personal data about ourselves and those we care about. That's why our platforms don't allow apps to access your user's personal data without their permission. Testing your apps handling of the user's privacy choices is important and simctl can help you. An important part of the privacy features on our platforms are various access prompts that appear when an app wants to access a protected resource. You must test your app to see how it works when permissions are both granted and withheld. With simctl you can grant and revoke your apps access to these protected resources from the command line. simctl supports many of the most important services. These include access to the calendar, your user's contacts, their location, and their photos. For more information and to see the full list of supported services, don't hesitate to visit the simctl help pages. To access it, just run simctl without any arguments.

To grant your app permission, you simply specify the action - here, grant, to grant permissions - followed by the service name and your app's bundle ID.

Here I am using the booted alias to refer to the currently booted simulator. If you have multiple booted simulators, you'll want to specify which simulator by its name or UDID. Of course it's equally possible to revoke an app's access to a resource. You just specify the revoke action. Instead of listing a specific service you can instead specify all as the service name. This applies a specified action to all services. Here we are taking away all of an app's permissions. And finally, you can reset all permissions to restore them to the defaults. With the reset command, it can apply either to a specific app or system-wide. Here we will reset all permissions to all apps on the system. Next let's discuss push notifications. Push notifications are a big part of the experience for users on Apple platforms. It lets you deliver messages to your users periodically to notify them about things they've asked to be notified about. Here you can see an example of a very very basic push notification containing a title and a body. When the push notification is displayed the body text appears below the title text in a different font. One special additional key is this Simulator target bundle key. This key specifies the bundle ID of the app that should receive the notification.

Here the notification is associated with our sample app. Let's dispatch the notification to our sample app. Now the push notification command takes at most two parameters. The app bundle ID and the payload file which I just showed you. If you drag and drop the file onto the device window then the bundle ID must be specified in the JSON file. With simctl, the bundle ID can be specified as a parameter or in the JSON file but it's still useful to include so that you can omit the bundle ID from your commands. Just note that whatever you specify on the command line will override anything from the file. So let's run this command and see what happens. Like typical UNIX commands, simctl prints no output if it succeeds. You'll see the contents of the push notification on the screen.

See? It works, just like that. Next up let's talk about video recording. Sometimes you really need a video to capture the essence of your app. Whether it's to market your app on its app store page, to send some work-in-progress UI to a colleague or to give a demo on your Web site. Lots of developers use screen recording software such as quicktime to do this. But you don't need to use any special software to record the simulator. Fortunately with simctl, you can capture a video of your app's functionality right from the command line and simulator.app does not even need to be running. Here's a sample simctl Command to record a video. It captures a video of the screen from the running device and saves it to the output file named video.mp4. Note that this file shouldn't already exist. If it already exists then the recording will fail unless you use the --force command line flag to indicate that it is safe to write over the existing file.

The video recording capabilities of simctl supports several different options to control the behavior. Here we are using the default options.

These options give a video file the HEVC codec with a black device mask. If the device mask is set to be black, then the sensor bar and curves of certain devices such as the iPhone 11 will be rendered black in the video, making the items behind them invisible. It's important to note that the video recording continues until you terminate it by pressing control-c in the terminal window. One nice thing about video recording with simctl is that it uses your GPU to accelerate the video encoding. So, it won't use too many system resources. Now let's record another video, but this time specifying some options. We're going to change the video codec and the device mask. This time we will ignore any device mask. This means the entire rectangular frame buffer will be recorded. We will also use the H.264 codec instead of the HEVC codec. Lastly certain devices running iOS support external displays. You can choose to capture the video footage from either the devices internal display or an external display. Here's how to capture a video from an external display. On iOS one app that supports sending output to an external display is the photos app. Right now we are flicking through the photos app looking at our photos. If we want to capture what is displayed on the external display we tell simctl to capture the output of the external display with a --display flag. And here is the video that we captured. You'll notice that it contains the output of the external display and not the simulated device's screen.

Now onto the status bar. As you know iOS and iPadOS devices have a status bar showing the time date cellular and Wi-Fi signal and other important information.

You can use simctl to customize this information so that it can display whatever information you want. For example, this can be used for screenshots for your app's Appstore page. Here's our iPhone. As it stands now. Currently the phone is showing that the current time is 9:41 a.m. with full cellular and Wi-Fi strength. The battery is also 100 percent charged. Suppose we want to customize the status bar up top so we can see what it would look like under different conditions.

The simctl status bar commands can override the status bar to display whatever information you like.

Let's say we want a screenshot of our app when the user's out and about.

Some time has passed so it is now 12:01. Perhaps the user has a low signal with one cellular bar signal, 3G data network, and no Wi-Fi. Let's see the result. After running the command you can see that our status bar has changed to match the settings we have specified. Of course at a certain point you may want to clear the overrides you have set. You can do that with a clear command. Let's run that now. And as you can see, the status bar has been returned to its default settings. Now there are actually more options that you can override in the status bar than I've shown here. To see all the available options run the status bar commands without any arguments.

Finally let's discuss the key chain management. simctl allows you to add CA certificates to the devices trusted root store. This is really useful when you are developing apps with network capabilities. Here's how it works. I have a certificate which I want to add to the trusted root store in the simulator. I can do this with the simctl key chain commands.

When you call this command, you need to give it the path to a valid CA certificate.

In addition to using simctl, you can also add the certificate by dragging it onto the Simulator window. But here we're going to stick with the simctl command. And after running this command your certificate will be added into the Simulator's trust store. But you still have to trust it manually. To do so, go into the settings app, then tap on general -> about -> certificate trust settings, and then tap on the switch to activate it. Be sure to acknowledge the warning message. And at this point this CA certificate is now trusted. So now any TLS certificates the CA issues will work without a problem.

You can do other tasks with the key chain command as well such as clearing any save passwords. For more information please check the help pages. And that's the end of our presentation on becoming a simulator expert. We hope the techniques we've talked about today will help you turbo charge your developer experience so that you can create better apps and delight your users. Thank you for watching and we hope that you enjoy the rest of the content for WWDC 2020.