Bring your high-end game to iPhone 15 Pro

Hello and welcome, my name is Jason Jane, and I am an engineering manager here at Apple. Today, I'm very excited to talk to you about bringing your most demanding games to iPhone.

Earlier this summer, we shared three great talks about the new tools and best practices for quickly bringing the most demanding PC and console games to the unified and scalable platform of Apple silicon Macs.

We shared that you have everything you need to deliver an amazing, uncompromising gaming experience with Mac’s advanced graphics, fantastic performance, immersive audio, stunning displays, and feature-rich software.

Now, I'm excited to share that the latest iPhone 15 Pro and iPhone 15 Pro Max models join Apple silicon Macs to build on this unified platform for next-level gaming.

These new iPhones play every iOS game and run every iOS app.

And now, iPhones can also play the highest end games without compromise. It has the power of a game console, but now in your pocket. iPhone 15 Pro and iPhone 15 Pro Max include new features, like faster CPU and GPU performance, hardware support for PC and console textures, and additional system resources, which make targeting these iPhones as easy as targeting a Mac.

For the rest of this talk, I'm going to cover the features and tools we've added to help bring your game to iPhone.

First, I'll describe the tools and strategies to create your game's porting plan. Then I'll describe how to specifically target the iPhone 15 Pro and iPhone 15 Pro Max.

I'll then tell you how to optimize for long play sessions, how to provide the best experience for games designed around controllers, and finally, how to handle large, high-quality assets in your game.

To start, let's think about creating a game plan. What is the porting process like, and how can you take incremental steps and create quick iteration techniques for yourself and your team when bringing your game to iPhone? You might not be surprised to hear that Kojima Productions started their porting journey to iPhone for the breakthrough game Death Stranding by first bringing it to Apple silicon Macs.

By starting with a port to the Mac, they were targeting the same software APIs, unified memory architecture, advanced hardware and GPU capabilities shared between Apple silicon Macs and the iPhone.

During the initial stages of work, they could rapidly iterate on Macs using Xcode and instruments.

They had the additional memory and resources of powerful desktop Macs. The extra storage and memory for debugging and test scaffolding is super useful during development.

And they could more easily stand up their work and test with command line automation and with keyboard, mouse, and controller input right at their desks.

Once running on the Mac, the unified architecture of Apple silicon and the resources and capabilities of iPhone 15 Pro and iPhone 15 Pro Max meant that bringing their Mac port over to iPhone was quick, taking just a few days to see the running game. There's simply no other platform with the same consistency and ability to scale in this way.

Scaling naturally from portable iPhones and iPads to the Mac Mini and iMac, and onto the most powerful desktop Mac Studio and Mac Pro, this is the power of Apple silicon.

To kickstart your port to Mac on your way to iPhone, we also recommend using the Game Porting Toolkit. This toolkit gives you early insight into how the graphics and shaders of your PC game will perform on Apple silicon. You simply run your unmodified Windows PC game right on a Mac to try it out.

So before you recompile your source code, convert your custom shaders from HLSL, reimplement your graphic subsystem, and convert your use of audio, input, display and HDR rendering, instead of doing that work before you see your first scene running on a Mac. And before you start debugging and optimizing for iPhone, we recommend that you run and analyze your unmodified windows binary on Mac to help you profile and understand your game's behavior on Apple silicon and to help create your game plan.

There's lots more detail about starting your porting journey in part one of our "Bring your game to Mac" series of talks.

If your game needs next level features, performance and capabilities of iPhone 15 Pro, then you can directly target those devices as your minimum specification.

We've defined an iPhone Performance Gaming Tier in iOS 17 and in the app store. This is a flag you add to your UI required device capability section of your app's info plist file.

By adding this flag, you are specifying your min spec. Your app will only be available for purchase, installation, and review by customers with devices at least this capable.

To establish this min spec, use Xcode to add an entry to the required device capability section of your info plist.

To set this min spec feature, you'll also target iOS 17 as your minimum deployment target.

Once you do this, the app still will mark your game as available for purchase and install only for customers with supported devices. Here's what a customer with iPhone 13 Pro might see.

If your game can scale its quality and performance dynamically across all iPhones on iOS 17, then you should also consider how you might enhance it for iPhone 15 Pro.

Scaling is not unusual for PC and console games, and it's a great way to target the largest addressable market.

To optionally take advantage of hardware-accelerated mesh shading or ray tracing, just query for the Apple 9 GPU family to ensure that you're on a device with an Apple A17 Pro GPU.

Now let's talk about how you can optimize for high quality consistent performance in long-running game sessions on iPhone.

I'll cover visualizing performance, a new performance profile, techniques for optimizing shaders, and how to use MetalFX on iPhone.

Visualizing your performance in real time is easy with a metal performance HUD.

Enable the metal performance HUD during day-to-day testing to quickly spot missed frames and memory leaks. The HUD shows you the GPU and resolution being used, the instantaneous frame rate, the presentation interval, and the approximation of the amount of GPU time being used. It highlights high and low values for these statistics, shows memory usage, and provides a graph of the last few seconds of data. To learn more, check out "Discover Metal Performance HUD." We're introducing a new system performance profile called Sustained Execution Mode with iPhone 15 Pro and iPhone 15 Pro Max. The default system performance profile aims to provide the best performance in three time domains, the highest possible performance for short burst workloads, followed by a period for providing consistent performance for the majority of customer sessions, and finally, the steady state, which honors device constraints for longer workloads. This leaves you needing to decide how to tune your game. Do you tune for initial performance and have your game performance change over time, or do you spend time during development allowing performance to settle and then retune accordingly? To alleviate this effort and provide a consistent experience, we're introducing Sustained Execution Mode for iPhone 15 Pro and iPhone 15 Pro Max.

By opting into this new profile, you can experience performance that is consistent with the steady state right from app launch. This allows you to quickly identify a level of quality and performance appropriate for the target device, and you can be confident your players will experience the exact same consistent quality.

To enable sustained execution mode while your game is running, add the entitlement to your application. Simply go into Xcode and in the sign in and capabilities tool, add the Sustained Execution Capability. Remember, you'll also need to provision this entitlement in the provisioning profile for your application. You do this in the profile section of your Apple developer account.

Metal supports shader pre-compilation, which is another important performance technique. On mobile devices, you should generally avoid on-device compilation of shaders from source as it isn't power efficient. On-device compilation at runtime can also cause unexpected hitches. When you are porting a PC or console game, pre-compiling is even more important as you likely have a lot more shaders and need them ready as quickly as possible.

Always use ahead of time compilation by the Metal shader converter and the Metal compiler to produce a metal lib at app build time and package it in your bundle.

In some specific cases, you can also take the metal lib all the way to finalized GPU binary by using the metal-tt to cogen for Metal GPU family 9.

For more details, please check out, "Bring your game to Mac, Part 2: Compile your shaders," as the same advice for porting your game to the Mac equally applies to iPhone.

Finally, MetalFX is another fantastic performance technique new this year to iOS 17. MetalFX was introduced last year for the Mac. It helps improve game performance by generate high quality, high resolution images for many fewer pixels, saving power and time.

MetalFX offers two upscaling effects, spatial, which offers greater performance benefit and temporal for the best quality. This year, we are bringing the power of MetalFX to iOS. Spatial upscaling is available on all iPhones supported in iOS 17, and temporal upscaling is available on devices with A14 and newer.

Rendering to a lower resolution is one method of improving the performance of your game. By reducing the GPU work, you can save power and achieve a higher frame rate. Every iPhone has a high quality, built-in display scaler that automatically scales the contents of a CAMetalLayer to the native display resolution. You're probably already using it like this.

With MetalFX, your target resolution can remain the same, so you continue to take advantage of the display scaler, but your game now renders to a lower resolution.

MetalFX can help demanding games achieve stable frame rates on both ProMotion and non-ProMotion displays. Depending on your game workload, you'll want to experiment combining different MetalFX and display upskilling ratios to achieve the best balance for performance and quality. For more details on best practices for using MetalFX, please refer to past WWDC sessions, "Boost performance with MetalFX Upscaling," and "Bring your game to Mac, Part 3: Render with Metal." Interacting with your game on iPhone has never been easier, thanks to great controller support that's now even better. Many PC and console games are designed around using game controllers for input. The great news is that doesn't change when you bring your game to iPhone. iOS supports connecting to all the most popular game controllers, including a new generation of form fitting controllers that fit and work perfectly with every size of iPhone 15 using low latency USBC. This PlayStation theme controller from Backbone is one of my favorites.

If your game is best experienced with a controller, let your players know by adding the game controller's capability, using the sign in and capability section in Xcode.

And then add the GCRequiresControllerUserInteraction dictionary to the info plist array, and include an iOS key with a Boolean value of YES.

With this configuration set, players will see a controller recommended badge in the app store, which leads to more details about using and comparing game controllers and where they can find and purchase compatible form-fitting and standalone controllers.

But the iPhone also revolutionized mobile gaming with low latency, high accuracy multi-touch.

It's important and even part of the app store review process that your game supports touch interaction too.

This year, we've made it even easier to quickly add even more customized touch controls to your app. The GC Virtual Controller API now supports drawing your own onscreen controls right where you want them. Your touch handling is converted into game controller input events, which lets you customize the look and feel of your game UI while still leaving intact the majority of your game controller input logic.

Lastly, iPhone 15 Pro and iPhone 15 Pro Max now seamlessly support your highest quality PC and console assets. The original base assets for your games can be very large, high resolution texture data, high polygon count geometry, and uncompressed audio and voice data.

It's the job of your asset pipeline to convert and package these files to fit into available PC and console system resources and to efficiently interoperate with the target hardware, taking advantage of accelerated features like texture decompression, geometry instancing and audio mixing, for example.

Before iPhone 15 Pro and iPhone 15 Pro Max, you may have worried about the effort of targeting mobile devices. You may have even created an entirely new, generic mobile asset pipeline to generate highly crunched assets to fit into limited resource space or to handle mobile-only texture formats.

But with iPhone 15 Pro and iPhone 15 Pro Max, you can simply use your existing PC and console asset pipeline.

Hardware accelerated BC and texture support means there's no longer a need to convert your existing large asset packages, although you can see even better compression and higher quality with ASDC compression. And more dedicated memory means seamlessly loading, storing, and moving large, high-quality assets that will really shine on the Retina ProMotion displays, and you'll save time with a simplified porting effort.

Query for BCn availability using the metal device inspection function, supportsBCTextureCompression.

If you also support other iPhone models running iOS 17, using assets compress with ASTC compression is a great choice for even better quality and higher compression ratios.

To maximize available memory, be sure to enable the existing increased memory limit entitlement in Xcode in your Apple developer profile. With more resources, you can render with more detail, use higher resolution depth and shadow buffers, and implement more complex, resource-intense rendering algorithms.

If you're scaling across devices, you can also use os_proc_available_memory to dynamically determine which resources are available. To bring large assets to iPhone, your app will need to dynamically download them. You have three ways to do this, On Demand Resources, Background Assets, or your own custom downloading implementation.

When downloading, be sure to store them into allocation where they aren't impacting customer backups and where the system can purge them if needed. As your assets arrive, move them into the caches directory under your data container's library directory. This directory is not backed up with important per application user data. The system also guarantees it will not purge this cache needlessly. Learn more about the latest downloading technologies in the most recent background assets talk from WWDC.

So that wraps up our best practices. We hope you bring your high-end game to iPhone 15 Pro and iPhone 15 Pro Max without compromises. The Apple A17 PRO GPU represents a huge leap in power and graphical features, which makes bringing your game to iPhone quick and easy. Together these iPhone 15 pro models join the family of Apple silicon Macs to build a unified target platform for your next-level gaming projects. I'm excited to see and play the games you bring to iPhone.