CarPlay is a smarter, safer way to use your iPhone in the car. Learn how CarPlay works and how to design your car's infotainment system to work seamlessly with iPhone. Understand best practices for a great user experience that integrates CarPlay with the car's native system.
[ Music ]
Hello, and welcome to our first session
on developing CarPlay Systems.
My name is Emily, and I'm responsible
for Car Experience engineering at Apple.
CarPlay was designed from the ground up as a smarter,
safer way to use your iPhone in the car.
CarPlay takes the things you want to do
with your iPhone while driving, and puts them right
on your car's built-in display.
CarPlay brings the familiar elements
and interaction patterns of iPhone into your car.
Users already know how to do things in CarPlay
because they know how to use their iPhone.
CarPlay is about more than just showing iOS
on the car's display.
CarPlay is thoughtfully integrated
with your car's built-in system
to deliver the best possible user experience.
CarPlay allows you to design for the future.
With CarPlay, you can build a car today,
and when Apple introduces new versions of iOS,
users will immediately benefit from new features.
Your car will remain fresh for years into the future.
Today we'll give you an overview
of how the CarPlay feature works,
and discuss the most important requirements
for you to consider.
We'll explain the user experience that we want
to enable with CarPlay and the design guidelines
that we've provided.
At the end, we'll talk about CarPlay supported apps.
IPhone user interface is designed to work perfectly
when held in the hand, when you can touch the phone.
CarPlay translates that user interface to the car,
making it appropriate in that setting, and safer to use.
It works with what you already have in your vehicle.
It's comfortable to use in that environment.
Now let's talk about how CarPlay works in detail.
Let's start with the native user interface of the vehicle.
When an iPhone is plugged in, the CarPlay session begins.
When a CarPlay session is brought up,
information is sent both directions via
Video can be sent from the iPhone
to the car as well as audio.
Audio can also be sent from the car's microphone to the iPhone.
Once a CarPlay session is established,
different combinations of these data streams are used.
When a CarPlay Home screen is first shown,
and no audio is playing, the communication protocols are used
for setup and location information.
An H264 video stream showing the Home screen is sent
from the iPhone to the car.
If we then play audio in CarPlay,
the communication protocols will additionally be used
to send information about the playing track.
The video stream here is encoding the now playing screen,
and audio for the song is sent to the car.
If the CarPlay UI is exited, and the car's screen is shown,
the CarPlay session remains active.
The communication protocols will continue to be used
for now playing and location information.
The video stream is no longer needed,
but CarPlay audio still plays,
even though CarPlay video is not shown.
The screen and audio are controlled independently.
Video from the native system can be shown while CarPlay audio
If the CarPlay interface were shown,
audio from the native system, like FM radio, could play.
Let's see what happens if Siri is then launched using the
push-to-talk steering wheel button.
The communication protocols will be used to tell the head unit
that iOS voice recognition is underway.
The CarPlay UI is sent to the screen,
and the Siri audio chime plays on the vehicle's speakers.
In this case, audio is also sent
from the car's microphones to the iPhone.
These examples gave you a high-level overview
of how CarPlay works.
Now let's look in more detail
at what's happening in the head unit.
For wired CarPlay, data goes over USB.
All data is wrapped in IP.
Except for the iAP2 communication protocol.
Information from vehicle sensors is sent over iAP2 as is audio,
telephony, and turn-by-turn metadata.
The communication plug-in is source code provided by Apple,
and that receives the incoming video and audio streams as well
as an additional communication stream.
The audio and video is sent to the head unit's infrastructure,
connected to the speakers, microphone, and display.
Control events from the touchscreen, knob, buttons,
or touchpad are sent to the plug-in.
Now in the case of wireless CarPlay,
much of the system looks the same.
Both Wi-Fi and Bluetooth are needed.
For wireless, all data is wrapped in IP.
Connections to the vehicle's subsystems are the same.
Now that you have a good understanding
of how CarPlay works, we'll go
over the most important vehicle requirements.
We will talk through each of these in detail.
We want to make sure you understand the requirements
before you begin to design a system so that you can be sure
that it's capable of supporting CarPlay.
First, the vehicle needs a great display.
A high-quality display is critical
to the CarPlay experience.
Several standard display resolutions are supported,
and those resolutions are shown here.
Most are approximately 16 by 9.
Extra-wide screens are also supported.
Today, all supported resolutions are landscape orientation.
We have support for screens with varying pixel densities.
Asset sizes in CarPlay have been adjusted depending
on the resolution of the screen.
So that assets will appear the same size on low
and high resolution screens.
Without this, icons on the 720 screens would be much smaller.
If you do have a display with a slightly different resolution
than what was listed here, please contact Apple to see
if it can be supported.
We require that aftermarket screens be no smaller
than six inches
since aftermarket systems could be installed
into a wide variety of vehicles.
There is no minimum for systems built by automakers.
The automaker has the responsibility to make sure
that CarPlay is appropriate on the selected vehicle screen.
I should mention several additional screen requirements.
Twenty-four bit color depth is required,
and a 60 hertz refresh rate is strongly recommended
for a responsive user interface.
Make sure that your H264 decoder supports the right profile.
This is a hard requirement.
A high-quality display in the vehicle is critical
to the CarPlay experience.
Let's talk about the car's audio system and microphones.
CarPlay audio is separated
into two streams main audio and alternate audio.
Main audio is bi-directional, and it is used for music
and other media plus phone calls and Siri.
Alternate audio is used for notifications,
and it must always be mixed by the head unit with main audio.
If the head unit is playing audio, like FM radio,
the head unit must mix alternate audio
from iPhone with that audio.
LPCM is used for wired CarPlay.
Wireless CarPlay requires compressed audio.
AAC-LC is used for media, and you have a choice between OPUS
and AAC-ELD for other audio.
CarPlay supports the various user input devices
in your vehicles.
The CarPlay UI can be controlled by a touchscreen
on the head unit, by knobs and buttons, or by a touchpad.
The Siri button on the steering wheel is also a critical user
Many vehicles have touchscreens,
and let's talk about those first.
When the CarPlay UI is active, and the user touches the screen,
the head unit sends the iPhone the x,
y coordinates of that touch.
The head unit sends all touches to the iPhone
when CarPlay is displayed on the screen.
CarPlay is controlled with single touches only.
We support two types of touchscreens,
what we call high-fidelity and low-fidelity.
High-fidelity touch systems have low latency.
So swipe gestures that track the user's finger are allowed.
A high-performance system is required, however,
one that has less than 140 milliseconds touch latency.
This is how list scrolling using a swipe gesture
that tracks the touch input works
with a high-fidelity touch system.
Low-fidelity touchscreens may have higher latency
and are often resistive screens.
Taps control the UI, and a swipe will not track the
This is how list scrolling with single taps works
with a low-fidelity touch system.
Some vehicles use knob, button, and touchpad controls.
Sometimes these are the main user input devices,
and sometimes they are secondary to a touchscreen.
All combinations are supported by CarPlay.
Knobs are supported by the head unit, sending rotate,
select, and back events.
This is the minimum set of supported features
for a knob that's used as a primary interface.
Rotate, select, and back.
Some knobs also support a tilt or nudge feature.
This capability is supported by the CarPlay UI.
Vehicle touchpads report x, y coordinates as well
as select and a back button.
If a knob or touchpad controller is primary,
back and select are required.
Secondary touchpads can also support character recognition.
Buttons for various functions such as next, previous,
and telephony functions are optional.
The CarPlay UI will automatically change
to match the user inputs available in the vehicle.
All combinations are possible.
If the main controller interface is touchscreen,
then combinations with a knob, touchpad,
or knob plus touchpad are also possible.
A hard button for Siri is required.
Siri is really important to the CarPlay experience.
The user needs a quick way to always access Siri even
if the CarPlay UI isn't showing.
Usually this requirement is satisfied
with a push-to-talk button on the steering wheel.
For the Siri button, we require
that the head unit send iPhone all button up/down events
so that interaction during the Siri conversation is possible.
This wraps up our discussion on user inputs.
Sensor information is important to CarPlay.
CarPlay needs location information from the vehicle
so that the map's app works well.
Speed and GNSS information can be provided.
Every vehicle needs to know its speed.
Speed information is used by iPhone for dead reckoning
and is critical if the vehicle does not have a GPS
or GLONASS receiver.
GNSS information includes latitude and longitude
if the vehicle has that capability.
Satellite location information from the vehicle is used
in addition to the iPhone's sensors
to determine the location of the user.
This data from the vehicle may be better
since a larger antenna can be integrated into the car.
This is the recommended configuration
since it will result in the best maps user experience.
Sending satellite location information
to iPhone is required
if the vehicle has the technology on board.
GNSS information is also required
for any system supporting wireless CarPlay
since it's more likely that the phone will be in a pocket, bag,
or somewhere where the phone itself has poor reception.
What are the requirements
for connecting an iPhone to the head unit?
Apple CarPlay supports both wired and wireless connections.
We're excited about the addition of wireless CarPlay
since it offers a very convenient experience
Users can leave their phones in pockets and bags,
which is especially nice on a short drive.
Let's first talk about wired connections.
USB is used here, and all audio, video,
and control information is sent over USB.
Your head unit will need to support a USB role swap.
The head unit will switch from a USB host to a USB device.
High throughput is a requirement, and you'll want
to make sure your USB infrastructure can support the
required data rates.
If you have multiple USB ports in the vehicle,
and not all of them support CarPlay,
you'll need to properly label the ports.
Wireless CarPlay is supported in iOS 9.
Both Bluetooth and Wi-Fi are required.
Bluetooth is used for discovery and the initial connection.
Once the Wi-Fi credentials are sent to the iPhone
over Bluetooth, Wi-Fi is connected,
Bluetooth is disconnected,
and all subsequent CarPlay communication is over Wi-Fi.
During the CarPlay session, all audio,
including phone call audio and video, is transmitted over Wi-Fi
as well as the control protocol.
CarPlay requires a Wi-Fi certified AP,
and 5 gigahertz is recommended.
Note that high throughput,
25 megabits per second, is required.
The Bluetooth subsystem needs
to support the communication protocol.
Wireless CarPlay offers a convenient comfortable user
experience, and we're excited
to see your vehicles that support it.
Now let's hear from Donald about the CarPlay Design Guidelines.
Hi. My name is Donald, and I work on design at Apple.
Now that you've seen how CarPlay works,
let's look at some design guidelines
that will help you provide the best CarPlay experience
in your cars.
One of the most important things to keep in mind is
that CarPlay is really designed to use the entire display.
CarPlay is instantly familiar to iPhone users,
and they feel comfortable with it
because it's a consistent environment with its own icons,
visual language, and system of internal navigation.
Every interaction with CarPlay is easier and faster
when users can trust that everything
on the screen is part of CarPlay.
If a user has to discern which elements are part of CarPlay
and which elements are part of your car's user interface,
then every interaction becomes more challenging.
Also, your car's native user interface may include icons
that resemble or have similar functions
to icons within CarPlay.
And it can be confusing if these icons are
on the display at the same time.
Instead, use the entire display to show CarPlay.
Let's take a look at what should happen when CarPlay starts.
CarPlay always starts the moment iPhone is plugged into your car.
Users should never have to think about how to start it.
When the user connects their iPhone to your car
for the first time, the car's display should switch
to show CarPlay right away.
This provides feedback that CarPlay is started
and creates a strong association between the act of connecting
and the familiar appearance of CarPlay.
Don't show notifications
to announce state changes related to CarPlay.
The needed appearance of CarPlay
in the display is the most effective visual feedback.
Now let's consider what should happen
after the initial CarPlay connection.
When users connect their iPhone a second time, it should feel
like a continuation of their previous session.
So if CarPlay was showing on the display
when the user left the car, CarPlay should be shown again
when iPhone is connected.
And if CarPlay media was being played
when the user left the car, CarPlay media should resume
when iPhone is reconnected as long
as the user did not select a different audio source
between their CarPlay connections.
In other words, we want
to always respect the last selection the user made
in the car.
Let's see how last user mode should work.
Notice that I'm using CarPlay,
and it's showing on the car's display.
Now if I disconnect, CarPlay ends, and the car falls back
to its native user interface.
If I plug in again, CarPlay should appear on the display
because that was the last thing I was doing in the car.
So if CarPlay was the last thing on the display
when it was disconnected, or the car was turned off,
CarPlay should be shown when iPhone is connected again.
If CarPlay media was playing, CarPlay media should play again.
Let's see a different example.
This time, CarPlay is active
with the car's display showing the native user interface.
Now if I disconnect and then plug in again,
CarPlay does not appear on the display
because the native user interface was the last thing I
was using in the car.
So if CarPlay was not the last thing shown, it should remain
in the car's user interface.
But even if CarPlay was not the last thing on the screen,
if CarPlay media was playing, CarPlay media should play again.
CarPlay is available the moment the user plugs in their iPhone,
but they can switch between the CarPlay display
and the native user interface at any time.
As we saw earlier, the user returns
to the car's user interface by selecting your brand's icon
in the CarPlay Home screen.
They can switch back to CarPlay
by using the Apple CarPlay button.
The Apple CarPlay button is the most important indication
that CarPlay is available.
As soon as CarPlay starts,
the Apple CarPlay button must become active
in your top level menu.
And the Apple CarPlay must only appear when iPhone is connected
and CarPlay is active.
Be sure to hide the Apple CarPlay button
when iPhone is disconnected.
And they also show the Apple CarPlay button
in an enabled state when CarPlay is active
and in a disabled state when CarPlay is inactive.
You can optionally replace an existing button the top level
menu with the Apple CarPlay button such as a phone button.
Just remember that the Apple CarPlay button must appear
on your car's Home screen whenever iPhone is connected.
When iPhone is disconnected,
the Apple CarPlay button must disappear or be disabled.
Let's talk about the Apple CarPlay button
that should appear in your interface.
There are a variety of approved Apple CarPlay logos available
to use as buttons in your car's native user interface.
Choose the one that makes the most sense with your design.
There's a black logo and a white logo as well as white
and black variance with backgrounds.
And, finally, there's a color background.
Let's talk about how media sources are handled
when iPhone is disconnected.
Let's say that CarPlay is active,
and the user is listening to CarPlay audio playback.
When iPhone is disconnected, it's a strong indication
that they're about to leave the car.
Your system should gracefully handle CarPlay media
When iPhone is disconnected, the user should hear silence
and audio should not fall back to another audio source.
If the iPhone is disconnected while CarPlay is displayed
on the car's display, you should gracefully fall back
to the last screen displayed before entering CarPlay.
Don't show a CarPlay unavailable or device disconnected screen.
So, again, when CarPlay media is playing,
and the user disconnects iPhone, they should just hear silence.
The hardware buttons in your car should also be used
to support CarPlay.
Siri is an integral part of CarPlay
and requires a dedicated tactile button.
Because it's so important, the button should be
on the car steering wheel.
Users expected Siri to work at any time, whether
or not CarPlay is currently visible on the car's display.
They also expect the Siri button to work
like the Home button on iPhone.
Pressing and holding it for 600 milliseconds should activate
Siri, and the activation delay must not be greater
than one second.
The media buttons in your car should be connected
to CarPlay media playback so users can play, pause,
move to the next track or previous track,
and fast forward and rewind.
Finally, if your car has shortcut buttons for phone,
media, or navigation, they should also be mapped
to the respective apps within CarPlay.
Now let's talk about apps that work in CarPlay.
Apps supported by CarPlay will appear
on the CarPlay Home screen in the vehicle.
Three categories of apps are supported.
Audio, those specific
to particular vehicles, and messaging apps.
If a user has an audio app that's been updated for CarPlay,
that app will appear
in any CarPlay car the user connects to.
The user interface for these apps is fixed
with the structure provided by iOS
and the content provided by the app itself.
You'll notice that audio apps look a lot
like the built-in music app.
Users choose what to listen to via a list-based UI,
and the Now Playing screen shows the appropriate content
The app needs to have a special entitlement in order
to be compatible with CarPlay.
We've made some improvements in iOS 10
to make the audio apps experience even better.
New in iOS 10 is support for tabs, which make it easier
to navigate list hierarchies.
In the Now Playing screen, you can now see the app name
and additional playback control buttons.
In the list views, you can tell which item is playing,
which are available for streaming,
explicit or live stream.
Some apps are designed to play audio as soon
as they're launched, and we've added support
for these apps in iOS 10.
All of these combined make for a much improved experience.
Check out the new tabs, the Cloud icon for streaming apps,
and the way tend to note what's currently playing.
In the now playing screen, here's the app name
and the additional playback controls.
Now let's talk about that next category, apps written
to work in specific vehicles.
Apps published by automakers are written
to control vehicle functions
or give information about the vehicle.
They only appear in vehicles specific to that automaker.
A CarPlay entitlement is required for these apps as well.
New in iOS 10 is Siri support for automaker apps
that control vehicle functions.
Siri now has built-in support for changing radio stations,
changing climate controls, and more.
The automaker app communicates with the vehicle
to perform these functions
after Siri tells the app what the user would like to do.
Let me explain how automaker apps work only
in the cars that support them.
An app icon is enabled in the CarPlay Home screen only
if the iPhone is connected to a head unit
that supports the same protocol name as the app.
A protocol name supported by the head unit
and the app need to match.
iAP external accessory is used for app
to head unit communication.
Head units can declare multiple protocol names in order
to support current and future apps
with different groups of target cars.
Head units should use a protocol name
to match all cars from that brand.
For example, com.brand.
They should also use protocol names to match cars
with specific features, for example,
electric vehicles or sports models.
A protocol name for the specific model should also be included
so that apps that target just
that model can be published in the future.
The head unit needs to use iAP2 to declare
at least one supported external accessory protocol.
Likely more than one.
Start and stop external protocol session must also be supported,
and external accessory protocol CarPlay must be set to true.
If the car supports wireless and wired CarPlay,
these iAP2 messages should be implemented
on all supported transports.
Most automaker apps will communicate with the car
to perform tasks such as changing the climate controls.
These apps need to design
and implement a communication protocol as well.
So how does the app support this feature?
The app's Entitlements.plist needs to be modified
to include the protocol names that match those
on the target head units, as shown here.
With these additions plus support for the protocol
to communicate with the car, and a CarPlay entitlement,
your app will be ready for CarPlay.
We're looking forward to seeing some great CarPlay apps
We have a new category of apps supported in CarPlay
and iOS10, messaging apps.
When a messaging app has been updated to support CarPlay,
that app will appear on the CarPlay Home screen.
Messages can be received and sent,
all driven by Siri interactions.
We're excited to extend CarPlay capability to messaging apps,
making that experience smarter and safer.
We hope this has been a useful overview of CarPlay.
CarPlay is a great way to keep the user experience
in vehicles fresh long into the future, and we're delighted
to get a chance to talk to you about the features
that are new in iOS 10.
For more information, check out this session's WWDC URL.
We also have a follow-on CarPlay deep dive video session
that may interest you.
Thank you for watching.
Looking for something specific? Enter a topic above and jump straight to the good stuff.
An error occurred when submitting your query. Please check your Internet connection and try again.