Design great visionOS apps

Hi, I’m Sarah, a design Evangelist. You may have met some of the fantastic folks on my team in the Vision Pro Labs we hosted worldwide over the past year.

Our job is to help share techniques and guidance for designers and developers building on Apple platforms.

In this session, I’ll be sharing what makes a great experience on visionOS and why.

You’ll see apps from a handful of developers that we worked with who have experiences that we think are unique, thoughtful and raise the bar for quality. You’ll even get to hear directly from some of the team members about their insights and learnings. There’s a lot to cover, so let’s jump in. It’s important that the core idea of your app or game is intentional and built for the platform. So I’ll start at a high-level and talk about your product solution on visionOS.

Then, I’ll unpack what it means to embrace immersion and show solutions that excel at this in different ways.

visionOS is designed with people at the center of the experience to ensure interactions are comfortable and your apps should be too.

And the best apps are delightful because they achieve a high-bar of quality with an extraordinary attention to detail First, let’s get started by exploring how visionOS opens a world of possibilities.

When we first work with a team we encourage developers to find their key moment, which is the moment in an app that’s optimized for visionOS. The Mindfulness app is an example on the system. The flower has a gentle pulse that encourages you to breathe and focus. When the flower expands it gives a sense of life to the scene. It’s an experience only possible on visionOS.

Finding your key moment will be different for every app, but let’s take a look at some strategies teams took for building spatial solutions.

One approach is to think of ways your app can make new things possible.

JigSpace is an app that helps visualize 3D assets for demos and pitch decks. Take a look at this jet engine! Notice the stunning level of detail, how I can view the electrical wiring inside and deconstruct the model piece…by… piece.

As a baseline, assets need to be photorealistic with a high degree of visual quality to feel convincing and compelling. To make new things possible, bring your content to life. Consider how animations can replicate real scenarios, like you see here with air simulation in the intake fan. Or provide ways to manipulate models that would be cumbersome in the real world. Like taking them apart, swapping out materials, or changing the design in real time.

Great solutions take time to build and they’re a result of iteration and prototyping.

This is Loona. The iOS app offers gracefully animated scenes that combine relaxing activities, storytelling, and atmospheric sounds to help you calm down or fall asleep.

Bringing this UI into a windowed experience on visionOS would have been reasonable. But listen to how the team approached it. This is Sergey Gonchar, the co-founder of Loona: We just put ourselves in the mindset, we’re going to build a lot of prototypes, because we want to find the best possible experience we can shape for this platform.

Take a look at the early sketches of their scooter concept. The 2D prototypes felt dimensional, they were begging to be broken into pieces to play with. And it took time, but these sketches evolved into 3D models, which took on a life of their own, with tons of fun, incredible detail and subtle animations.

On Vision Pro, Loona is a relaxing spatial game with 3D meditative puzzles that transform into vibrant, animated scenes. This game uses all standard gestures. You tap to pick up a piece, hover it over the position, and release it into place. The scene come to life as you complete the puzzles and it feels really rewarding. This approach was a slight deviation from their iOS app, but the interactive elements and 3D scenes are really fun and appropriate on a spatial platform. The last consideration I’ll share for thinking about intentional experiences is to build something complimentary. This is a great approach if your app has a large feature set that may be less relevant on the platform. Lowe’s is an excellent example. Their iPad and iOS apps are chock-full of features from inventory to store maps and product calculators. On Vision Pro, they’ve built an app entirely tailored to visionOS called Lowe’s Style Studio. You can step into an immersive 3D kitchen scene and customize everything from your backsplash to appliances.

Bringing this experience into a virtual space makes it easier to explore ideas, discover how different products look together and refine your design direction and personal style.

They complete the consumer journey by having full integration cross-platform. Your style boards save across devices for continuity. This is where you can shop for individual items, check inventory, or find products in a nearby store.

JigSpace, Loona, and Lowe’s are very different products. But each solution had deliberate focus when it came to translating their experiences to Vision Pro. They were intentional in their approach.

When building for the platform consider native frameworks like ARKit and RealityKit, to take full advantage of device capabilities. Take the upfront time to build and test ideas to learn how your apps will best excel on visionOS.

Great apps on Vision Pro have considered the spectrum of immersion. There are many ways to do this, I’ll share three inspiring examples.

Immersive apps can take people somewhere new, and you can do this with Full Immersion. Sky Guide by Fifth Star Labs excels at this.

They transport you to your very own planetarium where constellations come to life as you gaze across the night sky. You can choose any location in the world to star gaze, and even see the borealis.

You can pluck constellations out of the sky to learn more or use your hands as binoculars to get a closer look.

Great environments replicate an accurate sense of depth and scale to bring you into a space that’s a convincing representation of something recognizable, relevant and real. But keep in mind, custom Environments are not relevant for every app on Vision Pro, so don’t assume that this is the only way to immerse people. I find this is often overlooked, but consider how you can integrate with peoples’ physical surroundings. Let’s take a look at Super Fruit Ninja by Halfbrick. As you slice fruit with your hands notice how the juice splatters and drips down your walls, the remnants of the fruit bouncing off surfaces in your real space. Because sensors in the device scan the walls and surfaces of your surroundings, the game is able to feel super responsive, playful and unique each time I use it. They also thoughtfully considered how their characters, like Truffles the pig, interact with you and your space. This is Samantha Turner, the Lead Gameplay programmer for Super Fruit Ninja. He can chase after strawberries and fruit. You can play fetch with him, and pat him, and he integrates so well into your real world. I’ve even seen him jump up on my desk on my bed and it’s really delightful when that happens.

Immersion is all about exploring ways to make people feel like a considered extension of your app or game.

Another way to leverage immersion is to design meaningful auditory moments with thoughtful sound design.

To clarify why sound can be a type of immersion, it might be helpful to first understand how visionOS handles sound.

Sensors on the device understand the characteristics of your physical space, like the size and materials of the room. Then, the system adds reverberation to make it sound like the things you’re interacting with are really with you. This intricate level of recognition and accurate play-back make Spatial Audio on Vision Pro really powerful.

In the same way that haptics on iOS can be a form of feedback for people, think of audio as not just ambient music, but an opportunity to guide, encourage and reward people. An exceptional example is the the puzzle game Blackbox by Shapes and Stories.

Each of these bubbles host unique puzzles that leverage the device’s technology and encourage exploration of your world. This game is challenging, creative, and if you’re struggling to solve a puzzle, sometimes a little embarrassing. You’ll have to give it a try to know what I mean. Sound is fundamentally part of the experience. Let’s hear from Gus Callahan, the sound designer of Blackbox. For the resonating sound we asked, “How can we translate this elasticity, to convey something spatial and experiential?” So we wanted the sound to stretch with it, for the player to interact with it, like an instrument.

When all of these design elements come together, you get quite close to a one-to-one sensory experience.

Sound is a great way to immerse people, make your apps come to life and feel more memorable.

visionOS is designed for the unique considerations of input, comfort and usability. Let’s discuss why this matters and how to apply it to your app.

All of the apps I’m sharing today require minimal physical movement for people to enjoy them. Vision Pro is a device you wear on your head and control with your eyes, and you can’t predict the physical spaces of where your app will be used. In Super Fruit Ninja, the experience does feel active because I’m physically involved when I’m slicing fruit with my hands. But it doesn’t require a person to walk in their space. Their play area has this geometric guide on the ground to encourage the player to stay within this boundary during gameplay. I think it’s a really clever way to let people know that movement isn’t needed. And even though models within JigSpace can be life-size, like this F1 car, I can walk around it if I choose to, but I can also scale it down incase my physical space doesn’t allow me to walk around it.

Minimizing movement is about keeping people safe and understanding that there are variables in peoples physical space that you can’t control.

To keep apps ergonomically comfortable, minimize the amount of windows needed and keep your interface cohesive.

Great apps on Vision Pro are built for the infinite canvas. This term is a nod to the idea that apps are no longer bound by the size of a screen. This affords a lot more flexibility, but there are still guardrails you should consider to make sure your app feels cohesive and avoids confusion.

The content and UI of your app should be contained in a window. Controls like Tab bars and Toolbars can live outside the window, but they are still anchored to the view. This gives people a clear delineation of what your app offers and how to navigate its content. Making it more comfortable to reposition and arrange in their physical space. Spatial doesn’t mean buttons and UI should be arbitrarily floating in peoples field of view. An app that has taken their solution further with the infinite canvas, and designed a cohesive experience, is PGA Tour.

The main window of the app contains live streams and tournament information. And a volume represents the corresponding course and replays of a shot trail.

This is great, because the solution is centered around providing enhanced sports entertainment with augmentation that’s contextual.

I also like how the app doesn’t require separate windows for this content, which would have resulted in maintenance by the viewer to organize and move them around. This lockup feels cohesive and easy to manage. Visually comfortable experiences are those that feel part of the operating system which was designed to blend with the world.

Like the Glass material used for windows. Since there isn’t a concept of light and dark mode on visionOS, Glass is rendered through the system and intelligently adapts to the lighting in peoples environment.

One thing we discussed with teams a lot is the topic of branding.

There are few companies that have such recognizable and established branding as Red Bull. This is Kelly FitzGerald, a Senior Product Designer at Red Bull. So in our TV and mobile apps we used a dark blue background color as a cohesive branding element for RedBull TV. When we started adapting to Vision Pro, however, the passthrough came into play. So in this case, we had to decide whether to prioritize a prominent brand element or usability. The Red Bull TV app has opted to highlight their branding with high-quality, visual imagery that is so inherent to their identity.

But when you scroll the main view, the rest of the window uses the Glass background, opposed to a solid background color. Find ways like this to incorporate your branding without compromising comfort or usability.

It can be visually distracting and uncomfortable to use solid backgrounds on windows. They don’t adapt to our environments lighting and they block our view of the world. Here’s another thoughtful approach the Red Bull team had: We saw an opportunity to briefly incorporate our dark blue brand color just as images were loading. It’s these small branding moments that can really snowball to paint a larger picture. Elements on visionOS should have a hover effect. This is a highlight that gives people feedback about where they’re looking.

Carrot Weather is an app that makes viewing the weather playful and fun. The app has a lot of character and is well-crafted on multiple platforms.

Even though the interface requires a lot of data and information, this app does a great job of using hover effects correctly.

Every element that's interactive and has a detail view, like hourly temperatures, uses a hover effect. But the data that is meant to be read like details around dawn and dusk, don’t require a hover effect. And having one would be more confusing and complicate the interface.

Keep in mind, interactive elements need sufficient tap targets, so use at least 60 points of space.

If hover effects are too small, a person may feel like they need to be very precise to interact with your app and that becomes frustrating and uncomfortable. When hover effects are designed properly they instill confidence in someone’s interactions. Now, I’ll go a little bit deeper with just one app to share how exceptional craft leaves people feeling delighted.

This is one my favorite apps on visionOS, DJay by Algoriddim.

Delightful experiences match our expectations of reality and honor our spatial awareness.

DJay is exactly what people would expect from a DJ’ing app: a completely realistic DJ setup with two turn tables for your records, complete with tonearms and tempo syncing. The turn tables are placed in front of you, within arms reach, which compels people to interact with them directly. In this case, you can scratch the records, pickup the tonearms and slide the tempo in the same way you would with a physical turn table.

We expect objects in arms reach to be interactive, because they’re close to us. When designing spatial experiences, this is an important principle to follow. Keep interactive content close to people, even though they can reposition it wherever they'd like, or interact it with indirectly.

Our awareness of content around us is also heavily driven by animation and motion.

When you launch the app, the first thing you need to do is select a track. Notice the pulsing stroke outline around the music icon. This is a subtle way to guide you to tap on the music library. It’s a simple detail with a big impact. Movement and animation are great ways to get peoples attention and let them know something is interactive because we’re inherently sensitive and receptive to it.

They mapped the interactions to familiar experiences by building custom gestures that have parity with the real-world. Have you ever seen a DJay hold their headphones up to one ear? What they’re doing is previewing the next track to prepare for a seamless mix between the song they’re currently playing. And guess what? You can do this in DJay too. If you raise one of your hands to your ear you’ll hear the upcoming song through that speaker.

In this scenario, you don’t need to be wearing physical headphones, the gesture just mimics this familiar action for queuing audio.

But translating this experience presented some interesting challenges. They realized they had to account for false positives, such as when someone may be raising their hand to fix their hair or adjust the strap on Vision Pro. Here’s how Karim, the CEO and co-founder of Algoriddim, approached this: This required a lot of user testing. We had to gather feedback to ensure the gesture was accurately recognized. Getting feedback is so important. You may be surprised by the way people use your app and certain behaviors they have that may be different than your expectations. Understand the opportunities where it’s possible to abstract away the literal interpretation in order to make something more approachable and less complex. For DJay, let’s take a closer look at the turn table.

Turn tables have existed in their analog form for quite some time, but the technology surrounding them has evolved tremendously. For example, matching tempos and syncing tracks can be complex on traditional hardware. There’s a barrier to entry for people who aren’t familiar with it. The Algoriddim team saw this as an opportunity. In the 3D environment, we asked ourselves, “Why a DJ would even need to manually match tempos? When we could completely automate the process under the hood?" This abstraction not only reduces complexity, but also enhances the user’s ability to focus more on the creative aspects of DJing.

Removing complexity can help people enjoy the aspects of your app that resonate the most.

OK. Before we wrap up, here are some resources that may help you get started building your own visionOS app.

Apple’s human interface guidelines are living documentation of the latest best practices. When you’re curious about a pattern or component, check here first and see what the latest guidelines are.

And our design resources are built to help you expedite your prototyping. The Figma and Sketch kits have extensive libraries with native components, so you don’t have to rebuild these assets yourself. And every app I talked about today is available on the App Store for Apple Vision Pro, so I encourage you to give them a try for yourself. Thanks for listening!