visionOSのホバーインタラクションのデザイン

Hey I'm Nathan, a designer on the Apple Design Team. Welcome to this session on designing hover interactions for visionOS. With new ways to respond to where people look, your apps can feel more alive and make people feel like their mind is guiding the experience.

Here's the order for today: we'll review the fundamentals of eye input, explore custom effects, make content scroll just by looking, and see how to persist important controls. Let's jump in. As you may know, people navigate visionOS with their eyes and hands. They look at an element and tap their fingers together to select it. Here's a few quick reminders to make sure your apps work well with eyes and hands.

Place your most important content in front of people, so it's easy to see and use.

For interactive elements, prefer rounded shapes, like circles, pills, and rounded rectangles, they're easier to look at, drawing your eyes to the center of the shape.

For precise interactions, give each element at least 60 points of space. Elements can be visually smaller than this, as long they each have a total space of 60 points.

And for 3D objects with a fixed scale, 60 points is roughly equal to an angular size of two and a half degrees, which is about 4.4 cm for an object one meter away.

Apply highlight effects to all interactive elements. Standard components like this menu will highlight automatically.

For your custom components, add the highlight effect and make sure its shape matches the shape of the content.

And apply these highlight effects to selectable 3D objects too.

To learn more about the basics, check out the session "Design for spatial input" from 2023.

Today, let's talk about more powerful interactions for your apps, starting with custom effects.

The standard highlight effect works great in most cases, but you can extend it or replace it with your own animations. We call these custom hover effects. We use custom hover effects throughout visionOS.

Tab bars pop open to show the names of each tab.

Back buttons grow to show the title of the previous page.

Sliders show a knob to invite interaction.

Tooltips appear below buttons to describe what they do.

In Safari, the navigation bar expands to show your browser tabs. And in the Home View, environment icons reveal more of the landscape.

You can create custom effects like these in your apps too! They're perfect for giving feedback, while maintaining the personality of your app or visual style of your game.

But first, let's understand a little more how they work. To protect people's privacy, the system applies hover effects outside your app's process, so your app doesn't know where people are looking.

Instead of responding to a hover event directly, your views define two states: their standard appearance and their hovered appearance. The system animates between these states when people look at the view, or look away.

This is great for playing animations like we've seen so far but means custom effects cannot be used to perform actions within your app. Let me show you what I mean. Say I'm designing a photo-browsing app. I could add a download button to let people save their favorite photos.

I could use a custom effect to show the download size when people look at it, since this is just an animation. But I couldn't perform the download action just by looking at the button, since this would require my app to know when the effect is applied.

Instead, people will need to perform a tap gesture to save the photo.

Okay, so custom effects drive animations. What kind of animations work best? They generally fall into three buckets: Instant, Delayed, and Ramp animations. The first are "Instant" animations which start right when people look at a view In Mindfulness, the buttons show an arrow icon so people know more choices are available.

And the standard video player shows a timestamp when looking at the playhead. This extra information is small, contextual, and not interactive.

Sometimes instant animations are too quick since they cause motion as people are trying to look around so we can use "Delayed" animations instead. These usually show more content after a short delay.

Tooltips are a great example. This delay allows people to press the button quickly, and only displays the title if someone shows extended interest.

Another example is the Safari profile button, which follows the same pattern with a different layout.

Last are "Ramp" animations. They're like delayed animations but with a hint at the start and a custom curve.

Check out the environment icons in the Home View. Right as people look at them, they start scaling up slowly, then pop open to reveal more of the landscape. This scaling effect makes it clear that the icon will expand if you continue to look at it, but doesn't fully open until people show clear intent. This lets people look across the grid without icons popping open along the way.

Here's another example, where this card expands to reveal the full text. These ramp animations are a balance between "instant" and "delayed" animations. They give immediate feedback, without being distracting.

To create your own ramp animation, try a curve like this. It starts with a slow ease-in, then pops to completion with a quick spring. This animation curve tends to work best when views expand to show more content. As you're starting to think about these effects in your apps, here are some best practices to keep in mind.

First, provide anchoring elements.

The best custom effects keep part of the view the same. These static elements, like the title here, are anchoring — they give people context and help them understand what's new.

If text shifts when people look at a view, the motion can interrupt their reading, so try to keep text in a fixed position.

Or if a view is completely different in its two states, people often get confused about what they’re looking at.

And hover effects should always start from a visible element. If there's no hint about where something is, it'll be hard to find or surprising if people accidentally discover it.

Instead, if you have hidden elements, show them when looking at something visible. Here for example, the resize control appears when looking at the corner of the window. This helps people find the controls they're looking for.

Next, apply custom effects carefully.

Think about where custom effects could provide focus, like on this compass. Show additional information like the location status, or create delightful moments like an animating waterfall.

But don't change too much as people look around. Too many custom effects like showing the name of each pin can distract from your content, or even cause visual discomfort. Even a small scale effect might be distracting for high-usage views like toolbar buttons or table cells since they move the things people are trying to look at. For views like these, stick with the standard highlight effect Try to keep effects small.

The best custom effects are subtle, and work well on small views, like this download button we saw earlier.

Avoid applying custom effects to large views, like this entire photo, since it causes a lot of motion and can be difficult for people to know what's happening.

And for imagery like this, avoid effects that wash out the colors.

A good solution is a highlight effect that gives instant feedback, then fades away so people can see the true colors of the photo. And this applies to 3D objects like this plant. It highlights, then fades to show its real colors.

Last, avoid unexpected motion. This is something I run into a lot. You'll think you've got a good idea but it ends up having side effects. Let me show you. Safari's tab overview shows a grid of open tabs.

You might have a great idea: what if we hide all the close buttons by default, and only show the close button when looking at a tab? Looks cleaner, right? But when you look at a tab and the button appears, your eyes instinctively jump to it. This may cause people to accidentally close the tab when trying to select it, since they're now looking at the close button while performing the tap gesture.

Instead you could try something like this: fade the button in slowly halfway when people look at the tab, then fade it fully when people look at the button.

This still reduces the visual noise in a large grid of tabs, but doesn't make people accidentally close them.

It's critical to try these effects on a real device! It's impossible to experience them them by watching a video since they react to where you look. And hey, I love the visionOS simulator too, okay, but it's no replacement for testing on a real device.

Designing custom effects is an iterative process, more than other parts of the interface. Be prepared to try lots of options and spend time tuning to get it just right.

Over time you'll build an intuition for what works and what doesn't, but if you have an idea — try it! It might just work. Next, let's talk about another powerful interaction: Look to Scroll. Look to Scroll lets people scroll just with their eyes. Here in Safari, when I look at the bottom of the page, it smoothly animates up.

Or in Music, when I look at the last playlist, it scrolls toward the center.

This makes browsing easier because your eyes follow the content, almost pulling it into view. It's a lightweight interaction that works alongside scrolling with hands.

Scrolling starts when people look near the edge of the scroll view, along the top and bottom for vertical scroll views, or along the sides for horizontal scroll views.

Your app may have many scroll views and Look to Scroll isn't enabled by default, so you'll need to pick views should opt-in. To decide, here's a few things to think about: First, if your view is primarily for reading or browsing, opt-in.

Look to Scroll is great for reading an article in apps like Safari or browsing for the next show in apps like TV.

But in apps like Settings, people want to quickly look through to find the right option, and they don't read down the list one-by-one, so Look to Scroll isn't the right fit here. In general, if your view is mostly UI controls, don't enable Look to Scroll.

Next, consider your content. Generally, Look to Scroll works best on the primary content of the app. So here in Notes, the note itself scrolls when people look at the edges, but the list on the left does not, following the pattern we just saw with Settings.

Finally, think about consistency within your app.

Here in TV, there's a list of related shows, which look just like the list in the main library, so people expect Look to Scroll to work here too.

Now that you're getting a sense for which scroll views to opt-in, let's see some ways to make scrolling feel natural and predictable.

Ideally, your scroll views with Look to Scroll enabled should be the full width or full height of your window. This gives people generous space to scroll and provides clear edges to look at.

If your scroll view is inset from the window, provide clear boundaries so people know where to look.

And if your app uses the scroll position to animate content at a different speed, apply parallax effects or drive custom animations consider adjusting your design to scroll at the normal rate. Or maybe Look to Scroll simply isn't the right choice for these views.

But when your scroll view fills the window, has clear edges, and scrolls normally, browsing will feel smooth and predictable. Finally, let's look at persistent controls, a subtle feature that makes a big difference. Here's a standard video player on visionOS. The playback controls show and hide with a tap. And if the controls are up, they'll automatically hide after a delay. This works well, but the controls might hide while you're looking at them! So now, we keep the controls visible while you look at them, and hide them when you look back at the video. You might not even notice it at first, because it just works how you'd expect.

If your app uses a standard video player, either inline, windowed, or immersive, you'll get this behavior for free. But if your app uses custom video controls, you'll need to enable this behavior. And it's great for more than just video apps! FaceTime persists the call controls and Mindfulness persists the session controls. Any time you have UI that auto-hides after a delay, whether it's for a video or an immersive experience, adopt this persistence behavior.

As we've seen, there's a lot of ways to bring your apps to life with hover interactions: animating your interfaces with custom effects, scrolling views just by looking at them, and persisting media controls.

As you design your apps with these features in mind, start by trying a few ideas and see what works! Look for opportunities to apply custom hover effects in your apps. You can create them in SwiftUI or RealityKit, and you can learn more about how to build them in the sessions here from 2024. To enable Look to Scroll and persistence behaviors, check out the documentation on developer.apple.com Thanks, and have fun!