Sound advice
June 9, 2022
An immersive app doesn’t only look and feel great — it has to sound incredible, too. During WWDC, we spoke with Apple Design Award finalists about the sensational sounds of their apps and games. Come with us as we travel on a musical journey through cool jazz, Spatial Audio soundscapes, and even original album-length compositions from artists like Arcade Fire, St. Vincent, and Madlib.
Please, Touch the Artwork: A jazz thing
Thomas Waterzooi's elegant puzzle game Please, Touch the Artwork is inspired by Dutch painter Piet Mondrian, who, with his iconic style of lines, squares, and primary colors, is considered a pioneer of 20th century abstract art. And that kind of guy? He needs the right kind of music.
“I tried to imagine what Mondrian would listen to while painting in his workshop,” says Waterzooi, the game’s Brussels-based developer. “Some kind of jazz. And since the game is designed to be relaxing, it would have to be a calm, dynamic kind of jazz.”
That mix of timeless art and cool soundtrack creates a classy vibe in Please, Touch the Artwork, whose puzzles are based on three of Mondrian's most famous works: Composition with Red, Blue, and Yellow; Broadway Boogie Woogie; and New York City I.
As the puzzles grow and change, so does the music, which was created by composer Lars Burgwal. The music for the New York City section begins with only bass — and as you progress through each puzzle, the piano, saxophone, and vibraphone all come to play. (Waterzooi also added a little drum flourish whenever you tap a painting.) “With puzzle games, the music has to be relaxing," says Waterzooi. “It can't annoy you at any point.”
So not so much Broadway boogie woogie for Broadway Boogie Woogie? “It would be too fast!” laughs Waterzooi. “We couldn’t swing nearly that much.” He has, however, worked in a little nod to the style.
"The goal of that game is to join characters named Boogie and Woogie,” he says, “and when you do, there’s a little completion animation with a musical accent. It’s not much — just three or four notes — but it’s based on some boogie-woogie right-hand piano schemes.”
Download Please, Touch the Artwork from the App Store
Odio: Absolutely Spatial
Audio apps don’t get much more immersive than Odio. The Apple Design Award-winning 3D audio app employs a mesmerizing mix of Spatial Audio and head tracking to conjure up its AR soundscapes.
While you might flow between a rushing waterfall, the deep sea, and even a world of calming digital ambience, you’re no passive listener in these realistic realms: Each soundscape can be manipulated through a clever system of arcing sliders that reposition each sonic element — a rushing river, dreamy whalesong, or wash of digital static — around your head in 360 degrees.
Max Frimout is the app's audio engineer, and though his work is heavy on synthetic, otherworldly digital elements, his audio career started with something considerably more analog. “I was originally a harpist,” he says. "One day I opened the ES1 Synthesizer in Logic Pro, and now I’m here!"
Odio originally focused on nature sounds, but after a few months of development, the Netherlands-based team at Volst wanted more. “‘What if we have musicians compose their own environments?’” says Roger Kemp, co-founder and designer at Volst. “That’s when it all clicked.”
Frimout is also one of the app’s five composers. A musician and DJ by trade, he began creating his Odio soundscapes with lines of melody, then layered in effects and flourishes with names like “synthetic water,” “moving chords,” and “filtered drone.” Soundscapes are built in Logic Pro and tested with AirPods Max. “That’s how I look around to hear how it feels,” he says.
Most of Frimout’s compositions are the result of sonic experimentation, but the soundscape called “Wow!” followed a more organic path. “I started with a series of melodies that basically all came to me in the same evening,” he says. “I think that shows how you can have all this equipment and all these concepts but still be incredibly inspired by a single event.”
And yes, it contains harp: That’s Frimout playing on the loop called Heartbreak — though you might not recognize the sound as strings. "It’s just three chordal structures,” he says with a laugh, “but they’ve been processed and processed and processed.”
Download Odio from the App Store
A Musical Story: That ‘70s game
A Musical Story is inspired by a very groovy time: “It’s all about the freedom of ‘70s music,” says Charles Bardin, the French composer/developer who created the game with art director Alexandre Rey, composer Valentin Ducloux, and developer Maxime Constantinian. “Mostly, we were inspired by the sense that, back then, anything could happen.”
Conceived in 2017 and launched in March 2022, A Musical Story is a harmonious mix of song, narrative, and art. The story follows an up-and-coming band trying to break into the business, replete with vintage guitars, outfits, and hairstyles. To move the narrative along, you tap your screen to the beat, creating some great soul- and R&B-inflected music in the process.
But the game is mostly wordless, driven by the primal, powerful connection between music and memory. It’s an ideal playground for Bardin, who studied at the Conservatoire de Musique de Lyon and who’s been creating and covering game music for more than a decade.
As it happens, the development process didn’t begin with the music — Bardin and Rey started by establishing the circular tap-along play mechanic. “In most games, the notes come down on the screen and you play them when they arrive,” says Bardin. “I love that, but it’s also something you can play without any sound. I wanted a game that really relies on listening.”
Once the team landed on the mechanic, it was time to tune into the songs themselves. "We knew we wanted short sequences of music to unlock the story,” says Bardin, “but a lot of musical games rely on electronic or techno music, where the beat is very clear. We wanted to prove that we could make more organic music — something that wasn’t quite so thump-thump-thump-thump.”
He also made sure the music drove the story along. “I wrote a song called Her for a scene in which the character goes to a pub, sees a girl playing music, and instantly falls in love with her,” says Bardin. “It begins with just a Rhodes piano and some bass and drums, but as you move closer to the stage, you hear more and more of the music. When you get close enough, you discover her face and her voice.” It’s the only time vocals appear in the game itself aside from the credits. “We wanted this moment to be powerful,” Bardin says. “This is the voice of the most important character in the game.”
Download A Musical Story from the App Store
Headspace: The music of mindfulness
Over the past few years, the meditation and mindfulness app Headspace has partnered with A-list musical artists to help people concentrate, relax, lock in, or nod off. With Focus Music (found, appropriately, in the Focus tab), the app has amassed an array of original music and playlists from artists like Arcade Fire, St. Vincent, Erykah Badu, Madlib, and even film composer Hans Zimmer.
Focus Music was designed in part by John Legend, the app’s chief music officer. “There’s so much possibility right here on our phones,” says Legend. “It can be a scary thing for some artists; it’s not what we’re used to. But if we take advantage of the possibilities, there are all these different ways to reach people.”
The singer-songwriter Aluna trained in reflexology, transcendental meditation, and tai chi — all skills she wove into her hour-long Headspace composition. To create it, she designed six-minute blocks of sound, grounded in specific spaces like a crackling campfire, bustling park in late afternoon, or dripping cave.
Strictly speaking, it was not her usual approach. “Normally when you write a song, you’re doing wordplay and you want dynamics,” she says. “It’s completely different from music that for an hour has no start and no finish.” (It’s also more complicated than it sounds — there’s a lot of difference between the sound of water dripping from a cave and dripping from your faucet.)
The science at the intersection of music and mindfulness is clear, says UC Berkeley cognitive neuroscience professor Sahar Yousef, who partnered with Headspace on Focus Music. “We know that when we play music in rehab facilities, people improve quicker,” Yousef says.
Here’s the (extremely abridged) explanation of what’s going on when you listen: Your brain forges connections via neural networks, the little zaps of electricity that constitute all your thoughts. The good news is that these networks can be manipulated, and you’re probably doing it right now. You can train yourself to think that the aroma of coffee means it’s time to wake up, and you can train your brain to recognize the music designed to chill you out.
In other words, these soundscapes serve as little life hacks. “Michael Phelps listened to Eminem before every race,” says Yousef. “This is the same thing.”
Download Headspace from the App Store
More stories
