Howdy,
I'm hoping to start a conversation about getting access to live audio data for items playing from Apple Music.
As background, we've been authoring and selling music visuals on OS X and other platforms for over 13 years. We are behind some of the visualizers that reside in iTunes to this day. Our latest product is an iOS app called Tunr, and it features a unique UI for audio/streaming services and protocols, unifying the user music experience in the way iTunes 1.0 set out to do -- most importantly, we seek to pull multiple music sources under a single roof and do it in the context of mobile and streaming. For easier reference, grab Tunr from the app store and play with it.
As we rollout Tunr, we're getting requests to add support for Apple Music. MusicKit, like other ReST interfaces that we support (SoundCloud, Spotify, iHeartRadio, etc), fits into our model by design. However, because MusicKit requires playback through MPMusicPlayerController, there isn't a way to tap into the live audio/PCM samples (so that the user sees Tunr's realtime visuals react to the currently playing audio). In contrast, for example, the Spotify SDK has its own player API but exposes an AUGraph, allowing Tunr to call AudioUnitAddRenderNotify() where we can read the PCM data for visualization.
In general, not offering access to raw PCM data traces back to DRM policies originating in the 2000s. I also submit that the way apps like Tunr use live audio data is disjoint from the DRM politics and debate. However, in practice, we all know it's hard to disentangle the two, and that’s likely why AVPlayer also never had a live audio tap. For this reason, we had to develop our own player for Tunr.
As a proposed solution, I’d like to suggest that MPMediaPlayback be extended such that a callback gets downsampled audio PCM data to, say, 5k-11k samples/sec. This sample rate is too low for it to be "ripped" but would be sufficient for rendering visualizations.
In any case, I at least wanted to stimulate conversation on this topic. Alternatively, it would be to get access to the audio stream to decode and play for a given Apple music item, but I'm pretty sure that's a non-starter.
Suggestions? Thoughts? Other ideas?
Andy O'Meara
SoundSpectrum CTO
