DeviceDiscoveryExtension basics, can't get sample project to work

Question

I'm trying to understand what exactly is made possible by Media Device Discovery Extensions, what responsibility the containing app has, and what exactly is made available to other apps or the system, if anything.

I haven't been able to find any meaningful high level documentation, and WWDC 2022 session 10096 only mentions these new extensions in passing. The most comprehensive body of information I found is the example project: https://developer.apple.com/documentation/devicediscoveryextension/discovering_a_third-party_media-streaming_device?changes=latest_beta&language=objc

However, I don't think it's working the way it should out of the box:

I've got the Client target app built and running on an iPad Pro running iPadOS 16 Beta 2
I've got the MacServer target running on a Mac Mini with macOS 13 Ventura Beta 2
I've got the Server target running on an iPhone with iOS 15.5. (Non-beta)

If I tap the AirPlay icon on the Client's video player, I can see the two servers, but selecting one just causes a spinner to show up next to its name. This keeps going for a while, eventually the spinner goes away again, but the device selection tick stays next to 'iPad'. The text "Select route" also doesn't change, which I think it's supposed to, judging by the code.

I've tried a variety of combinations of settings on the servers - bluetooth only, bonjour only, different protocols, etc., but I'm always getting the same behaviour.

Has anyone had any success in getting the example to work, and how? Is there any high level documentation available that I've missed?

Can someone explain what exactly we can build with this in more detail than "implementations of custom A/V streaming protocols?" The WWDC session video talks about 3rd party SDKs, so do these extensions have to be embedded in every app that would be streaming the video, implying that it's not useful for mirroring?

1.3k

Posted by

pmdj

Reply

Update: I've now tested between the iPad Pro and an iPhone 8 both running iOS/iPadOS 16b2, using first the one and then the other as client, and I'm still not getting anywhere with the demo project.

—
pmdj
I can see no apparent change in the behaviour of the demo project (which I redownloaded in case it had changed) when using iOS/iPadOS 16 beta 3, Xcode 14 beta 3, and macOS 13 beta 3. Again, any servers that have started advertising show up on the client's video player's AirPlay target list, but selecting one just causes the activity spinner to appear and spin for a while. No video/audio played on the server, no tick next to the selected streaming target.

—
pmdj

Add a Comment

Answer 1

I wanted to address a few things that you mentioned in your description:

I've got the Server target running on an iPhone with iOS 15.5. (Non-beta)

I would run this on a physical iOS 16 + device as stated in the project description. This target takes advantage of new APIs such as IOServiceGetMatchingService that may not work properly on a iOS simulator or older OS that does not support this API.

Regarding:

Can someone explain what exactly we can build with this in more detail than "implementations of custom A/V streaming protocols?" The WWDC session video talks about 3rd party SDKs, so do these extensions have to be embedded in every app that would be streaming the video

A good description of these capabilities is found here in the DeviceDiscoveryExtension. These APIs work to discover local network services or Bluetooth devices to connect and stream AV data back and forth from your device to the other local network device. So, if you have a 3rd party device that is sitting on your local network that you want to stream video from, then these API would give you the option to do so.

Posted by

meaton

"I would run this on a physical iOS 16 + device as stated in the project description." -

Thanks; I did suspect that, which is why I also tried the Mac server on Ventura, with the same effect. (Although the IOServiceGetMatchingService call is only made on macOS as far as I can see, iOS uses UIDevice.current.identifierForVendor ?? UUID() which seems like it should work on iOS 15 as well.

I'll try to identify another device we can safely move to iOS 16 beta and try again.

—
pmdj
"A good description of these capabilities is found here in the DeviceDiscoveryExtension." - I did read that before posting, but it left me with some ambiguity. To be abundantly clear: if we want to enable owners of our device to stream to it, do we need to persuade the developer of every app they want to use it with need to embed our extension, or can we ship it in our app to provide the streaming target system wide? Obviously the demo project would answer that if I could get it to work…

—
pmdj
I have now tried the demo project in a combination of an iPad Pro 11" 3rd gen and iPhone 8, both running iPadOS/iOS 16 beta 2, in both combinations of roles, as well as the iPhone 8 with the MacServer on macOS 13b2. All combinations fail in exactly the same way, the activity spinner just spins and spins and eventually disappears again; the tick never appears next to the selected server, and no video/audio is ever output on any server. It seems like this should work, so I have filed FB10511946.

—
pmdj

Add a Comment

Answer 2

To be abundantly clear: if we want to enable owners of our device to stream to it, do we need to persuade the developer of every app they want to use it with need to embed our extension, or can we ship it in our app to provide the streaming target system wide? Obviously the demo project would answer that if I could get it to work…

The short answer is that you can provide one app / extension that allows the discovery of local devices on the network that is providing a Bonjour service endpoint. Once that service is discovered, it will be up to your app / extension to delegate how the AV content is communicated over the local network connection.

Posted by

meaton

Add a Comment

DeviceDiscoveryExtension basics, can't get sample project to work

Replies