Handling of WebRTC Audio with split VPN tunnel

App & System Services Networking
fabianbaechli OP
Created Oct ’22
Replies 7
Boosts 1
Views 1.3k
Participants 2

Our app lets users connect to their homes via an OpenVPN connection. Specifically, we want them to be able to receive and send audio data from and to the intercoms which they have in their homes. For us, establishing the VPN connection and communicating with the home works like a charm. But sending or receiving the aforementioned audio data isn't possible. While the phone and the intercom are able to establish a SIP connection, they aren't able to send each other any audio data. Using the packet capturing tool on our firewall, we can see, that the audio data isn't being passed into the VPN tunnel but rather it is being sent over the local network connection. This affects only the audio data, meaning that for every other type of network traffic, communication over the VPN is working fine. If we route all traffic over the VPN tunnel via networkSettings?.ipv4Settings?.includedRoutes = [NEIPv4Route.default()], the audio data is being sent and received successfully (this isn't a viable solution for us though). Our guess is, that the packets aren't being sent to the network interface, to which they should be sent according to the routing table on the phone.

While going through the a sysdiagnose of the whole process, this stood out to us: Error acquiring assertion: <Error Domain=RBSServiceErrorDomain Code=1 "(originator doesn't have entitlement com.apple.runningboard.assertions.webkit AND originator doesn't have entitlement com.apple.multitasking.systemappassertions)"... because it matches the time of the establishing of the call. Could this be the culprit? I've seen people mention this error in combination with problems of sound-playback in the WebView.

Generally speaking, this wasn't a problem in iOS 14 and it affects our OpenVPN implementation in the app, but has the same result when we use our OpenVPN configuration with the official "OpenVPN Connect" app. To reproduce:

  • Use sip.js (newest version) to connect over a VPN split tunnel to an Asterisk server
  • Call another party which is connected to the same server
  • Be able to establish a SIP call to that other party
  • But notice how the audio packets aren't being sent over the VPN tunnel, although they are addressed to the exact same Asterisk server
Answered by DTS Engineer in 735075022

But isn't the WebRTC implementation of WebKit choosing which network interface to bind the UDP traffic to?

Quite possibly. I’m familiar with this issue from the networking side, I don’t have any expertise in how that breaks down on the web side.

You have a number of potential paths forward here:

  • WebKit is open source. If its WebRTC infrastructure is included in the open source, you could dig into that to see what it’s doing.

  • You could also discuss this with WebKit experts via WebKit’s numerous community communication channels.

  • You could seek formal support from Apple by opening a DTS tech support incident.

  • You could file a bug about the change in behaviour as compared to iOS 14. If you do, please post your bug number, just for the record.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Replies  7
Boosts  1
Views  1.3k
Participants  2
DTS Engineer OP
Apple
Oct ’22

While going through the a sysdiagnose of the whole process, this stood out to us

This is a red herring. To quote the runningboardd man page [1]:

runningboardd is a daemon that manages process assertions to ensure those processes are kept in the appropriate state while assertions are in effect.

This has nothing to do with networking.

Coming back to your real issue, I’m a bit confused about the original of these audio packets. Is that code native to Safari? Or a web view? Or JavaScript code running within one of those?

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] Which is for macOS, but it does the same thing on iOS.

1
fabianbaechli OP
Oct ’22

Thank you for the response. The Audio Packets originate from JavaScript running inside a WKWebView. Specifically, we use the library sip.js to establish a sip connection and gather the microphone data. Sip.js uses WebRTC to collect the microphone data.

I hope this helps, Cheers

0
DTS Engineer OP
Apple
Oct ’22

The Audio Packets originate from JavaScript running inside a WKWebView.

OK. That makes things a bit trickier. You’ll have to burrow down through the JavaScript to find out exactly how it’s sending that UDP traffic.

Keep in mind that a VPN in destination IP mode does not receive all packets for the destinations it claims. If the program binds to a specific interface, by specifying the source address or with an option like IP_BOUND_IF, it’s traffic will go over that interface. This is common for UDP apps.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

0
fabianbaechli OP
Oct ’22

Thanks for the reply. We have verified that the audio packets really are being sent over the Wi-Fi network interface instead of the VPN tunnel. How can we route all packets addressed to a certain IP over our VPN tunnel? It seems to us that TCP packets are being passed into the tunnel correctly (for example: establishing the sip connection, these TCP packets are being sent correctly to the remote address) but UDP packets addressed to the same remote address seem to be passed to the Wi-Fi network. This is the configuration of our tunnel (all packets addressed to 201.253 should be passed into the tunnel):

func openVPNAdapter(_ openVPNAdapter: OpenVPNAdapter, configureTunnelWithNetworkSettings networkSettings: NEPacketTunnelNetworkSettings?, completionHandler: @escaping (Error?) -> Void) {
  networkSettings?.dnsSettings?.matchDomains = [""]
   
  let includedRoutes: [NEIPv4Route] = [NEIPv4Route(destinationAddress: "192.168.201.253", subnetMask: "255.255.255.0")]
  networkSettings?.ipv4Settings?.includedRoutes = includedRoutes
   
  setTunnelNetworkSettings(networkSettings, completionHandler: completionHandler)
 }

Thanks for your help!

0
DTS Engineer OP
Apple
Nov ’22

How can we route all packets addressed to a certain IP over our VPN tunnel?

You can’t, at least not with a packet tunnel provider in destination IP mode. If the sender binds their flow to a specific interface, any off-device packets will be sent via that interface.

You have to look at the sender to find out why it’s binding to the Wi-Fi interface rather than the interface that leads to the destination IP address. It’s not uncommon to see this in UDP code, because UDP presents some unique challenges in this regard.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

0
fabianbaechli OP
Nov ’22

But isn't the WebRTC implementation of WebKit choosing which network interface to bind the UDP traffic to? We couldn't find anything of that matter in the Sip.js code. Also important to remember here is, that this worked in iOS 14 and still works on Android with the same sip.js implementation.

0
DTS Engineer OP
Apple
Nov ’22
Accepted Answer

But isn't the WebRTC implementation of WebKit choosing which network interface to bind the UDP traffic to?

Quite possibly. I’m familiar with this issue from the networking side, I don’t have any expertise in how that breaks down on the web side.

You have a number of potential paths forward here:

  • WebKit is open source. If its WebRTC infrastructure is included in the open source, you could dig into that to see what it’s doing.

  • You could also discuss this with WebKit experts via WebKit’s numerous community communication channels.

  • You could seek formal support from Apple by opening a DTS tech support incident.

  • You could file a bug about the change in behaviour as compared to iOS 14. If you do, please post your bug number, just for the record.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

0
Handling of WebRTC Audio with split VPN tunnel
First post date Last post date
 
 
Q