Advanced UDP with Network.framework

Quoting TN3151 Choosing the right networking API:

For UDP flows—where you have a stream of unicast datagrams flowing between two peers—Network framework is the best choice … However, not all UDP communication is that straightforward … If you need something that’s not supported by Network framework, use BSD Sockets.

The exact limits of Network framework’s UDP support are kinda fuzzy, but your situation:

My app … uses UDP networking to create a mesh between multiple devices where each device acts as a server and client at the same time.

is likely to be outside those limits.

I have a question about your real product, not your test case. You wrote:

It does that by creating one UDP socket on each device bound to a *:<port> and then uses that socket to sendmsg to other peers.

So, the packets on the wire contain UDP datagrams where both the local and remote port values are <port>?

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Thanks @eskimo for your insights, as always.

We use http://enet.bespin.org for reliable peer to peer communication, it's very old and very reliable UDP library, not dissimilar to QUIC - except that it existed way before QUIC. It provides multiple independent streams of data (reliable and unreliable) - on top of one UDP flow. It was created for multiplayer games and is super easy to work with.

I have a nice and small Swift based API written for that, in production on Linux and iOS/macOS since 2014 - that I plan to open source at some point.

I want to move it over to Network.framework - to make it work on the Apple Watch where BSD sockets do not work, and to gain performance by moving away from BSD sockets. Networking guys (from Apple) I had a chance to talk to claimed that moving to Network.framework will make everything faster...

I will try to explain as easy as possible what the problem is. Please bear with me :)

On each device you bind one UDP socket to a random port. And that socket/port is used for all outgoing and incoming communication with the other peers. The UDP datagrams contain a simple protocol that handles streams, retransmits, datagram re-ordering, etc.

So imagine App1 opens UDP Socket1, and binds it to ::Port1.

App2 opens Socket2, and binds it to ::Port2.

App3 open Socket3, and binds ::Port3.

Communication from App1 to App2 goes via local Socket1 leaving the machine on Port1 and going to App2 via Socket2 and remote Port2.

Communication from App1 to App3 goes also via local Socket1 leaving the machine on Port1 and going to App3 via Socket3 and remote Port3.

Communication from App2 to App3 goes also via local Socket2 leaving the machine on Port2 and going to App3 via Socket3 and remote Port3.

Replies the other way around.

In the BSD sockets world this is possible even on the same networking interface, and in the same UNIX process, because Socket1, Socket2, and Socket3 are independent file descriptors in the kernel and sendmsg and recvmsg can use these sockets to send to and receive from any UDP address. Via IIRC something that is called "unconnected UDP socket" mechanism.

And you essentially use the same fd for both sendmsg and recvmsg.

In the Network.framework world, Listener1 will allow me to receive UDP datagrams on Port1. Listener2 on Port2, and Listener3 on Port3.

Connecting from App1 (Port1) to App2 (Port2) is only possible by creating new NWConnection with requiredLocalEndpoint set to Listener1.requiredLocalEndpoint.

This works fine, as long as Port2 is on another networking interface or another machine.

But it breaks when Port1, and Port2 are used via NWListener or NWConnection in the same (UNIX) process and on the same networking interface. Say for example in one XCTestCase.

So if I want to create a peer1 (NWListener1) and peer2 (NWListener2) both bound to localhost, and then open a NWConnection1to2 it will break.

If I bind peer1 to en0 and the peer2 to lo0 it will work.

I have no insight into how is the Network.framework implemented, and also this only affects NWListeners and NWConnections using UDP, in the same process, and on the same interface. So not a real world use case, but only a testing use case.

Sorry for the long and chaotic description. I have a simple swift package demonstrating the code ready, and if you want I can open a DTS request to investigate this further.

At the moment I am proceeding in a way that for XCTestCase I create one peer using Network.framework and the other peer using BSD sockets and shuffle data between them.

But I would love to verify all the functionality when both peers use Network.framework. Which is easier over lo0.

I am thinking along the lines of enumerating all interfaces on the machine and binding peer1 to one interface (say en0) and the other to another interface (say en1) so that they can communicate together successfully.

On each device you bind one UDP socket to a random port. And that socket/port is used for all outgoing and incoming communication with the other peers.

OK. In theory that should be compatible with Network framework’s UDP support.

But it breaks when Port1, and Port2 are used via NWListener or NWConnection in the same (UNIX) process and on the same networking interface.

Right. I think this is a known issue. It’s related to the issue discussed here, but it’s not exactly the same.

Consider the program pasted in below. This starts two connections, with the flow tuples:

• localIP / 12345 / 93.184.216.34 / 23456

• localIP / 12345 / 93.184.216.34 / 23457

This should be feasible because UDP flows are uniquely identified by their tuples, and these tuples are distinct.

However, when you run it you get this [1]:

connection 23456 did change state, new: preparing
connection 23456 did change state, new: ready
connection 23457 did change state, new: preparing
connection 23457 did change state, new: waiting(POSIXErrorCode(rawValue: 48): Address already in use)

The second connection is failing with EADDRINUSE.

This result confirms that your on-the-wire protocol won’t work with the current Network framework. You absolutely need to be able to start two connections to different peers with the same source port. Given that, I encourage you to file your own bug. Please post your bug number, just for the record

Note that doesn’t involve NWListener at all; it’s just two outgoing connections. Adding NWListener into the mix is not going to improve things |-:

to make it work on the Apple Watch

You’ve read TN3135 Low-level networking on watchOS, right?

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] I’m testing on 14.3.1 but this isn’t a new problem.

import Foundation
import Network

let localPort: NWEndpoint.Port = 12345

var connections: [NWConnection] = []

func startFlow(remotePort: UInt16) {
    let params = NWParameters.udp
    params.allowLocalEndpointReuse = true
    params.requiredLocalEndpoint = NWEndpoint.hostPort(host: "0.0.0.0", port: localPort)
    let conn = NWConnection(host: "93.184.216.34", port: .init(rawValue: remotePort)!, using: params)
    conn.stateUpdateHandler = { newState in
        print("connection \(remotePort) did change state, new: \(newState)")
    }
    conn.start(queue: .main)
    connections.append(conn)
}

func main() {
    startFlow(remotePort: 23456)
    startFlow(remotePort: 23457)
    dispatchMain()
}

main()

Thanks @eskimo for the explanation. You nailed it completely. So I reused your example code which explains the bug much more clearly than my original description and submitted a bug FB13678278.

Our app (https://cloudbabymonitor.com) actually back deploys to iOS 12 at the moment so I hoped to switch to Network.framework with the next update that will support iOS 13+, but this bug is a stopper, as our app relies at opening multiple UDP flows from the same local port, and that seems to be broken, and I am not sure we will ever see this fixed in iOS 13+ (since iOS 12 is no longer receiving updates).

As for the watch, we do stream audio and video at the same time, so on the Watch that should satisfy the requirement for using the Network.framework. However it all comes down to the final experience. At the moment we can establish successfully UDP audio/video stream between two iPhones on WiFi in about 1 second - and that includes Bonjour discovery, DNS resolution, and everything else. I'm not sure yet I can get to 1 sec on the Watch from looking at the wrist and seeing live video, but I want to have the code ready and see it running and try to get it that fast purely for the engineering pleasure of it :) - if not for the happiness of our customers who beg for the Apple Watch live video feature now for years already :)

thanks for your help, Martin

This is a bit off-topic, but hoping one of you might reply. I just learned about the Network framework. In an introductory WWDC talk, they show a live-streaming video example, but the code isn’t available (sadly).

There’s a reference to “breaking the frame up into blocks” because of delivery over UDP: I assume this is because of message lengths?

At any rate, if someone can give me a quick idea of the strategy of sending video frames from device to device, over UDP (which must assume some things can get lost), I’d greatly appreciate it. I assume UDP has message length constraints in Network, but I don‘t see them mentioned.

Surely I can’t just send an entire 2K jpeg image (i.e. 1920x1080 pixels) in one UDP message. Or can I?

I assume UDP has message length constraints in Network

Not in Network framework, but rather in the UDP protocol. UDP’s maximum datagram size is just under 64 KiB. Beyond that, you really want to avoid sending datagrams that overflow the MTU, because that results in IP-level fragmentation (which is Bad™).

I talk about this in more detail here. And it terms of how to actually fragment and reassemble, we’ve been talking about it recently on this thread.

Finally, keep in mind that TCP and UDP aren’t the only game in town these days. You have the option of using:

• QUIC, which gives reliable transfer without the head-of-line blocking problem you get with TCP

• QUIC datagrams, which is unreliable like UDP but potentially with a larger datagram size [1]

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] The size limit is controlled by the receiver but it’s expected to be 65535. That’s useful if your typical payload is larger than the MTU but smaller than the UDP limit.

Thank you for your reply (and, indeed all the advice and monitoring you’ve been doing for the past few years — I‘ve read most of your posts.)

One thing that I just can’t discern, through neither documentation, nor discussions, is what happens when I try to send a large message over UDP through Network framework. Given that there are no constraints on the size of a message in Network, let’s suppose I (stupidly) just send 1 megabyte of Data as a single message.

Does Network:

1. Say, “forget it!” (i.e. error). That’s just too big.
2. Break it into large packets (anything way over the MTU) and send it (which would require reassembling it?)
3. Break it into packets of approximately MTU size (say between 500 bytes and 1500 bytes) which again requires Network to reassemble on the other side.

Suppose I ask to send 30K as a single message:

1. Does Network just send this as a single large packet? (I assume “yes”)
2. Or does Network still break it down into more MTU sized chunks, again requiring reassembly by Network on the other side? (I assume “no”)

Last question: I understand fragmentation is “bad”. Does this mean if I opt to use Network and UDP, I should always try to break my messages up into chunks of between 500 and 1500 bytes (depending on what I believe the MTU is)?

Thanks for the QUIC note, which I am unfamiliar with. I will read about this, and maybe that’s what I want.

—————————-

My actual scenario is trying to use my iPad to see the camera stream from my iPhone, which is located perhaps 10 to 20 feet away from the iPad, but up on a tripod (perhaps in a dockkit accessory). I want to both see what the iPhone sees and possibly control the positioning of the iPhone in the dockkit accessory.

So yes it is streaming video, but from extremely close range. Assuming Network forms a peer-to-peer connection in places where I don’t have a wifi network (say, outside!) this would fantastic.

is what happens when I try to send a large message over UDP through Network framework.

Nothing good.

I thought it might reject the send but it seems to send a truncated packet.

Actually, the results are kinda weird. Consider the code pasted in below, which attempts to send a 66K datagram, that is, beyond the UDP limit. Looking at a packet trace, I see that:

• The UDP length seems to be set to the request length mod 64 KiB.

• The actual packet contains data beyond the UDP length, up to the limit of the link MTU.

Suppose I ask to send 30K as a single message:

So that’s valid for UDP, because the limit is just under 64 KiB site. In that case it’ll fragment the packet at the IP level.

I should always try to break my messages up into chunks of between 500 and 1500 bytes (depending on what I believe the MTU is)?

Yes. You don’t need to guess though; instead, look at the maximumDatagramSize property.

I should always try to break my messages up into chunks of between 500 and 1500 bytes (depending on what I believe the MTU is)?

Yep. You do have to opt in to this by setting includePeerToPeer.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

import Foundation
import Network

func main() {
    let conn = NWConnection(host: "example.com", port: 12345, using: .udp)
    conn.stateUpdateHandler = { newState in print(newState) }
    DispatchQueue.main.asyncAfter(deadline: .now() + 1.0) {
        var message = Data()
        for i in 0..<8250 {
            let i64 = UInt64(i)
            let bytes = (0..<(UInt64.bitWidth / 8)).reversed().map { byteNumber -> UInt8 in
                UInt8((i64 >> (byteNumber * 8)) & 0xff)
            }
            message.append(contentsOf: bytes)
        }
        conn.send(content: message, completion: .contentProcessed({ error in
            print(error)
        }))
    }
    conn.start(queue: .main)
    withExtendedLifetime(conn) {
        dispatchMain()
    }
}

main()