Inconsistency in FD creation of NWConnection on server side when compared within system (using localhost) and between systems.

During my exploration of NWConnection and NWListener I created sample code to implement communication between client and listener over UDP. To achieve that client and listener were created on the same machine using localhost. Same client and listener codes were also deployed on different macOS machines. Following are the observations:

Observation1:

Within the same macOS machine, Listener was created on a port X and from client side I used "localhost" and Port X to connect to the listener.

To capture the behavior of the how the connections/FDs are getting created I used lsof & netstat commands and following were the output:

LSOF: tw-macoffice-02-studio:~ nikhil.singh$ lsof -Pi :9001

COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME

NWListner 16775 nikhil.singh 4u IPv6 0x402b8381d3a12129 0t0 UDP *:9001

NWListner 16775 nikhil.singh 5u IPv6 0x402b8381d3a11129 0t0 UDP localhost:9001->localhost:64723

NWConnect 16785 nikhil.singh 4u IPv4 0x402b8381d3a10529 0t0 UDP localhost:64723->localhost:9001

Here, we can see that separate FDs are getting created for:

  • NWListner - Listener FD - 4u and NWConnection (connection request) from listener to client FD - 5u on localhost with Process ID 16775

  • NWConnect - NWConnection from client to listener on localhost FD - 4u Process ID 16785

NETSTAT:

tw-macoffice-02-studio:~ nikhil.singh$ netstat -an |grep 9001

udp4 0 0 127.0.0.1.9001 127.0.0.1.64723
udp4 0 0 127.0.0.1.64723 127.0.0.1.9001
udp46 0 0 *.9001 .

Connection wise also we can see the two connections are listed.

Observation 2:

I used different macOS machines within the same network to implement listener and client side of the code. Here, the behavior in terms of socket FD creation on the listener side with incoming connection request was different.

Listener Side: LSOF: nikhil.singh@Mac-Pro ~ % lsof -Pi :9001

COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME

UDP_Serve 31480 nikhil.singh 5u IPv6 0xb3b7cb5ed6d2edd7 0t0 UDP *:9001

NETSTAT:

nikhil.singh@Mac-Pro ~ % netstat -an |grep 9001

udp46 0 0 *.9001 .
udp4 0 0 10.20.16.144.9001 10.20.16.250.62758

Inconsistency: **For listener side, a new socket FD was not created when listener accepted the incoming connection. **

Client Side: LSOF tw-macoffice-02-studio:~ nikhil.singh$ lsof -Pi :9001

COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME

NWConnect 29015 nikhil.singh 4u IPv4 0x402b8381d3c74129 0t0 UDP 10.20.16.250:62758->tw-macoffice-01.tallysolutions.com:9001

NETSTAT

tw-macoffice-02-studio:~ nikhil.singh$ netstat -an |grep 9001 udp4 0 0 10.20.16.250.62758 10.20.16.144.9001

For Client Side, a new FD was created from client to listener.

Question: Why a new socket FD was not created on the Listener side when incoming connection was from a different machine over IP.

Also, one Digression question: As a cross platform application, we plan to use Network Framework for Apple Kernel and BSD sockets for others. Similar to BSD, is there a way we can block a thread on connection.receiveMessage?

Apple platforms include a user-space networking stack. Network framework objects, like NWConnection and NWListener, have multiple back-end implementations, including one that talks to the user-space networking stack and one that talks to the kernel via BSD Sockets. lsof works entirely in terms of file descriptors and thus won’t ‘see’ Network framework objects that are using the user-space networking stack.

If you want to examine the user-space networking stack state in your process, check out the skywalkctl tool.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Inconsistency in FD creation of NWConnection on server side when compared within system (using localhost) and between systems.
 
 
Q