C10k and number of sockets

hello,


the wiki article for C10k problem states this:


>>> The term [C10k problem] was coined in 1999 by Dan Kegel, citing the Simtel FTP host, cdrom.com, serving 10,000 clients at once over 1 gigabit per second Ethernet in that year. The term has since been used for the general issue of large number of clients, with similar numeronyms for larger number of connections, most recently "C10M" in the 2010s.


By the early 2010s millions of connections on a single commodity 1U server became possible: over 2 million connections (WhatsApp, 24 cores, using Erlang on FreeBSD), 10–12 million connections (MigratoryData, 12 cores, using Java on Linux) <<<


i wonder how is it possible to have a single physical server supporting more than 64K open connections. is there no limitation on open sockets?


as this is an apple forum let's presume this quesion is about OSX (though relevant comments for other platforms are welcomed).

This topic, albeit very interesting, does not look like it was posted in the right location. This forum is focused on Apple API and specific Apple development questions. High level topics such as this will need to be discussed in another forum.


Matt Eaton

DTS Engineering, CoreOS

meaton3 at apple.com

is there no limitation on open sockets?

This whole discussion assumes that every network connection requires a socket. This isn’t the case on most Apple platforms, which have a user-space networking stack that you can access via the Network framework [1].

I’m not aware of any hard limit to how many connections can be run by the Network framework. Feel free to test it and file a bug if you run into any inconvenient limits.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

[1] The one exception here is macOS, where Network framework has to run through the kernel in order to support NKEs. This is one of the reasons we’re in the process of phasing out NKE support, starting with there deprecation in the macOS 10.15 SDK.

what is the right Apple forum for a question: "what is the maximum number of open sockets on macOS?"

>This whole discussion assumes that every network connection requires a socket. This isn’t the case on most Apple platforms, which have a user-space networking stack that you can access via the Network framework


Quinn, thanks, that's interesting.


Then maybe it is possible to create an alternative implementation of user-space socket API on top of Network framework, so that the socket API and apps that use it also benefit?


Obviusly i had macOS in mind, hard to imagine C10k server running on iPhone or apple watch 🙂

Then maybe it is possible to create an alternative implementation of user-space socket API on top of Network framework

This is not feasible, alas. BSD Sockets has to implemented in the kernel because sockets themselves are file descriptors, and file descriptors need to work in all sorts of kernel-y contexts (like in a call to

select
).

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

maybe that's a huge undertaking... but calls like "select" can be wrapped as well, i believe. normally user apps do not care (or even know) whether they are talking to a system version of read/select, etc or some wrapper library - provided the API exposed is the same.

calls like

select
can be wrapped as well, i believe.

I’ve seen people go down this path before. It does not end well.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

> This whole discussion assumes that every network connection requires a socket. This isn’t the case on most Apple platforms, which have a user-space networking stack that you can access via the Network framework [1].


what are the underlying kernel/system primitives/APIs Network framework using? i assume the Network framework itself is not open source, is the underlying layer opensource?

The underlying infrastructure is

libnetwork
, which is not public API nor open source.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

This whole discussion assumes that every network connection requires a socket. This isn’t the case on most Apple platforms, which have a user-space networking stack that you can access via the Network framework [1]. [1] The one exception here is macOS, where Network framework has to run through the kernel in order to support NKEs. This is one of the reasons we’re in the process of phasing out NKE support, starting with their deprecation in the macOS 10.15 SDK.

Hi Quinn, it's been a while since you wrote the above. Do you know if macOS is still an unfortunate exception that requires a socket per Network framework's connection? cc @eskimo

C10k and number of sockets
 
 
Q