macos 26 - socket() syscall causes ENOBUFS "No buffer space available" error

As part of the OpenJDK testing we run several regression tests, including for Java SE networking APIs. These APIs ultimately end up calling BSD socket functions. On macos, starting macos 26, including on recent 26.2 version, we have started seeing some unexplained but consistent exception from one of these BSD socket APIs. We receive a "ENOBUFS" errno (No buffer space available) when trying to construct a socket(). These exact same tests continue to pass on many other older versions of macos (including 15.7.x). After looking into this more, we have been able to narrow this down to a very trivial C code which is as follows (also attached):

#include <stdio.h>
#include <sys/socket.h>
#include <string.h>
#include <unistd.h>
#include <sys/errno.h>

static int create_socket(const int attempt_number) {
    const int fd = socket(AF_INET6, SOCK_STREAM, 0);
    if (fd < 0) {
        fprintf(stderr, "socket creation failed on attempt %d,"
                " due to: %s\n", attempt_number, strerror(errno));
        return fd;
    }
    return fd;
}

int main() {
    const unsigned int num_times = 250000;
    for (unsigned int i = 1; i <= num_times; i++) {
        const int fd = create_socket(i);
        if (fd < 0) {
            return -1;
        }
        close(fd);
    }
    fprintf(stderr, "successfully created and closed %d sockets\n", num_times);
}

The code very trivially creates a socket() and close()s it. It does this repeatedly in a loop for a certain number of iterations.

Compiling this as:

clang sockbufspaceerr.c -o sockbufspaceerr.o

and running it as:

./sockbufspaceerr.o

consistently generates an error as follows on macos 26.x:

socket creation failed on attempt 160995, due to: No buffer space available

The iteration number on which the socket() creation fails varies, but the issue does reproduce. Running the same on older versions of macos doesn't reproduce the issue and the program terminates normally after those many iterations.

Looking at the xnu source that is made available for each macos release here https://opensource.apple.com/releases/, I see that for macos 26.x there have been changes in this kernel code and there appears to be some kind of memory accountability code introduced in this code path. However, looking at the reproducer/application code in question, I believe it uses the right set of functions to both create as well as release the resources, so I can't see why this should cause the above error in macos 26.x.

Does this look like some issue that needs attention in the macos kernel and should I report it through feedback assitant tool?

Answered by jaikiran in 873033022

Once you’re done, please post your bug number, just for the record.

Thank you Quinn, I've now filed FB21686886.

And thanks, as always, for your efforts here. Your test suite is certainly doing its bit to keep us honest (-:

I'm happy to help :) The OpenJDK testsuite is indeed vast. One of our bigger challenges in recent times has been trying to get a resolution for several stability issues/regressions with BSD socket APIs on macos. I'm glad that these dev forums exist and there are knowledgeable engineers from Apple who regularly respond and provide help. That helps stay motivated to investigate these issues and report them. Feedback assistant tool on the other hand is a bit of a disappointment - lack of any response and visibility on the filed issues there for several months/years even for basic questions makes it a bigger challenge to take such reported issues to a proper resolution. Hopefully that tool/process will see some improvement in future.

Thanks again for the help.

I don't have knowledge of the xnu kernel code, nor do I have an easy/quick way to debug/investigate this issue myself. However, a very brief look at the socket code seems to suggest that when a socket fd is closed using "close(fd)", like in that reproducer, the necessary decrement of memory accounting doesn't seem to be happening.

Very specifically, it looks like the sodealloc(...) in bsd/kern/uipc_socket.c seems to be responsible to "releasing"/decrementing the memory accounting counter here https://github.com/apple-oss-distributions/xnu/blob/xnu-12377.61.12/bsd/kern/uipc_socket.c#L797. Looking at the soclose_locked(...) function in that same file, it calls sofree(so) here https://github.com/apple-oss-distributions/xnu/blob/xnu-12377.61.12/bsd/kern/uipc_socket.c#L1332. The implementation of sofree(struct socket *so) then calls sofreelastref(so, 0) https://github.com/apple-oss-distributions/xnu/blob/xnu-12377.61.12/bsd/kern/uipc_socket.c#L7315. The second param there is dealloc and is being passed a value of 0 indicating "don't deallocate". So the implementation of sofreelastref(...), then skips the deallocation call to sodealloc(...) https://github.com/apple-oss-distributions/xnu/blob/xnu-12377.61.12/bsd/kern/uipc_socket.c#L1074 - the one which is responsible for decrementing the memory accounting counter.

Like I said, I have zero knowledge of this code, so it's possible that I maybe looking at completely unrelated code. If so do let me know (that will help me in future when looking into such issues).

After looking into this more, we have been able to narrow this down to a very trivial C code which is as follows (also attached)

Looks like I forgot to attach that file (at least I can't see it attached in this thread) and I can't edit the original post. So attaching it here:

should I report it through feedback assitant tool?

Yes please. I have general info on that process in Bug Reporting: How and Why? In this case specifically, please include:

  • The reproducer you’ve shared here
  • The analysis from your second post

Once you’re done, please post your bug number, just for the record.

And thanks, as always, for your efforts here. Your test suite is certainly doing its bit to keep us honest (-:

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Once you’re done, please post your bug number, just for the record.

Thank you Quinn, I've now filed FB21686886.

And thanks, as always, for your efforts here. Your test suite is certainly doing its bit to keep us honest (-:

I'm happy to help :) The OpenJDK testsuite is indeed vast. One of our bigger challenges in recent times has been trying to get a resolution for several stability issues/regressions with BSD socket APIs on macos. I'm glad that these dev forums exist and there are knowledgeable engineers from Apple who regularly respond and provide help. That helps stay motivated to investigate these issues and report them. Feedback assistant tool on the other hand is a bit of a disappointment - lack of any response and visibility on the filed issues there for several months/years even for basic questions makes it a bigger challenge to take such reported issues to a proper resolution. Hopefully that tool/process will see some improvement in future.

Thanks again for the help.

macos 26 - socket() syscall causes ENOBUFS "No buffer space available" error
 
 
Q