BSD socket APIs and macOS entitlements

please correct me if I misunderstand

Entitlements only apply to the sandbox. They "entitle" the sandboxed app to break free of the sandbox for very specific needs.

Those Java tools would not be distributed as sandboxed executables.

Entitlements only apply to the sandbox. They "entitle" the sandboxed app to break free of the sandbox for very specific needs.

Thank you @Etresoft for that input - that was useful. I now read up on App Sandbox in macos here https://developer.apple.com/documentation/Security/app-sandbox. From what I understand of that, sandboxing an application is a choice that's left to the application itself. So if an application isn't sandboxed then access to resources/operations (like the networking operations) aren't enforced at runtime through entitlements. An application is mandated to be sandboxed if the application is distributed through the Mac App Store. Given all this, it then means that the entitlements enforced for a sandboxed app aren't applicable to the JDK (since it isn't sandboxed).

Having said that, while reading through that doc, I found another construct in macos called the "Hardened Runtime" https://developer.apple.com/documentation/security/hardened-runtime. This too states that applications decide whether they want hardened runtime. However, importantly, and something that's relevant for the JDK, that document states that for an application that will be notarized (the JDK will be), it must enable the hardened runtime. That same page mentions that when an application is run with hardened runtime then certain entitlements are necessary if the application does some specific operations. That page lists the exact entitlements necessary for a hardened runtime and in context of the JDK, some of them are relevant. In that list, I don't find any networking related entitlements. So I take it that an hardened runtime doesn't enforce restrictions on networking resources/operations through entitlements and thus the JDK doesn't have to apply for these networking entitlements when running in a hardened runtime. Again, please correct me if I got this wrong.

Thanks again for the help so far.

No correction necessary. That's all correct. Something complicated like the JDK would likely need entitlements for the hardened runtime if notarization was desired.

But you're starting to get into obscure technical details here. Why do you care about the JDK anyway?

For the vast majority of Mac apps, the Mac App Store sandbox is all you ever need to worry about. For all practical purposes, you don't have to worry about anything more complicated than the sandbox unless you have guaranteed external funding - and a lot of it - for some very specific end users.

Look at it this way, if you are considering this JDK-based app for someone that you don't already know by name, you are in deep, deep trouble.

No correction necessary. That's all correct. Something complicated like the JDK would likely need entitlements for the hardened runtime if notarization was desired.

Thank you for those inputs.

But you're starting to get into obscure technical details here. Why do you care about the JDK anyway?

I contribute to the OpenJDK project and in recent releases of macos, we have been seeing various odd networking failures on macos, involving Java applications that do very basic operations with sockets. They are hard to debug or narrow down. So I'm going through the macos manuals and trying to understand some of the changes in the recent releases and the general guidelines about packaging applications.

Some of these failures I believe are related to the recent "Local Network" restrictions which have been introduced in 15.x of macosx. I will create separate thread for that one, but before getting to that, I wanted to make sure I understand some of these existing primitives in macos.

Thank you for your helpful inputs so far.

Anything involving local network restrictions should be easy to debug. Construct a program that fails. Then try it on a non-local connection. If it consistently works there and fails locally, then you've definitively confirmed that it's related to local networking.

However, there were more changes in macOS 15 than just local networking. There are two big issues that end users encounter.

Firstly, macOS 15 adds a new feature called "Private WiFi address" and defaults it to "rotating". As someone who sometimes does interesting networking myself, I immediately changed that to "fixed". This is not my problem to deal with. You can also change this setting to "fixed" and see if that eliminates the problem. Most people encountering problems are willing to change this setting to "fixed".

Next, macOS 15 mades some low-level networking changes that broke many VPNs and network filtering apps, including Apple's own Application firewall. It turns out that most people really don't understand networking and believed that the Application firewall was a required security layer. Typically those folks added additional 3rd party filters and VPNs on top of that. From a technical perspective, this one is easy to solve too. Just don't do any of that. From a social perspective, this is much more difficult. You won't convince anyone that the Application firewall isn't doing anything more than protecting their Mac from hacking attempts from their printer. It would be a good idea to double-check this just to confirm that this isn't where your problem lies. If so, you've avoiding a major problem.

There may be technical ways to solve the firewall problem. I encountered this a little bit back when I used to use libcurl. For that kind of low-level networking, you may have to perform some unusual operations in the presence of these kinds of exotic networking configurations. Those basic operations with sockets simply may not be viable anymore. Any connection may involve significantly more effort to deal with all the different hoops that these network modifications will have put in your path. It may have worked just fine before, but it doesn't anymore.

From an OpenJDK perspective, you have to support a much wider range of networking operations. I'm sure you can't just require Private WiFi address to be fixed and you can't assume no networking filters. But I still think it's important to look at these settings to confirm or discount that they are involved with the problems you are having. That will give you important clues about where you need to look for solutions.

For the local network questions, see this technote, which has more details on exactly what is required for macOS.

The Hardened Runtime list of entitlements mentioned above is just the list of entitlements that specifically affect the restrictions applied when the Hardened Runtime is enabled.

My understanding, based on what I have read of MacOS entitlements is that, the entitlements are set on the executable and any libraries that would be loaded and used by that executable will be evaluated against the entitlements of the executable (please correct me if I misunderstand).

This is correct. For the JDK, I'd be somewhat surprised to see a lot of multicast networking (as an example) being performed by javac itself, but of course applications developed with the JDK that use the Java APIs to do networking operations from java could. Whether or not those calls succeed will be based on the properties of the executable that's running, and some of the common situations there are covered in the linked technote.

For the JDK, I'd be somewhat surprised to see a lot of multicast networking (as an example) being performed by javac itself, but of course applications developed with the JDK that use the Java APIs to do networking operations from java could.

Right - javac executable wouldn't be involved in many of these networking calls. Like you note, java would.

Thank you both of you for these extensive details about local network changes. I'm reading through the technote and other linked material. I'll experiment a bit and come back with additional questions.

These replies so far has helped me understand the relevance of entitlements, so thank you very much.

jaikiran, Thanks for asking about this here. Any changes you make to OpenJDK will benefit a lot of developers, so it’s good to get these detailed nailed down.

Lemme separate out the various topics…

That’s correct. Creating distribution-signed code for macOS is really clear about this:

You apply entitlements to a main executable.

and:

Don’t apply entitlements to library code.

The above means that, as a Java VM developer, you don’t have direct control over the entitlements applied to your code. I have a couple of hints on that front:

• Most third-party tools have some sort of ‘packaging’ process that wraps the developer’s code into an app that the user can double click in the Finder. This is a good place to encode your knowledge about what entitlements you need. For example, if your JIT needs com.apple.security.cs.allow-jit, your packager can either fails if it’s not present, or just add it unilaterally.

• On macOS it’s possible to programmatically determine what entitlements the executable of the current process was signed with. You could use that to trap at runtime, with a nice clear message, if you’re run in a process with the wrong entitlements. Lemme know if you’re interested in exploring that and I’ll dig up some examples.

That varies by API:

• Most APIs just fail, returning some sort of error. That’s certainly the case for the entitlements you mentioned.

• Some APIs trap.

• Some APIs silently do nothing.

• Or log something and then do nothing.

As others have noted, these only apply if the App Sandbox is enabled. See App Sandbox Resources for more about that technology.

There’s one subtlety related to the App Sandbox. If a sandboxed app spawns a child process, that process inherits the parent’s static sandbox. The child process is running a main executable, but that executable should not use normal sandbox entitlements. Rather, sign it with com.apple.security.app-sandbox and com.apple.security.inherit. For more on this, see:

• Embedding a command-line tool in a sandboxed app documentation

• Resolving App Sandbox Inheritance Problems here on the forums

I wanna stress that this only applies to sandbox entitlements. Other entitlements, like com.apple.security.cs.allow-jit, behave normally in this case.

One of my colleagues has already pointed you at TN3179 Understanding local network privacy, which has a bunch of info on this topic.

As to whether this entitlement is required on macOS, that’s definitely not the case. TN3179 says that, but you have to join the dots.

The “macOS 10.0+” at the top of this page is just wrong. I filed a bug and we fixed that a while back (r. 105890246). Sadly, that patch seems to have come unstuck, so I’ve filed a new bug (r. 146015779).

The hardened runtime enables a bunch of additional security checks. None of them are related to networking. Some of them are very important to a Java VM author, most notably the com.apple.security.cs.allow-jit -> com.apple.security.cs.allow-unsigned-executable-memory -> com.apple.security.cs.disable-executable-page-protection cascade. My advice on that front:

• This sequence is a trade off between increasing programmer convenience and decreasing security. com.apple.security.cs.allow-jit is the most secure, but requires extra work in your code.

• Only set one of these entitlements, because each is a superset of its predecessor.

• com.apple.security.cs.disable-executable-page-protection is rarely useful. Indeed, on Apple silicon [1] it’s the same as com.apple.security.cs.allow-unsigned-executable-memory.

If you want to investigate moving from com.apple.security.cs.allow-unsigned-executable-memory to com.apple.security.cs.allow-jit, lemme know because there are a bunch of additional resources on that topic.

[1] I’m talking about native code here. If you’re running under Rosetta and thus generating Intel code, this distinction is still relevant.

There are two ways we typically announce new constraints like this:

• Announcements on Developer > News — That has an RSS feed which is well worth monitoring.

• In WWDC sessions

But, yeah, this can be tricky. When we add constraints we usually try to do it in a binary compatible way, so that existing apps continue to run. However:

• That’s not always possible.

• In some cases the constraint only applies to updated apps, either via an App Store ingestion check or via a linked-on-or-later check.

Another pain point for Java developers is TCC, that is, the various permissions in System Settings > Privacy & Security. The specific problem is that Java developers often use a script as their main executable. I talk about this more in the TCC and Main Executables section of On File System Permissions.

I expect the above will give you plenty to think about |-: Reply here if you have any follow-up questions.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

if the executable hasn't been configured with a relevant entitlement, what happens when the executable invokes on such operation.

That varies by API:

Most APIs just fail, returning some sort of error. That’s certainly the case for the entitlements you mentioned.

Some APIs trap.

Some APIs silently do nothing.

Or log something and then do nothing.

Thank you for that detail. I was hoping there would be a central place to find some logs/hints to correlate some odd failure resulting out of a missing entitlement. Is it even technically feasible for macos to generate such logs and if yes, do you recommend filing a feedback assistance issue for it?

The hardened runtime enables a bunch of additional security checks. None of them are related to networking.

This is good to know. Thank you.

Some of them are very important to a Java VM author, most notably the com.apple.security.cs.allow-jit -> com.apple.security.cs.allow-unsigned-executable-memory -> com.apple.security.cs.disable-executable-page-protection cascade. My advice on that front:

This sequence is a trade off between increasing programmer convenience and decreasing security. com.apple.security.cs.allow-jit is the most secure, but requires extra work in your code.

Only set one of these entitlements, because each is a superset of its predecessor.

com.apple.security.cs.disable-executable-page-protection is rarely useful. Indeed, on Apple silicon [1] it’s the same as com.apple.security.cs.allow-unsigned-executable-memory.

I went back and looked at the entitlements we apply for a notarized package of the JDK. In addition to some other entitlements, it currently includes both com.apple.security.cs.allow-jit and com.apple.security.cs.allow-unsigned-executable-memory.

If you want to investigate moving from com.apple.security.cs.allow-unsigned-executable-memory to com.apple.security.cs.allow-jit, lemme know because there are a bunch of additional resources on that topic.

Yes, I would interested in those details. Actually, the JVM team of the JDK would very likely be interested in that detail (JVM isn't my area of expertise). But yes, any additional details and recommendations that you can provide on moving away from any of these entitlements is very much useful for us.

[1] I’m talking about native code here. If you’re running under Rosetta and thus generating Intel code, this distinction is still relevant.

I'm pretty sure we don't support Rosetta for Apple Silicon (I will check with others). The JDK does support x86_64 of macos and from what I see we don't use the com.apple.security.cs.disable-executable-page-protection entitlement in any of these platforms. Reading through the documentation of com.apple.security.cs.disable-executable-page-protection https://developer.apple.com/documentation/bundleresources/entitlements/com.apple.security.cs.disable-executable-page-protection, it states:

The system causes an app that attempts to directly modify sections of its own executable files on disk to forcefully exit. Use the Disable Executable Memory Protection Entitlement to enable this kind of unsafe software update.

The JDK doesn't do such modifications. So I think that's understandable why we don't use that entitlement.

com.apple.developer.networking.multicast

One of my colleagues has already pointed you at TN3179 Understanding local network privacy, which has a bunch of info on this topic.

As to whether this entitlement is required on macOS, that’s definitely not the case. TN3179 says that, but you have to join the dots.

The “macOS 10.0+” at the top of this page is just wrong. I filed a bug and we fixed that a while back (r. 105890246). Sadly, that patch seems to have come unstuck, so I’ve filed a new bug (r. 146015779).

Thank you very much. For me that was a major source of distraction and confusion while investigating some multicast failures, because I wasn't sure if I should focus on this entitlement or look elsewhere during the investigation. This confirmation helps.

The closest thing we have here is sandbox violation reports, per Discovering and diagnosing App Sandbox violations, but:

• The generation of these reports has been kinda on-and-off over the years )-:

• That’s only relevant if the app is sandboxed, and not all permissions checks are enforced by the sandbox.

Apropos that last point, I discuss the various permissions checks at the file system level is On File System Permissions. You see similar things in other subsystems, like networking. Maybe one day I’ll sit down and write an equivalent post for the network subsystem, but today is not that day.

Yeah, you’ll probably find that it works just fine with only the latter.

OK. I’d like to split that off to a separate thread, because it has a bunch of backstory and general complexity. I also want to call in some help from a colleague who tracks this issue more closely than I do.

Please start a new thread in App & System Services > Core OS topic area. Post the thread URL in your next reply here; that way I’ll be sure to see it.

Oh, speaking of multicast failures, I have various posts hung off Extra-ordinary Networking that might be of interest to you.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"