Why are we doing this nonsense?
We want to be able to run builds in a sandbox such that they can only see the paths they are intended to depend on, to improve reproducibility. With builds with a very large number of dependencies, there's a very large number of paths added to the sandbox, and it breaks things inside libsandbox.
Either it hits some sandbox length limit (sandbox-exec: pattern serialization length 66460 exceeds maximum (65535), Nix issue #4119, worked around: Nix PR 12570), or it hits an assert (this report; also Nix issue #2311).
The other options for sandboxing on macOS are not viable; we acknowledge sandbox-exec and sandbox_init_with_parameters are deprecated; App Sandbox is inapplicable because we aren't an app. Our use case is closer to a browser, and all the browsers use libsandbox internally.
We could possibly use SystemExtension or a particularly diabolical use of Virtualization.framework, but the former API requires notarization which is close to a no-go for our use case as open source software: it is nearly impossible to develop the software on one's own computer, and it would require us to ship a binary blob (and have the build processes to produce one in infrastructure completely dissimilar to what we use today); it also requires a bunch of engineering time.
Today, we can pretend that code signing/notarization doesn't exist and that we are writing an old-school Unix daemon, because we are one.
The latter is absolutely diabolical and hard to implement.
See this saga about the bug we are facing: Nix issue #4119, Nix issue #2311, etc.
What is going wrong
I can't attach the file fail.sb as it is too large (you can view the failing test case at Lix's gerrit, CL 2870) and run this:
$ sandbox-exec -D _GLOBAL_TMP_DIR=/tmp -f fail.sb /bin/sh Assertion failed: (diff <= INSTR_JUMP_NE_MAX_LENGTH), function push_jne_instr, file serialize.c, line 240. zsh: abort sandbox-exec -D _GLOBAL_TMP_DIR=/tmp -f fail.sb /bin/sh
Or a stacktrace:
Credits
Full credits to Jade Lovelace (Lix) for writing the above text and filing a bug. This is submitted under FB16964888
What you’re doing here is unsupported. While there is a public API for setting up a custom sandbox, albeit a deprecated one, the sandbox programming language (SBPL) has never been documented for third-party use, and thus is not considered API. That’s why everything within <sandbox.h> is deprecated: Without a supported SBPL, the API is effectively useless.
Yep. And that raises the subtle question of what “supported” means. I’m using it in the very specific sense that DTS doesn’t support this. From our perspective it’s all considered to be an implementation detail.
Now, plenty of third-party code relies on implementation details, sometimes deliberately but more often accidentally [1]. This is an ongoing binary compatibility concern for our platform engineers. They regularly have to make hard choices between improving the implementation and breaking third-party products. DTS tries not to make this situation worse, and hence our focus on supported APIs.
As to what you should do about this issue.
To start, filing a bug makes sense. It’s possible that the sandbox team will change the implementation to help you out.
Beyond that, you wrote:
You mean an Endpoint Security system extension, right?
Because there are a few other types, and only the ES one seems relevant.
True, but there’s definitely a trade-off to be made here. App Sandbox can be used by non-app code, and you have to balance the risks of that against the risks of relying on SBPL. So, yeah, I definitely think you should explore that option in more depth.
Although I’m not sure it’ll actually help. App Sandbox relies on the same underlying sandbox implementation, so it might well run into the same limits. However, I think it’s worth the time creating a quick prototype, just to be sure.
Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"
[1] And hence Hyrum’s law.