how is one supposed to dynamically load libraries in the age of SIP?

Question

This might seem flippant, but it's very serious. My use case, this is literally torturing me to death:

We have a language (Allegro Common Lisp) which has a module for making SSL connections. This module is requires OpenSSL. We don't want to supply OpenSSL with our product for obvious reasons--I'm not going to go into that here.

So, what's the problem? Well, first, the problem is that Apple decided long ago to not allow developers to build against their version of SSL (which is based on LibreSSL). So, we must depend on Macports or Homebrew for the installation of OpenSSL. Those libraries are in non-standard places and require DYLD_LIBRARY_PATH to be set so that the libraries can be dynamically loaded.

The problem is, with SIP, any environment variables that start with DYLD_ or LD_ are stripped from the environment passed to programs. That is, if I set DYLD_LIBRARY_PATH in BASH and start my product, I can't load a signed .dylib that depends on some version of the OpenSSL libraries.

To make matters worse, this works on macOS 11.6.5 (20G527) x86_64:

DYLD_LIBRARY_PATH=... ./mlisp
(sys:getenv "DYLD_LIBRARY_PATH")`

That is, it returns the value ... set on the command line.

It does not work on macOS 11.6.5 (20G527) arm64.

To dynamically load the OpenSSL libraries, we have a glue library (that adds functionality) called acliss11.dylib which depends on @rpath/libssl.1.1.dylib and @rpath/libcrypto.1.1.dylib.

For x86_64 we have a solution, but the same solution does NOT work for arm64.

How are we supposed to do this?

1.5k

Posted by

dklayer

Reply

Add a Comment

Answer 1

I just found out about the com.apple.security.cs.allow-dyld-environment-variables entitlement. This seems to be the answer to my needs, but I'm worried about notarization with this entitlement being used.

Posted by

dklayer

Add a Comment

Answer 2

So, to start, SIP is not the deciding factor here but rather the hardened runtime. The hardened runtime opts you in to a variety of security checks, including the one that nobbles DYLD_LIBRARY_PATH. You can get around this by disabling SIP but that’s a very heavy hammer. It effectively disables all security on the platform. It’s much better to opt out of specific hardened runtime security checks via entitlements.

And it seems that you’ve discovered this for yourself:

I just found out about the com.apple.security.cs.allow-dyld-environment-variables entitlement.

Right. It’s likely that you’ll also need to disable library validation (com.apple.security.cs.disable-library-validation) so that you can actually load these libraries (which are most likely unsigned or ad hoc signed but certainly aren’t signed by you, which is what library validation requires).

This seems to be the answer to my needs, but I'm worried about notarization with this entitlement being used.

I don’t think you need to worry about that. Notarisation is not App Review. See this post for details on the motivation behind the notary service.

Having said that, I have some questions. You wrote:

We have a language (Allegro Common Lisp) which has a module for making SSL connections.

Can you explain more about the relationship between this language and your product. Is your product the language itself? Or are you using the language to build your actual product?

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Posted by

eskimo

Add a Comment

Answer 3

Thanks for the confirmation, @eskimo. Appreciated.

Having said that, I have some questions. You wrote:

We have a language (Allegro Common Lisp) which has a module for making SSL connections.

Can you explain more about the relationship between this language and your product. Is your product the language itself? Or are you using the language to build your actual product?

The language is our product (google "Allegro Common Lisp" if you are curious). The language has a networking layer and one of the features is an SSL socket stream. Using OpenSSL (or LibreSSL) is how we provide SSL socket streams.

We dynamically load the OpenSSL libraries because we want to get the latest, secure libraries. On Linux, we use the system libraries in standard places and loading it is trivial. On macOS, it's been a real hardship on our users (and us). Hopefully the entitlements above will make things much easier going forward.

Posted by

dklayer

Add a Comment

Answer 4

"Allegro Common Lisp"

I knew that rang a bell. Indeed, I coulda sworn we bought that product back in the day. Some rummaging on my hard disk revealed an ancient technote, TN.231 “Macintosh Allegro Common Lisp Features”, which:

describes some known problems and provides solutions to these problems for the Macintosh Allegro Common Lisp™ package which is available from Apple Computer, Inc. You should note, however, that although Apple acquired Coral Software and is selling Macintosh Allegro Common Lisp, Apple is not currently distributing any other products which had been developed or previously sold by Coral Software.

Weirdly, your Wikepedia page makes no mention of this (-:

Hopefully the entitlements above will make things much easier going forward.

Right. That assumes that folks use your language interactively. If you allow folks to take their code and bundle it up into a standalone Mac app, that app will also have to deal with this issue. Do you support such a feature?

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Posted by

eskimo

@eskimo

Re: Coral Software. That version of Allegro was separate from Franz's Allegro CL--they shared no code. It's a pretty strange story. Honestly, I don't even remember the details very well, even though I was around at the time. It was a marketing deal and not technical, so I know why I don't remember. :)
Right. That assumes that folks use your language interactively. If you allow folks to take their code and bundle it up into a standalone Mac app, that app will also have to deal with this issue. Do you support such a feature?
Our Lisp (and all Lisps) are definitely interactive, but that's a myth that has persisted for too long. You can do scripting and building from make just like you can with C. And you can deliver standalone images with Allegro CL. The reason it's not a problem is that the actual binary image doesn't change when an app is bundled up. Only the "heap" image (a file with extension .dxl) changes. The signed binary with all the entitlements is still there. If they distribute an application with SSL, then they definitely have to worry about the issue just like we do, though.

—
dklayer

Add a Comment

Answer 5

That version of Allegro was separate from Franz's Allegro CL--they shared no code.

Oh, interesting.

If they distribute an application with SSL, then they definitely have to worry about the issue just like we do, though.

Right. My specific concern is setting DYLD_LIBRARY_PATH. Is that the approach you recommend to your customers? Or something else?

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Posted by

eskimo

Add a Comment

Answer 6

Right. My specific concern is setting DYLD_LIBRARY_PATH. Is that the approach you recommend to your customers? Or something else?

Yes, we tell them to use DYLD_LIBRARY_PATH or put the OpenSSL libraries into the application directory, if they aren't in an environment where security is a concern.

Posted by

dklayer

Add a Comment

how is one supposed to dynamically load libraries in the age of SIP?

Replies