dyld not searching in /usr/local/lib (cmd line tools 2397)

I should add, that the issue is very easy to reproduce (at least on my system). Install a library in /usr/local/lib. Write a test program that links that library. Build it (no errors). Run it, and it fails, with that "Expected in: <no uuid> unknown". In my case I used an example program from the GEOS library, cc -I/usr/local/include capi_indexed_predicate.c -o capi_indexed_predicate -L/usr/local/lib -lgeos_c

There are others seeing this, but so far it seems a small group. https://stackoverflow.com/questions/77137284/dynamic-linker-fails-to-find-symbol-at-runtime-after-updating-to-xcode-15

YOLO, so I just updated to 13.6 and nothing has changed, still cannot resolve libraries in /usr/local/lib.

YOLO, so I installed 13.6, but it didn't change anything here, dyloading @rpath deps from /usr/local is still borked.

pwramsey3 wrote:

the issue is very easy to reproduce

So lemme see if I understand your steps here:

1. Create a dynamic library called libfoo.dylib with an install name of @rpath/libfoo.dylib.

2. Install that in /usr/local/lib.

3. Create a command-line tool that calls a symbol exported by that dynamic library.

Is that it?

Because I wouldn’t expect that to work without an extra step, namely, adding an rpath entry to the tool using -rpath /usr/local/lib.

What am I missing here?

ps Just to set the scene, I’m coming at this from the perspective described in Dynamic Library Identification.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

I can confirm this behavior, and that it started causing troubles with the recent XCode update. Ran into it with pact-go, which installed a libpact_ffi.dylib under /usr/local/lib with install_name being just libpact_ffi.dylib. Long-winded post here - https://github.com/pact-foundation/pact-go/issues/345 According to dyld man page, this should work, because /usr/local/lib is in the default fallback search paths - and it used to work ok. Have you opened a ticket with Apple already @pwramsey3 ?

@eskimo In our case there's a minor difference, we don't use rpath, but overall the process is similar:

1. Create a dynamic library called libfoo.dylib with an install name of libfoo.dylib (no rpath)
2. Install that in /usr/local/lib.
3. Create a binary that calls a symbol exported by that dynamic library.

Interestingly, linker works fine, in our case its go compiler - it can find this library without issues even if it's install path is just the file name.

This means linker can search /usr/local/lib by default. However when running the newly built binary, dyld fails to locate it.

This used to work. My understanding is that dyld used to search /usr/local/lib by default. According to man dyld description of DYLD_FALLBACK_LIBRARY_PATH:

If a dylib is not found at its install  path, dyld uses this as a list of directories to search for the
dylib.  By default, it is set to /usr/local/lib:/usr/lib.

If this change is intentional, man page should probably be updated, or if not, it seems like a bug?

Long-worded thread in pact-go github: https://github.com/pact-foundation/pact-go/issues/345

\

Just a reminder, if you reply in the comments I’m not notified. See tip 5 in Quinn’s Top Ten DevForums Tips.

My understanding is that dyld used to search /usr/local/lib by default.

That’s understandable, based on what’s in the man page.

I originally suspected that this might be related to the hardened runtime, which radically limits the amount of searching done by the dynamic linker. However, I tried this with the hardened runtime disabled and it doesn’t work their either.

I’m inclined to agree with pwramsey3 that this is a SIP thing. Looking at dynamic linker source, it seems to be controlled by the allowClassicFallbackPaths value which is in turn controlled by AMFI. AMFI stands for Apple mobile file integrity per the amfid man page, and it’s the subsystem responsible for enforcing various restrictions like this.

If this change is intentional, man page should probably be updated … ?

Agree. Please file a bug against it, and post your bug number here, just for the record.

Still, best practice here is to identify your library with either an rpath-relative path or an absolute path, and that’s the approach I recommend going forward.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Is there any way to change allowClassicFallbackPaths? Grepping the source I see it shows up via getAMFI() but not where getAMFI() is defined. Seems like flipping that bit would be the cleanest way for people like me to get back the old behaviour for a while, while we change over all our library builds to use full paths on install. For the moment, I have turned off SIP and set DYLD_FALLBACK_LIBRARY_PATH in /etc/zprofile, but that makes me nervous as a full-time solution.

Feedback was submitted on FB13226043

Thanks.

Is there any way to change allowClassicFallbackPaths?

Once you disable SIP there are all sorts of obscure options to opt in and out of various feature. However, I don’t maintain expertise in those, so I can’t offer any guidance.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Feedback was submitted on FB13226043

Oops, I've also submitted mine (didn't see @pwramsey3 submitting theirs), mine is FB13233867

Has there been any update on this issue? Is this going to be fixed or is this something developers should get used to?

I forgot to mention @eskimo in my previous post - could you comment on whether this is going to be fixed or whether this is an intented feature?

Updated to Sonoma recently, the issue is still there, but the error message is now at least descriptive.

dyld[89082]: Library not loaded: @rpath/libgeos_c.1.dylib
  Referenced from: <A23D3060-FF70-3AEA-B4AA-F5E92F7D0C13> /Users/pramsey/tmp/capi_indexed_predicate
  Reason: no LC_RPATH's found
zsh: abort      ./capi_indexed_predicate

I guess this means that Apple is definitely not going to change the behaviour back to the original, UNIX'y behaviour.

So, there are two parts to this:

• How does it behave?

• Why isn’t the man page accurate?

The man page disconnect is definitely a bug; the man page should accurately reflect the behaviour of the system. We’re using FB13233867 to track that fix.

The situation with the actual behaviour is less clear. There are competing requirements here, between traditional Unix-y behaviour and macOS’s security goals. I’m not the person who gets to balance those requirements, so I can’t offer any predictions as to how that’ll pan out.

However, if you asked me I’d vote for the behaviour we have. As a developer you have a number of good ways to set up your library search path, something that I discuss in detail in Dynamic Library Identification. If you follow the paths (hey hey) described by those posts, you shouldn’t run into any problems.

original, UNIX'y behaviour

If you’re old enough, “original, UNIX'y behaviour” mean statically linking everything (-:

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Wanted to post a possible solution as I've had a similar issue specifically with golang itself linking shared libraries.

There is the potential issue that you are running with go run main.go. Verify that directly executing the executable from your shell or OS environment doesn't solve the issues as it helped me. This looks like a interoperability issue between golang and the underlying environment

The difference between running an executable directly with ./main and using go run main.go lies in how the Go program is built and executed:

1. Running with ./main:

Precompiled Binary: When you run ./main, you are executing a precompiled binary. This binary has been built using go build or a similar command, which compiles the Go source code into an executable file. RPATH Handling: The RPATH (runtime library search path) is embedded in the binary during the build process. This means any dynamic library paths specified during the build are included in the binary, allowing it to locate shared libraries at runtime. Environment Independence: Since the binary is precompiled, it does not depend on the Go environment or source files being present. It can be run on any compatible system where the necessary libraries are available. Running with go run main.go: On-the-fly Compilation: go run compiles the Go source code on-the-fly and then executes it. This is convenient for quick testing and development but is not suitable for production deployment. No Embedded RPATH: Since go run compiles the code temporarily, it does not embed RPATH information in the same way a precompiled binary does. This can lead to issues if the program relies on shared libraries that are not in standard locations or specified in the environment's library path. Dependency on Go Environment: go run requires the Go environment to be set up correctly, including having the source files and any dependencies available. In your case, the issue with "no LC_RPATH found" is likely due to the fact that go run does not embed the necessary RPATH information, whereas the precompiled binary does. This is why running the executable directly solves the problem, as it can correctly locate the required shared libraries.