dyld not searching in /usr/local/lib (cmd line tools 2397)

I should add, that the issue is very easy to reproduce (at least on my system). Install a library in /usr/local/lib. Write a test program that links that library. Build it (no errors). Run it, and it fails, with that "Expected in: <no uuid> unknown". In my case I used an example program from the GEOS library, cc -I/usr/local/include capi_indexed_predicate.c -o capi_indexed_predicate -L/usr/local/lib -lgeos_c

There are others seeing this, but so far it seems a small group. https://stackoverflow.com/questions/77137284/dynamic-linker-fails-to-find-symbol-at-runtime-after-updating-to-xcode-15

YOLO, so I just updated to 13.6 and nothing has changed, still cannot resolve libraries in /usr/local/lib.

YOLO, so I installed 13.6, but it didn't change anything here, dyloading @rpath deps from /usr/local is still borked.

pwramsey3 wrote:

the issue is very easy to reproduce

So lemme see if I understand your steps here:

1. Create a dynamic library called libfoo.dylib with an install name of @rpath/libfoo.dylib.

2. Install that in /usr/local/lib.

3. Create a command-line tool that calls a symbol exported by that dynamic library.

Is that it?

Because I wouldn’t expect that to work without an extra step, namely, adding an rpath entry to the tool using -rpath /usr/local/lib.

What am I missing here?

ps Just to set the scene, I’m coming at this from the perspective described in Dynamic Library Identification.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

I can confirm this behavior, and that it started causing troubles with the recent XCode update. Ran into it with pact-go, which installed a libpact_ffi.dylib under /usr/local/lib with install_name being just libpact_ffi.dylib. Long-winded post here - https://github.com/pact-foundation/pact-go/issues/345 According to dyld man page, this should work, because /usr/local/lib is in the default fallback search paths - and it used to work ok. Have you opened a ticket with Apple already @pwramsey3 ?

@eskimo In our case there's a minor difference, we don't use rpath, but overall the process is similar:

1. Create a dynamic library called libfoo.dylib with an install name of libfoo.dylib (no rpath)
2. Install that in /usr/local/lib.
3. Create a binary that calls a symbol exported by that dynamic library.

Interestingly, linker works fine, in our case its go compiler - it can find this library without issues even if it's install path is just the file name.

This means linker can search /usr/local/lib by default. However when running the newly built binary, dyld fails to locate it.

This used to work. My understanding is that dyld used to search /usr/local/lib by default. According to man dyld description of DYLD_FALLBACK_LIBRARY_PATH:

If a dylib is not found at its install  path, dyld uses this as a list of directories to search for the
dylib.  By default, it is set to /usr/local/lib:/usr/lib.

If this change is intentional, man page should probably be updated, or if not, it seems like a bug?

Long-worded thread in pact-go github: https://github.com/pact-foundation/pact-go/issues/345

\

Just a reminder, if you reply in the comments I’m not notified. See tip 5 in Quinn’s Top Ten DevForums Tips.

My understanding is that dyld used to search /usr/local/lib by default.

That’s understandable, based on what’s in the man page.

I originally suspected that this might be related to the hardened runtime, which radically limits the amount of searching done by the dynamic linker. However, I tried this with the hardened runtime disabled and it doesn’t work their either.

I’m inclined to agree with pwramsey3 that this is a SIP thing. Looking at dynamic linker source, it seems to be controlled by the allowClassicFallbackPaths value which is in turn controlled by AMFI. AMFI stands for Apple mobile file integrity per the amfid man page, and it’s the subsystem responsible for enforcing various restrictions like this.

If this change is intentional, man page should probably be updated … ?

Agree. Please file a bug against it, and post your bug number here, just for the record.

Still, best practice here is to identify your library with either an rpath-relative path or an absolute path, and that’s the approach I recommend going forward.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Is there any way to change allowClassicFallbackPaths? Grepping the source I see it shows up via getAMFI() but not where getAMFI() is defined. Seems like flipping that bit would be the cleanest way for people like me to get back the old behaviour for a while, while we change over all our library builds to use full paths on install. For the moment, I have turned off SIP and set DYLD_FALLBACK_LIBRARY_PATH in /etc/zprofile, but that makes me nervous as a full-time solution.

Feedback was submitted on FB13226043

Thanks.

Is there any way to change allowClassicFallbackPaths?

Once you disable SIP there are all sorts of obscure options to opt in and out of various feature. However, I don’t maintain expertise in those, so I can’t offer any guidance.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Feedback was submitted on FB13226043

Oops, I've also submitted mine (didn't see @pwramsey3 submitting theirs), mine is FB13233867

Has there been any update on this issue? Is this going to be fixed or is this something developers should get used to?

I forgot to mention @eskimo in my previous post - could you comment on whether this is going to be fixed or whether this is an intented feature?

Updated to Sonoma recently, the issue is still there, but the error message is now at least descriptive.

dyld[89082]: Library not loaded: @rpath/libgeos_c.1.dylib
  Referenced from: <A23D3060-FF70-3AEA-B4AA-F5E92F7D0C13> /Users/pramsey/tmp/capi_indexed_predicate
  Reason: no LC_RPATH's found
zsh: abort      ./capi_indexed_predicate

I guess this means that Apple is definitely not going to change the behaviour back to the original, UNIX'y behaviour.

So, there are two parts to this:

• How does it behave?

• Why isn’t the man page accurate?

The man page disconnect is definitely a bug; the man page should accurately reflect the behaviour of the system. We’re using FB13233867 to track that fix.

The situation with the actual behaviour is less clear. There are competing requirements here, between traditional Unix-y behaviour and macOS’s security goals. I’m not the person who gets to balance those requirements, so I can’t offer any predictions as to how that’ll pan out.

However, if you asked me I’d vote for the behaviour we have. As a developer you have a number of good ways to set up your library search path, something that I discuss in detail in Dynamic Library Identification. If you follow the paths (hey hey) described by those posts, you shouldn’t run into any problems.

original, UNIX'y behaviour

If you’re old enough, “original, UNIX'y behaviour” mean statically linking everything (-:

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"