Library fails to load with hardened runtime

Even though I disable library validation my application won't load library with hardened runtime.

I strongly recommend against disabling library validation.
As discussed in the docs, it makes it harder to pass Gatekeeper.

AFAICT the only good reason to disable library validation is if you app needs to load in-process plug-ins from other third-party developers.

It's important to note that library I'm trying to load is NOT signed at all.

Yeah, don’t do that either. Apple silicon requires that all code be at least ad hoc signed. So you have to fix this for the Apple silicon slice of the library, which means you might as well fix it for the Intel slice as well.

Signing them myself is not an option.

Why?

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

My intention is to let clients of my application to load plugins from other third-party developers.

Oh, cool, and in that case disabling library validation is absolutely required.

Those are well known plugins but for licensing reasons I can't distribute or sign them myself.

OK.

You mentioned that on Apple silicon those libraries have to be at least ad hoc signed.

Yes. Apple silicon will not run unsigned code at all.

However, I'm running it on Intel (macOS Catalina 10.15.7) and yet those libraries are not being loaded.

That is some other problem. Now that I’ve confirmed your circumstances, we can dig into that. More below.

how can I ad hoc sign libraries?

In Xcode, select Sign to Run Locally. If you’re signing from the command line, pass - to the -s argument in codesign. For example:

% codesign -s - test

You can also tell whether code is ad hoc signed:

% codesign -d -v test
…
CodeDirectory v=20400 size=614 flags=0x2(adhoc) …
…

As to your specific problem, it’s possible that the system is refusing to load the library’s code signature because it doesn’t use a sufficiently modern SDK. What does this print:

% vtool -show-build /path/to/a/problematic/plugin

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

I think the SDK version is fine. As I understand it has to be at least 10.9. vtool -show-build prints this for my plugin:

<name>.dylib:
Load command 8
      cmd LC_VERSION_MIN_MACOSX
  cmdsize 16
  version 10.12
      sdk 10.14

Also, I tried to ad hoc sign my plugin and the error message changed from:

Failed to load plugin '<path_to_my_dylib>': dlopen(<path_to_my_dylib>, 2): no suitable image found.  Did find:
	<path_to_my_dylib>: code signature in (<path_to_my_dylib>) not valid for use in process using Library Validation: mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed.

to:

Failed to load plugin '<path_to_my_dylib>': dlopen(<path_to_my_dylib>, 2): no suitable image found.  Did find:
	<path_to_my_dylib>: code signature in (<path_to_my_dylib>) not valid for use in process using Library Validation: mapped file has no Team ID and is not a platform binary (signed with custom identity or adhoc?)

I also verified that entitlements are properly applied by running codesign -d -v --entitlements :- "<path_to_my_app>. It gives me this output:

...
<plist version="1.0">
<dict>
    <key>com.apple.security.cs.disable-library-validation</key>
    <true/>
    <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
    <true/>
    <key>com.apple.security.cs.allow-dyld-environment-variables</key>
    <true/>
</dict>
</plist>%

Thanks, bkarasm

I think the SDK version is fine.

Did you get the same for both architectures? Use the -arch option to target a specific one.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

I'm working on one architecture. Both my library and my app are x86_64.

I'm working on one architecture.

OK, I’m officially out of ‘obvious’ ideas. Please open a DTS tech support incident and I’ll carve out some time to dig into this in detail.

Note DTS is officially closed this week for the US Thanksgiving holiday. If you open an incident I won’t get it until next week.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"