Prevent malicious software detection on signed library

There are two possible causes here:

• Library validation

• Notarisation

You have to make sure that the program loading your library has library validation disabled. This breaks down as follows:

• If the program has the hardened runtime enabled, it must be signed with the com.apple.security.cs.disable-library-validation entitlement.

• If not, it must not include the library validation flag.

You can determine this with the following command:

% codesign -d -vvv --entitlements - /path/to/third/party/program

The hardened runtime option is rendered as runtime. The library validation flag is rendered as library.

However, I suspect the real issue here is notarisation. You didn’t mentioned that, so I suspect that you’re not notarising your library. You must do that if you intend to distribute it widely.

For more about notarisation, see the links in Notarisation Resources.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Hi @eskimo, thank you very much for your quick reply!

The third-party program does seem to have library validation disabled:

...
CodeDirectory v=20500 size=477 flags=0x10000(runtime) hashes=4+7 location=embedded
...
[Dict]
	[Key] com.apple.security.cs.allow-jit
	[Value]
		[Bool] true
	[Key] com.apple.security.cs.disable-library-validation
	[Value]
		[Bool] true
	[Key] com.apple.security.cs.allow-unsigned-executable-memory
	[Value]
		[Bool] true
	[Key] com.apple.security.cs.disable-executable-page-protection
	[Value]
		[Bool] true

I thought about notarising, but as far as I understand you can only notarise Apps or Zip files. Am I missing something? This is the error I get when I try to notarise:

xxxxx@MacBook-Pro-JS Downloads % xcrun notarytool submit xxxxx.so --wait --apple-id "xxxxxxx" --password "xxxxxxx" --team-id "xxxxxxxx"
Conducting pre-submission checks for xxxxx.so and initiating connection to the Apple notary service...
Error: xxxxx.so must be a zip archive (.zip), flat installer package (.pkg), or UDIF disk image (.dmg)
Usage: notarytool <subcommand>
  See 'notarytool --help' for more information.

Thank you very much in advance for your help!

The third-party program does seem to have library validation disabled:

I recommend that you discuss this with the tool’s author. The only good reason to disable library validation is if the tool is loading plug-ins from other third-party developers, which seems unlikely.

I see a lot of folks disable library validation because they think it’ll make their life easier but, in reality, it makes things harder. Specifically, the combination of disabling library validation and dangling load commands is the number one cause of mysterious Gatekeeper rejections. See Resolving Trusted Execution Problems for the details.

So, unless this tool needs to disable library validation, I recommend that you re-enable it. That’s better for security and it helps with Gatekeeper.

I thought about notarising, but as far as I understand you can only notarise Apps or Zip files. Am I missing something?

Yes. As a general rule you should not notarise each code item separately. Rather, build all your code items into a complete product and notarise that. Mac products are typically distribution as zip archives, disk images, or installer packages, and the notary service will accept all of those.

For general info on this subject, see Packaging Mac Software for Distribution.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"