Codesigning helper executables

Question

Created Jul ’20

Replies 6

Boosts 1

Views 2.6k

Participants 2

I'm following the advice from this earlier discussion.

I put my helper executables in /Contents/Helpers. I added a "Run Script" build phase to sign the binaries in the helper directory. It contains these lines:

Code Block shcodesign --verbose --timestamp --options=runtime -s "${CODE_SIGN_IDENTITY}" -v "${HELPER_DIR}/rustybates"codesign --verbose --timestamp --options=runtime -s "${CODE_SIGN_IDENTITY}" -v "${HELPER_DIR}/${IMAGE_MAGICK}/bin/magick"

This script completes successfully, but the final codesign step for the application does not. It fails with:

Code Block CodeSign /Users/jeff/Library/Developer/Xcode/DerivedData/Easy_Bates-hbfethteogzgwmghlgwdflcwddow/Build/Products/Development/Easy\ Bates.app (in target 'Easy Bates' from project 'Easy Bates')		cd /Users/jeff/gitrepos/code/code/macbates		export CODESIGN_ALLOCATE=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/codesign_allocateSigning Identity:		 "-"		/usr/bin/codesign --force --sign - -o runtime --entitlements /Users/jeff/Library/Developer/Xcode/DerivedData/Easy_Bates-hbfethteogzgwmghlgwdflcwddow/Build/Intermediates.noindex/Easy\ Bates.build/Development/Easy\ Bates.build/Easy\ Bates.app.xcent --timestamp=none /Users/jeff/Library/Developer/Xcode/DerivedData/Easy_Bates-hbfethteogzgwmghlgwdflcwddow/Build/Products/Development/Easy\ Bates.app/Users/jeff/Library/Developer/Xcode/DerivedData/Easy_Bates-hbfethteogzgwmghlgwdflcwddow/Build/Products/Development/Easy Bates.app: bundle format unrecognized, invalid, or unsuitableIn subcomponent: /Users/jeff/Library/Developer/Xcode/DerivedData/Easy_Bates-hbfethteogzgwmghlgwdflcwddow/Build/Products/Development/Easy Bates.app/Contents/Helpers/ImageMagick-7.0.10Command CodeSign failed with a nonzero exit code

Here's what my helpers directory looks like in my .app:

Code Block Contents/HelpersContents/Helpers/rustybatesContents/Helpers/ImageMagick-7.0.10Contents/Helpers/ImageMagick-7.0.10/.DS_StoreContents/Helpers/ImageMagick-7.0.10/binContents/Helpers/ImageMagick-7.0.10/bin/magickContents/Helpers/ImageMagick-7.0.10/etcContents/Helpers/ImageMagick-7.0.10/etc/ImageMagick-7Contents/Helpers/ImageMagick-7.0.10/etc/ImageMagick-7/quantization-table.xmlContents/Helpers/ImageMagick-7.0.10/etc/ImageMagick-7/colors.xmlContents/Helpers/ImageMagick-7.0.10/etc/ImageMagick-7/type-apple.xmlContents/Helpers/ImageMagick-7.0.10/etc/ImageMagick-7/type-windows.xmlContents/Helpers/ImageMagick-7.0.10/etc/ImageMagick-7/type.xmlContents/Helpers/ImageMagick-7.0.10/etc/ImageMagick-7/policy.xmlContents/Helpers/ImageMagick-7.0.10/etc/ImageMagick-7/delegates.xmlContents/Helpers/ImageMagick-7.0.10/etc/ImageMagick-7/thresholds.xmlContents/Helpers/ImageMagick-7.0.10/etc/ImageMagick-7/mime.xmlContents/Helpers/ImageMagick-7.0.10/etc/ImageMagick-7/type-ghostscript.xmlContents/Helpers/ImageMagick-7.0.10/etc/ImageMagick-7/log.xmlContents/Helpers/ImageMagick-7.0.10/etc/ImageMagick-7/type-dejavu.xmlContents/Helpers/ImageMagick-7.0.10/etc/ImageMagick-7/type-urw-base35.xmlContents/Helpers/ImageMagick-7.0.10/libContents/Helpers/ImageMagick-7.0.10/lib/libxml2.2.dylibContents/Helpers/ImageMagick-7.0.10/lib/libMagickCore-7.Q8.dylibContents/Helpers/ImageMagick-7.0.10/lib/libMagickCore-7.Q8.laContents/Helpers/ImageMagick-7.0.10/lib/libMagickWand-7.Q8.laContents/Helpers/ImageMagick-7.0.10/lib/libMagickWand-7.Q8.7.dylibContents/Helpers/ImageMagick-7.0.10/lib/liblcms2.2.dylibContents/Helpers/ImageMagick-7.0.10/lib/libpng16.16.dylibContents/Helpers/ImageMagick-7.0.10/lib/libomp.dylibContents/Helpers/ImageMagick-7.0.10/lib/libfreetype.6.dylibContents/Helpers/ImageMagick-7.0.10/lib/libMagickCore-7.Q8.7.dylibContents/Helpers/ImageMagick-7.0.10/lib/libMagickCore-7.Q8.aContents/Helpers/ImageMagick-7.0.10/lib/libMagickWand-7.Q8.aContents/Helpers/ImageMagick-7.0.10/lib/libMagickWand-7.Q8.dylibContents/Helpers/ImageMagick-7.0.10/lib/libomp.aContents/Helpers/ImageMagick-7.0.10/lib/libjpeg.9.dylibContents/Helpers/ImageMagick-7.0.10/lib/libtiff.5.dylibContents/Helpers/ImageMagick-7.0.10/lib/liblzma.5.dylibContents/Helpers/ImageMagick-7.0.10/shareContents/Helpers/ImageMagick-7.0.10/share/.DS_StoreContents/Helpers/ImageMagick-7.0.10/share/ImageMagick-7Contents/Helpers/ImageMagick-7.0.10/share/ImageMagick-7/francais.xmlContents/Helpers/ImageMagick-7.0.10/share/ImageMagick-7/locale.xmlContents/Helpers/ImageMagick-7.0.10/share/ImageMagick-7/english.xmlContents/Helpers/ImageMagick-7.0.10/share/docContents/Helpers/ImageMagick-7.0.10/share/doc/.DS_StoreContents/Helpers/ImageMagick-7.0.10/share/doc/ImageMagick-7Contents/Helpers/ImageMagick-7.0.10/share/doc/ImageMagick-7/LICENSEContents/Helpers/ImageMagick-7.0.10/share/doc/ImageMagick-7/ChangeLogContents/Helpers/ImageMagick-7.0.10/share/doc/ImageMagick-7/NEWS.txt

How do I make codesign happy?

Boost

Answer 1

DTS Engineer OP

Apple

Jul ’20

Presumably rustybates is a standalone executable. If so, you can put it in Contents/Helpers/ but I’d advise Contents/MacOS/.

As far as ImageMagick-7.0.10 is concerned, that’s going to be a serious challenge. Putting it in Contents/Helpers/ won’t work because that directory is expected to be populated by either standalone executables or bundled executables, and this folder structure is not a bundle.

The best way to fix this problem, IMO, is to put the ‘foreign’ structure in Contents/Resources/ and then move all the signable stuff (executables and libraries) to the correct location within the app’s bundle and replace them with symlinks. You can find a concrete example of this in this post.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@apple.com"

1

Answer 2

coder44 OP

Jul ’20

Thank you Quinn for the quick reply. This is very helpful.

I now have a plan; I'll reply when I get it working.

0

Answer 3

coder44 OP

Jul ’20

I put rustybates in MacOS, signed it, and it works fine and Apple Notary accepts it.

I also signed magick, libMagickCore-7.Q8.7.dylib, and libMagickWand-7.Q8.7.dylib in place in their current directories under Contents/Resources, and Apple Notary notarized my application. I exported it, and it works! Wahoo!

However if I try to build and run the application locally, magick fails with the error below.

So I'm glad that it works, but I'm a little worried for two reasons:

My local development process includes commenting some lines in my build script to avoid signing magick. Then uncommenting those lines to archive and notarize the app. :-/
Notary notarizes my application today, but will it notarize it tomorrow? I haven't found any Apple docs that prohibit putting executables and libraries in the /Resources directory, but Quinn mentioned in the linked posts that it will cause problems.

Code Block % ./magick dyld: Library not loaded: @executable_path/../lib/libMagickCore-7.Q8.7.dylib	Referenced from: /Users/jeff/Library/Developer/Xcode/DerivedData/Easy_Bates-hbfethteogzgwmghlgwdflcwddow/Build/Products/Development/Easy Bates.app/Contents/Resources/ImageMagick-7.0.10/bin/./magick	Reason: no suitable image found.	Did find:				/Users/jeff/Library/Developer/Xcode/DerivedData/Easy_Bates-hbfethteogzgwmghlgwdflcwddow/Build/Products/Development/Easy Bates.app/Contents/Resources/ImageMagick-7.0.10/bin/./../lib/libMagickCore-7.Q8.7.dylib: code signature in (/Users/jeff/Library/Developer/Xcode/DerivedData/Easy_Bates-hbfethteogzgwmghlgwdflcwddow/Build/Products/Development/Easy Bates.app/Contents/Resources/ImageMagick-7.0.10/bin/./../lib/libMagickCore-7.Q8.7.dylib) not valid for use in process using Library Validation: mapped file has no Team ID and is not a platform binary (signed with custom identity or adhoc?)				/Users/jeff/Library/Developer/Xcode/DerivedData/Easy_Bates-hbfethteogzgwmghlgwdflcwddow/Build/Products/Development/Easy Bates.app/Contents/Resources/ImageMagick-7.0.10/bin/./../lib/libMagickCore-7.Q8.7.dylib: stat() failed with errno=1				/Users/jeff/Library/Developer/Xcode/DerivedData/Easy_Bates-hbfethteogzgwmghlgwdflcwddow/Build/Products/Development/Easy Bates.app/Contents/Resources/ImageMagick-7.0.10/bin/../lib/libMagickCore-7.Q8.7.dylib: code signature in (/Users/jeff/Library/Developer/Xcode/DerivedData/Easy_Bates-hbfethteogzgwmghlgwdflcwddow/Build/Products/Development/Easy Bates.app/Contents/Resources/ImageMagick-7.0.10/bin/../lib/libMagickCore-7.Q8.7.dylib) not valid for use in process using Library Validation: mapped file has no Team ID and is not a platform binary (signed with custom identity or adhoc?)				/Users/jeff/Library/Developer/Xcode/DerivedData/Easy_Bates-hbfethteogzgwmghlgwdflcwddow/Build/Products/Development/Easy Bates.app/Contents/Resources/ImageMagick-7.0.10/bin/../lib/libMagickCore-7.Q8.7.dylib: stat() failed with errno=1				file system relative paths not allowed in hardened programszsh: abort			./magick

0

Answer 4

DTS Engineer OP

Apple

Jul ’20

Our consistent recommendation is that you put nested code into nested code sites. See the Nested Code section of Technote 2206 macOS Code Signing In Depth.

If you put nested code into Contents/Resources/ then weird things start to happen. Imagine an app with a single nested helper tool. If you do the nesting correctly then you’ll see this:

Code Block  % grep '<key>MacOS/Helper</key>' -A 8 Test655881.app/Contents/_CodeSignature/CodeResources		<key>MacOS/Helper</key>		<dict>			…			<key>requirement</key>			<string>identifier Helper and anchor apple generic and certificate leaf[subject.CN] = "Apple Development: Quinn Quinn (7XFU7D52S4)" and certificate 1[field.1.2.840.113635.100.6.2.1] /* exists */</string>		</dict>

The reference from the app to the helper is via the helper’s designated requirement. Contrast it to this:

Code Block  % grep '<key>Resources/Helper</key>' -A 6 Test655881.app/Contents/_CodeSignature/CodeResources… first hit elided …--		<key>Resources/Helper</key>		<dict>			<key>hash2</key>			<data>			BfAki7kUY0KJmTybHSvU5STm1DA7+ug7wxajaNHxEYE=			</data>		</dict>

Note In this example I ignored the first hit because that’s the legacy resource seal (within files). What matters these days is the modern resource seal (within files2).

Here the reference is the hash of the file itself. This prevents various code signing features from working properly. For example, if you use lipo to remove an architecture from the helper tool, that works in the Contents/MacOS/ case (because both architectures in the tool meet the requirement encoded in the app’s code signature) whereas it’ll break in the Contents/Resources/ case (because you’ve changed the hash of the file).

Note To actually test this you have to use Xcode 12.0b3, which supports 64-bit Arm, because, with the demise of 32-bit Intel, it’s hard to build multi-architecture macOS binaries with Xcode 11.

Incorrect nesting can also cause tools problems. The most obvious of these is an incompatibility with --deep. Now, you shouldn’t be using --deep anyway (see --deep Considered Harmful) but, if you use it against incorrectly nested code, it will do completely the wrong thing.

However, this isn’t the only tools problem you’ll encounter. Over the years I’ve helped numerous folks with problems caused by incorrect nesting. A typical symptom is that things work during development but go wrong when you export an archive from the Organizer. In many cases the resulting app works on your development machine but fails when you try to use it elsewhere (uploading it to the store, submitting it for notarisation, when testing the notarised product, and so on).

The tools have got a lot better about this in recent releases but I don’t think you should rely on that. TN2206 is pretty clear about this:

While strict compliance with these rules may not affect your app today, anything that doesn't meet these requirements … may be rejected by code signing verification (and the Mac App Store validator, in the case of Mac App Store apps) at any point in the future

Oh, one last thing. All of the above has been about putting code in a data nesting site. There’s another flavour of incorrect nesting, namely putting data in a code nesting site. This is much more problematic. Don’t do it.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@apple.com"

0

Answer 5

coder44 OP

Jul ’20

Thank you for the thorough explanation. Technote 2206 contains exactly the details I've been looking for for over a week. Table 3 in particular is super helpful.

0

Answer 6

coder44 OP

Jul ’20

Thanks again Quinn. I finally got all the files in the right places with symlinks left in Contents/Resources.

Also, I published the script I wrote to do it. I hope others find it useful.

1