Signed & notarized macOS app still translocated by gatekeeper

The question

What is wrong with our signing process?

Long story short

At my project we are starting to use Azure to do continuous integration and use their macOS agent pool to do macOS builds. We do signing and notarization of the build artifacts, but even though multiple tools confirm that the app is signed and notarized we still get the following dialog on the downloaded artifacts and the app gets translocated.

Long story long

As mentioned we use Azure pipelines macOS agent pool to do macOS builds. We have created a developer signing certificate (not a distribution one) via a typical process of creating the certificate signing request and submitting it to Apple. Then we packaged the certificate along with a private key into a .p12 fule and uploaded it to Azure's Secure File system.

We do not use a provisioning profile since we are targeting macOs.

During the build we use the Azure's "Install Apple certificate" task to download the .p12 file and create a keychain with it's contents. That keychain is also set as a default during this process.

The project architecutre is set to x86_64

Later during the build we layout the app as follows:

fullDeploy/
    OurApp.app/
    data/
    License Agreement.txt

where OurApp.app is laid out by XCode - we just copy it.

Then we sign things as follows

codesign \
	-s "Developer ID Application: XXXXXXXXXXXXXX" \
	--deep \
	--timestamp \
	-o runtime \
	--entitlements "yyyyyyy.entitlements" \
	"OurApp.app"

find "data/python" -name "*.so" |\
    while read f; do codesign -s "Developer ID Application: XXXXXXXXXXXXXX" "$f" --timestamp; done;

ditto -c -k --rsrc --keepParent "fullDeploy" "fullDeploy.zip"

codesign -s "Developer ID Application: XXXXXXXXXXXXXX" "fullDeploy.zip" --timestamp

xcrun altool \
    --notarize-app \
    --primary-bundle-id "zzzzzzzz" \
    --username "email@domain.com" \
    --password "password" \
    --file "fullDeploy.zip"

The zip fullDeploy.zip is then subsequently tarballed and uploaded to Azure artifact storage.

After a while the notarization successfully completes.

However when we download the tarball from Azure, untar and unzip we get the above warning.

Some additional verification:

> ls -@al fullDeploy.tar
-rw-r--r--@ 1 xxxxxxxx  staff  143296512 Jul  5 12:13 fullDeploy.tar
	com.apple.macl	       72
	com.apple.metadata:kMDItemWhereFroms	      357
	com.apple.quarantine	       58

> tar -xzvf fullDeploy.tar
...

> ls -@al fullDeploy.zip
-rw-r--r--@ 1 xxxxxxxx  staff  143260582 Jun 24 13:35 fullDeploy.zip
	com.apple.cs.CodeDirectory	      139
	com.apple.cs.CodeRequirements	      176
	com.apple.cs.CodeRequirements-1	      175
	com.apple.cs.CodeSignature	     9055
	com.apple.quarantine	       58

> codesign -dvvvv fullDeploy.zip
Executable=/Users/xxxxxxxx/Downloads/fullDeploy.zip
Identifier=ZZZZZZZZZZZZZZZZ
Format=generic
CodeDirectory v=20200 size=175 flags=0x0(none) hashes=1+2 location=embedded
Hash type=sha256 size=32
CandidateCDHash sha1=7c4b454d07d7d52bfb5dceb6df615804ee387fb4
CandidateCDHashFull sha1=7c4b454d07d7d52bfb5dceb6df615804ee387fb4
CandidateCDHash sha256=205d5a079e980d6b3d545b15328f31ca873150c7
CandidateCDHashFull sha256=205d5a079e980d6b3d545b15328f31ca873150c75a7ac574c2a37177da3f696a
Hash choices=sha1,sha256
CMSDigest=1bfe7216f426e2403c628233e1bb32c20bd09b5a0456ef16c8a4d685e6ba45ed
CMSDigestType=2
Page size=none
CDHash=205d5a079e980d6b3d545b15328f31ca873150c7
Signature size=9056
Authority=Developer ID Application: XXXXXXXXXXXXXX
Authority=Developer ID Certification Authority
Authority=Apple Root CA
Timestamp=24 Jun 2022 at 13:35:45
Info.plist=not bound
TeamIdentifier=YYYYYYYYYY
Sealed Resources=none
Internal requirements count=1 size=176

> codesign --deep-verify --deep -vvvv fullDeploy.zip
fullDeploy.zip: valid on disk
fullDeploy.zip: satisfies its Designated Requirement

> unzip fullDeploy.zip
...

> cd fullDeploy

> codesign -dvvvv OurApp.app
Executable=/Users/xxxxxxxx/Downloads/fullDeploy/OurApp.app/Contents/MacOS/OurApp
Identifier=outcompany.OurApp
Format=app bundle with Mach-O thin (x86_64)
CodeDirectory v=20500 size=622277 flags=0x10000(runtime) hashes=19435+7 location=embedded
VersionPlatform=1
VersionMin=659200
VersionSDK=787200
Hash type=sha256 size=32
CandidateCDHash sha256=9f364417c1498c12e89f2ca5e2b8de18594ba3a6
CandidateCDHashFull sha256=9f364417c1498c12e89f2ca5e2b8de18594ba3a64f6dd268b379bda90be741b5
Hash choices=sha256
CMSDigest=9f364417c1498c12e89f2ca5e2b8de18594ba3a64f6dd268b379bda90be741b5
CMSDigestType=2
Executable Segment base=0
Executable Segment limit=38354944
Executable Segment flags=0x1
Page size=4096
CDHash=9f364417c1498c12e89f2ca5e2b8de18594ba3a6
Signature size=8975
Authority=Developer ID Application: XXXXXXXXXXXXXX
Authority=Developer ID Certification Authority
Authority=Apple Root CA
Timestamp=24 Jun 2022 at 13:35:25
Info.plist entries=27
TeamIdentifier=YYYYYYYYYY
Runtime Version=12.3.0
Sealed Resources version=2 rules=13 files=8
Internal requirements count=1 size=188

> codesign --deep-verify --deep -vvvv OurApp.app
--prepared:/Users/xxxxxxxx/Downloads/fullDeploy/OurApp.app/Contents/MacOS/libtbbmalloc.2.dylib
--validated:/Users/xxxxxxxx/Downloads/fullDeploy/OurApp.app/Contents/MacOS/libtbbmalloc.2.dylib
--prepared:/Users/xxxxxxxx/Downloads/fullDeploy/OurApp.app/Contents/MacOS/libembree3.3.dylib
--validated:/Users/xxxxxxxx/Downloads/fullDeploy/OurApp.app/Contents/MacOS/libembree3.3.dylib
--prepared:/Users/xxxxxxxx/Downloads/fullDeploy/OurApp.app/Contents/MacOS/libtbb.12.dylib
--validated:/Users/xxxxxxxx/Downloads/fullDeploy/OurApp.app/Contents/MacOS/libtbb.12.dylib
--prepared:/Users/xxxxxxxx/Downloads/fullDeploy/OurApp.app/Contents/MacOS/libOpenImageDenoise.1.dylib
--validated:/Users/xxxxxxxx/Downloads/fullDeploy/OurApp.app/Contents/MacOS/libOpenImageDenoise.1.dylib
OurApp.app: valid on disk
OurApp.app: satisfies its Designated Requirement

> spctl --verbose=4 --assess --type execute OurApp.app
OurApp.app: accepted
source=Notarized Developer ID
Signed & notarized macOS app still translocated by gatekeeper
 
 
Q