Monteray can’t be opened because Apple cannot check it for malicious software?

Hi,

I'm responsible for generating the macOS installer for Apache NetBeans. Up until today I've not had any issues, the DMG created has never had any issues being used on others machines.

The process we have signs and notarizes the app, as expected. However when downloading this from the internet at attempting to install the app, users are reporting that it cannot be opened because it cannot be checked by Apple for malicious software.

Heres a link to the app: https://dist.apache.org/repos/dist/dev/netbeans/netbeans-installers/13/Apache-NetBeans-13-bin-macosx.dmg

If I try to open this myself(after downloading), I can see 2 messages in Console:

1: assessment denied for Apache-NetBeans-13-bin-macosx.dmg com.apple.message.domain: com.apple.security.assessment.outcome2 com.apple.message.signature2: bundle:UNBUNDLED com.apple.message.signature: denied:no usable signature com.apple.message.signature3: Apache-NetBeans-13-bin-macosx.dmg com.apple.message.signature5: UNKNOWN com.apple.message.signature4: 3 SenderMachUUID: 8702454A-423C-33A4-BDAA-656186E59614

2: assessment denied for Apache NetBeans 13.pkg com.apple.message.domain: com.apple.security.assessment.outcome2 com.apple.message.signature2: bundle:UNBUNDLED com.apple.message.signature3: Apache NetBeans 13.pkg com.apple.message.signature5: UNKNOWN com.apple.message.signature4: 2 com.apple.message.signature: denied:Unnotarized Developer ID SenderMachUUID: 8702454A-423C-33A4-BDAA-656186E59614

I'm struggling to work out how to further diagnose this, to allow us to release this application? The notarization of this app comes back as "Package Approved", so I don't get that reference in those messages.

Any help or advice on how to continue with this?

Are all certificates current?

Hi,

Yes:

john@Johns-MacBook-Pro 13 % pkgutil --check-signature Apache\ NetBeans\ 13.pkg Package "Apache NetBeans 13.pkg": Status: signed by a developer certificate issued by Apple for distribution Signed with a trusted timestamp on: 2022-02-25 21:16:15 +0000 Certificate Chain: 1. Developer ID Installer: The Apache Software Foundation (2GLGAFWEQD) Expires: 2025-05-29 13:28:11 +0000 SHA256 Fingerprint: B4 DF 1A CA F9 1F AF 40 E8 75 EC 6B 70 FE FB E7 28 9C A6 68 22 17 5A 8F 40 17 04 B8 8D 04 23 3E ------------------------------------------------------------------------ 2. Developer ID Certification Authority Expires: 2027-02-01 22:12:15 +0000 SHA256 Fingerprint: 7A FC 9D 01 A6 2F 03 A2 DE 96 37 93 6D 4A FE 68 09 0D 2D E1 8D 03 F2 9C 88 CF B0 B1 BA 63 58 7F ------------------------------------------------------------------------ 3. Apple Root CA Expires: 2035-02-09 21:40:36 +0000 SHA256 Fingerprint: B0 B1 73 0E CB C7 FF 45 05 14 2C 49 F1 29 5E 6E DA 6B CA ED 7E 2C 68 C5 BE 91 B5 A1 10 01 F0 24

About the only change this time to all the other times, I've done this, is that I've upgraded to Monterey

Did you notarise the app? Or the installer package? Because the installer package doesn’t seem to be notarised )-:

% cp "/Volumes/Apache NetBeans 13/Apache NetBeans 13.pkg" .
% xcrun stapler staple -v "Apache NetBeans 13.pkg" 
…
CloudKit query for Apache NetBeans 13.pkg (1/eb70c1e2dad8795cb48dbf7e33a62a6fa99d38c9) failed due to "Record not found".
Could not find base64 encoded ticket in response for 1/eb70c1e2dad8795cb48dbf7e33a62a6fa99d38c9
The staple and validate action failed! Error 65.

Also, did you use Apple tools to create this installer package? Or a third-party tool?

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

I reran the staple on the DMG and got

OK, that makes sense. Sorry about the bum steer.

We have a custom installer creation in Apache NetBeans

Define “custom”:

  • Is it “custom” as in “runs a whole bunch of Apple tools that eventually creates the .pkg”?

  • Or is it “custom” as in “creates the .pkg file directly”?

And if it’s the latter, do you do the same thing for the disk image?

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Hi @eskimo,

It is built using pkgbuild and productbuild from an ant build script.

https://github.com/apache/netbeans/search?q=pkgbuild

https://github.com/apache/netbeans/search?q=productbuild

 <argvalue="pkgbuild --root ${builddir}/app --scripts ${scriptsdir} --identifier org.netbeans.ide.nbide.${nb.id} --install-location ${install.dir} ${sign_iden_name_arg} ${basedir}/../../build/nbide-${nb.id}.pkg"/>
 <arg value="pkgbuild --root '${module.builddir}/netbeans' --scripts '${scriptsdir}' --identifier org.netbeans.ide.${module.name}.${nb.id} ${sign_iden_name_arg} --install-location '/${app.name}/Contents/Resources/NetBeans' '${basedir}/build/${module.name}-${nb.id}.pkg'"/>
<arg value="pkgbuild --nopayload --scripts '${target.scriptsdir}' --identifier org.netbeans.postinstallscripts ${sign_iden_name_arg} '${basedir}/build/postinstallscripts.pkg'"/>
<arg value="productbuild --distribution '${pkg.file}/distribution.xml' --package-path '${pkg.file}/packages' --resources '${pkg.file}/resources/' ${sign_iden_name_arg} '${pkg.file}/inst_package/${pkg.name}.pkg'"/>

@JohnMcDonnell correct me if I am wrong, but I don't think that the main executable is code signed Apache Netbeans 13.app/Contents/MacOS/netbeans.

Could that be part of the problem?

--Christian

It is built using pkgbuild and productbuild from an ant build script.

Something very weird is going on here. Consider this:

  1. I downloaded your disk image from the link above.

  2. I mounted the disk image.

  3. I copied Apache NetBeans 13.pkg to my hard disk.

  4. I removed quarantine from that.

  5. I dropped it on Pacifist.

  6. Pacifist spits out the following error:

    The file “cpp-13.pkg” couldn’t be opened because there is no such file.
    
    [Stop Extraction] [Continue]
    

    That’s not a good sign.

  7. I clicked Continue.

  8. I used Pacifist to extract Apache NetBeans 13.app.

  9. I checked its code signature:

    % codesign -d -vvv "Apache NetBeans 13.app"
    Apache NetBeans 13.app: code object is not signed at all
    

So, there are two issues here:

  • What’s the cpp-13.pkg error about?

  • How did your installer package pass notarisation when the main app isn’t signed?

I can’t help with the first one. I have a basic knowledge of the installer but I’m not an expert on setting up complex installer packages. And likewise for the Specifying [allow-external-scripts='true'] is deprecated. message. So, I’m going to focus on the second issue.


Consider this:

% codesign -d -vvv Apache-NetBeans-13-bin-macosx.dmg 
Apache-NetBeans-13-bin-macosx.dmg: code object is not signed at all
% codesign -d -vvv "/Volumes/Apache NetBeans 13/Apache NetBeans 13.pkg"
/Volumes/Apache NetBeans 13/Apache NetBeans 13.pkg: code object is not signed at all

This is super weird. Neither your disk image nor your installer are actually signed. And that brings me back to something you wrote earlier:

I reran the staple on the DMG and got

How does that work if the disk image isn’t signed? To investigate this I ran stapler for myself:

% stapler staple validate -vvv Apache-NetBeans-13-bin-macosx.dmg         
…
Signing information is {
    cdhashes =     (
        {length = 20, bytes = 0xe8c895384c939910dc83eb774fe9377c943a97ef}
    );
    …
    identifier = ADHOC;
    …
}
…
Downloaded ticket has been stored at file:///var/folders/ts/89wlmlw971x8k8ds6y_48kn80000gp/T/e41e37f0-d80f-4428-aa1f-a0d748f74403.ticket.
…

There’s a couple of things to note here:

  • identifier = ADHOC indicates that the notarisation infrastructure has decided to use an ad hoc signature for this disk image.

  • e8c895384c939910dc83eb774fe9377c943a97ef is the code directory hash (cdhash) it’s consed up. A cdhash uniquely identifies a code item and its critical to how the notarisation infrastructure works. Indeed, a notarised ticket is simply a list of cdhashes that’s been signed by Apple.

  • I then dumped the ticket:

    % NotarizationTicketDump /var/folders/ts/89wlmlw971x8k8ds6y_48kn80000gp/T/e41e37f0-d80f-4428-aa1f-a0d748f74403.ticket
    e8c895384c939910dc83eb774fe9377c943a97ef
    

Note NotarizationTicketDump is a tool I wrote myself to dump the cdhashes in a ticket. I can’t share that tool but you, as the person who did the notarisation, can get the same information from the notarisation log. More on this below.

So, the only cdhash in the ticket is the one for the ad hoc signature of your disk image. In short, neither your installer package nor any of your code are notarised O-: That explains why it won’t pass Gatekeeper.

At this point my best guess is that something is wonky about your installer package construction and that has caused the notary service to not look inside it. To investigate further I recommend that you get the notarisation log to see if it’s flagged any issues.

How you do this depends on how you notarise:

  • If you’re using altool, run the --notarization-info subcommand, grab the LogFileURL from its output, and then fetch that.

  • If you’re using notarytool, run the log subcommand.

IMPORTANT altool has been deprecated for the purposes of notarisation. Switch to notarytool; it’s better, stronger, and faster. For the details, see WWDC 2021 Session 10261 Faster and simpler notarization for Mac apps.

Look in the contents and issues entries in the log. The contents entry should include items for every piece of code in your installer package, and the installer package itself. I suspect that it’ll be empty, or may have a single entry for the disk image, and that issues will explain why.

Also, I’d appreciate you posting the log here so that I can see this for myself. Use the text attachment feature (click the paperclip icon and then choose Add File) to avoid clogging up the timeline. You may need to change the extension from .json to .txt.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Hi,

So I removed the cpp package reference, and that has opened the floodgates to alot more errors from the notarization process:

I'm trying to sign Apache NetBeans 13.app but I am now getting an error saying that the certificate is not in Keychain:

     [echo] Signing App: codesign --verbose -s 'Developer ID Installer: The Apache Software Foundation (2GLGAFWEQD)' /Users/john/Apache/WIP/distpreparation/netbeans/installer/nbbuild/installer/mac/newbuild/netBeans/nbide/build/app/Apache NetBeans 13.app
     [exec] error: The specified item could not be found in the keychain.

However I can see it there:

I was going to see if we'd get away with the -deep option with codesign, I know you've a post saying its harmful1, however I just want to see if for at least this release we'd get away with it(I doubt it but got to try :))

So the first log you posted indicates that the notary service wasn’t able to unpack your installer package and thus didn’t notarise anything within that. The fact that this is a warning seems kinda bogus to me. I’d appreciate you filing a bug about that. Make sure to include the request UUID (17cde505-4b91-4593-b979-c741d6016a47) so that the notary service folks can track things down based on that.

Please post your bug number, just for the record.


So I removed the cpp package reference, and that has opened the floodgates to alot more errors from the notarization process:

Oh yeah!

I am now getting an error saying that the certificate is not in keychain

If you run the same codesign command directly from Terminal, does that work?

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Hi @eskimo

So I got there in the end I believe... I'm attaching the latest notarization log, which shows that it was Accepted:

I also uploaded and downloaded the dmg from a remote location and I didn't get any error about the malicious software, I've asked @oyarzunc to see if he also has no issues.

However, there are warnings in the logs, how can find out more info about these warnings? So that we don't run into the same issue later on, as in the warnings masking a major issue?

I raised this ticket earlier: FB9943557

Thank you.

So I got there in the end

Yay!

there are warnings in the logs

I don’t have a good explanation for those warnings, but I agree that they are worrisome.

Regardless of what else you do here, you should file another bug (sorry) about the poor diagnostic here. The message Unable to notarize … is not helpful.

My best guess is that there’s yet more weirdness going on with your installer packages. Consider the paths in the warnings:

Apache-NetBeans-13-bin-macosx.dmg/
  Apache NetBeans 13.pkg/
    javaee-13.pkg
      Contents/
        Payload/
          Applications/
            NetBeans/
              Apache NetBeans 13.app
    javase-13.pkg
      Contents/
        Payload/
          Applications/
            NetBeans/
              Apache NetBeans 13.app
    php-13.pkg
      Contents/
        Payload/
          Applications/
            NetBeans/
              Apache NetBeans 13.app
    webcommon-13.pkg
      Contents/
        Payload/
          Applications/
            NetBeans/
              Apache NetBeans 13.app

They’re all for the same app! And if you look further up in the log, in ticketContents, you’ll see yet a successful entry for that app:

{
  "path": "Apache-NetBeans-13-bin-macosx.dmg/Apache NetBeans 13.pkg/nbide-13.pkg Contents/Payload/Applications/NetBeans/Apache NetBeans 13.app",
  "digestAlgorithm": "SHA-256",
  "cdhash": "46860a4c3c08ffb2c9cd48cf6c99fdc469648b43",
  "arch": "x86_64"
},

The difference is in the installer package name. The successful one is in nbide-13.pkg and the unsuccessful ones are in javaee-13.pkg, javase-13.pkg, php-13.pkg, and webcommon-13.pkg. Does that ring any bells with you? Do those other installers perhaps create the Applications/NetBeans/Apache NetBeans 13.app directory but not populate it?

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Monteray can’t be opened because Apple cannot check it for malicious software?
 
 
Q