Best container for automated download & running of installer .pkg

(Skippable) Backstory:

I have an app that customers initially install by:

  1. Going to our website in a browser,
  2. downloading a (notarized) disk image,
  3. mounting it,
  4. opening the (Developer ID-signed) installer package therein, and
  5. following the prompts to completion.

Once installed, this app has a button you can click that will check for updates and possibly install them. When you click this button:

  1. Our app contacts an HTTPS API on our servers to see if there is an update
  2. If there is an update, the API responds with a download URL to the aforementioned notarized disk image, and the SHA-512 hash of the disk image
  3. Our app downloads the disk image, and verifies the SHA-512 hash of the download
  4. Our app mounts the DMG
  5. Our app uses /usr/sbin/pkgutil --check-signature (although I do hope to switch to SecStaticCodeCheckValidity() in the future) to verify that
    1. the OS thinks that the installer package is properly signed, and also that
    2. the organization unit on the leaf certificate for the code signing key used to sign the installer is exactly equal to our Apple Team ID
  6. Our app uses /usr/sbin/installer -package ... -volinfo to verify that the installer package thinks it can install onto the host
  7. Our app uses /usr/sbin/installer -package ... -target / to install the software update

For 98%+ of our users, this update process works great. For a tiny portion of users, we're seeing that hdiutil reports that it failed to attach the disk image. For the even tinier portion of users from whom we have obtained debug logs, one of the errors we've seen is "not recognized"; however, I don't know if that's the most common error, due to the small sample size.

Before I get much further, I think it's prudent to acknowledge that the above system we're using today is pushing a decade old now, and it's probably wise to verify the foundations.

So...

For a Developer-ID-signed macOS app that is distributed outside the Mac App Store using an installer package, what is the "best" container/packaging system for an automated system to obtain and consume the installer package with the goal of a user-initiated self-update? For example:

  1. Notarized disk image (and failures to attach the image need to be bug reports to Apple)
  2. Just the installer package (If I understand correctly, this is bad because it bypasses the automatic propagation (normally performed by macOS) of the DMG's notary ticket to the pkg, right?)
  3. Something else?

Secondly, what are some common developer mistakes to avoid? For example, these come to mind:

  1. When saving the DMG to disk, explicitly enable quarantine on the DMG, so that macOS runs appropriate security checks as intended (is this correct?)
  2. When running the installer package, do not use low-level tools (like cp) to copy the pkg out of the DMG, because macOS won't be able to automatically find the notary ticket when the pkg is installed (is this correct?)
  3. Anything else?

Additional context:

  • We currently support macOS 10.13+, but we will soon support only macOS 10.15+.

Thank you!

Does your app need an installer at all? That is, does the installer package install components outside of the main app in /Applications?

If so, what sort of components?

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Does your app need an installer at all? That is, does the installer package install components outside of the main app in /Applications? If so, what sort of components?

You're asking an engineer whether or not something is possible? ;)

No, we do not technically need an installer. We do currently install files outside of the main app bundle, yes — but, honestly, the only reason we're using an installer package today is because that's the way it was originally designed.

The only components remaining outside of the main application bundle are launch agents, launch daemons, and command-line executables. For pre-macOS 10.15, we also have a kernel extension. And, for supported browsers, we also have browser extensions that go in "magical" system folders, such as (for example) in /Library/Application Support/Mozilla/Extensions/. The installer package understands these details, and also correctly requires a system reboot when needed.

More components used to exist; as they have been replaced and/or upgraded over the years, more often than not, they get moved to inside the main application bundle.

I have hope that we will eventually become an app that can be installed by dragging a single app into the Applications folder, but it's a slow process. We'll make a huge step forward when we drop support for 10.14 and older, because the kext and the conditional reboot requirement will both go away.

In summary, I think my answer is actually both. Today, yes, we "require" an installer package. But, I'm also quite interested in the future where our app is a single bundle.

Yeah, so it sounds like an installer package is pretty much a requirement at the moment.

For pre-macOS 10.15, we also have a kernel extension.

So that’s only necessary, and only installed, on systems prior to macOS 10.15?

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

For pre-macOS 10.15, we also have a kernel extension.
So that’s only necessary, and only installed, on systems prior to macOS 10.15?

Correct. That's a big part of why I'm itching to drop support for 10.14 and earlier. Just waiting for the populations to drop low enough.

Yeah, KEXTs make anything hard.

In your original post you wrote:

4. Our app mounts the DMG

Are you using hdiutil for that? And, if so, I presume you hide that mount from the user?

Lemme explain a little bit about where I’m going with that…

Your primary concern seems to be the point at which the system ingests your stapled ticket. I think that happens when Gatekeeper checks an item, so when the user opens a downloaded disk image or installer package. However, I’ve never really researched that properly. I want to do some playing around, but it’d be best if I focused on the scenario you care about.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Are you using hdiutil for that? And, if so, I presume you hide that mount from the user?

Why yes, I am using hdiutil to mount the image. Specifically, I'm using:

(translating Obj-C to pseudocode just for brevity)

diskImagePath = "<path to dmg>"

mountPath = "/Volumes/" + UUID().uuidString

/usr/bin/hdiutil attach "$diskImagePath" -mountpoint "$mountPath" -nobrowse -noautoopen

And, later, to unmount, I'm using:

/usr/bin/hdiutil detach "$mountPath" -force
Your primary concern seems to be the point at which the system ingests your stapled ticket.

That's correct.

More precisely, I'm actually not completely sure what to be scared of in this ecosystem. I definitely want normal security measures to work as intended. This is why I phrased the original question (albeit somewhat confusingly) as "what is the best practice here", rather than "how do I fix this rare problem".


For example, if you were to (hypothetically) come back and say, "in this scenario, a notarized DMG is appropriate because <reasons>, and you should also do <thing> to avoid <problem>", then what that tells me is:

  1. The general approach I'm using today is here to stay, at least for now.
  2. Thus, it's worth my time to add more automated metrics around my hdiutil attach failure (see "Skippable Backstory" in the original post), and file a (useful) bug report.
  3. And, because I'm using the technologies in a recommended way, I'm more likely to get a useful answer back, which is good for me.

On the other hand, if you were to say, "in this scenario, a notarized DMG is bad because <reasons>, and you should use <thing> instead", then what that tells me is:

  1. There's a risk that my undesirable use of this technology is contributing to my hdiutil attach problem.
  2. Thus, it's not worth my time to fix the hdiutil attach problem, and instead, I should reengineer my fancy self-update system to operate in a way that is recommended by Apple
  3. And, applying the recommendations may completely sidestep my problems, which is good for me.

Yeah, I’m not on solid ground here, and I don’t have time today to run the tests required to compact it.

My impression is that ticket ingestion is done as part of Gatekeeper’s trust evaluation, so it won’t happen when you mount the disk image with hdiutil. However, I haven’t tested that. However however (-: you can test this yourself.

Imagine a disk image containing an executable. When you notarise the disk image, the notary service generates a ticket that includes that executable. If the disk image is quarantined, macOS will only run the executable if it can find a notarised ticket. A valid approach for this is:

  1. You staple the ticket to the disk image.

  2. The user downloads the disk image, which quarantines it.

  3. They then disable networking.

  4. And then they double click the disk image in the Finder. The system then ingests the stapled ticket.

  5. The user runs the tool from the disk image; this works, even though networking is off, because the system has ingested the ticket in step 4.

Imagine repeating these steps but, in step 4, you mount the disk image with hdiutil. Does that ingest the ticket or not? Well, step 5 provides the answer to that.

You could run a similar test with an installer package, although you’d have to manually quarantine the tool because the one laid down by the installer won’t be quarantined.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Best container for automated download &amp; running of installer .pkg
 
 
Q