Launching MacOS app via Url Scheme

Sorry for the late reply to this, it was outside of the areas Quinn and I normally monitor and we both missed it. In any case, some additional questions were passed over to me through a DTS ticket, so I'm going to answer your questions above and then the other questions that were passed to me.

(Splitting posts for length)

First of all, as a general comment here:

I am looking for alternatives and was previously under the impression that if we were to move to a DMG then that would provide the user a better user experience for moving it.

So, one thing to understand here is that the pattern of using disk images for software distribution is primarily a holdover of what is now "ancient" history, not technical merit. It started when software was being distributed on floppy disk as the easiest way to "exactly" duplicate the contents of the original floppy disk. Originally, these images couldn't even be mounted, so their only purpose was to be written back out to physical media. That pattern was then carried over to CDs and DVDs for largely similar reasons— that is, software was being distributed as a collection of files (one or more apps, documentation, possibly an installer...) and it wasn't clear what the requirements of that particular "source" were*, so the simplest and most reliable way to duplicate "any" given source was to create an image.

*For example, some software would only install from a volume configuration that "looked" like the installer source looked.

If it seems like none of that is all that relevant to modern software distribution... you’re right, it isn't. The system doesn't have any particular "preference" for one mechanism over another and there isn't really any technical reason to prefer one format over the other.

That leads to here:

However, now that I am getting around to looking into it, I am seeing some implied statements that this is not the case and that the quarantine bit will just be moved from the DMG to the app.

In terms of security protections, any security mechanism that we apply to one format MUST also be applied to other formats, otherwise an attacker would simply switch to the other format to avoid that protection.

Now there are differences in the details of how we behave with different formats; however, those differences are generally caused by the specifics of that particular scenario. For example, we generally don't translocate apps that are launched off of read-only media, but that's because the app is inherently protected from modification, so translocation is unnecessary.

Moving to specifics:

1. Does a DMG allow the app to be launched via custom protocol without prior launch or movement?

2. With a notarized app, will the custom protocol work on a subsequent launch, even without prior movement?

Yes, both of these generally work for fully notarized apps with or without disk images.

Moving to the other questions I received:

At this point, I am looking at whether I need to be worried about running from a randomized path.

So, the first thing to understand is that a check like this:

To ensure user compliance, we added a check to make sure that the lock has been removed by looking at the quarantine attribute.

...isn't reliable. The quarantine flag is used to store metadata the system uses to make these decisions (which is why stripping it changes behavior), but its existence doesn't guarantee any specific behavior.

Does distributing via DMG automatically handle this? It appears the answer is yes from what I am seeing, even though the quarantine attribute doesn’t fully go away

...but that's simply because it encourages the user to copy the app out of the disk image, and that copy changed the quarantine state. That's exactly the same thing you're doing by asking the user to move their application. If the user is running the app directly out of the DiskImage, then you're in exactly the same situation you were already in without the DiskImage.

As a broader point, I'm not sure the focus on "preflighting" (meaning, trying to validate that your update will someday work) is all that helpful. That's because:

• It's entirely possible that the user will never need to update your app.

• It's entirely possible that the system state will change by the time you do need to update, invalidating the entire preflight.

• It gets in the way of "smart" users. For example, it's possible the reason it's in a read-only location is because it's being shared from a central server, which is also where someone is making sure it's up to date.

• The preflight heuristic itself is both complicated and error-prone.

The bottom line here is that you're doing all of this work to determine whether or not you should present a somewhat confusing "Please move this app" dialog which may or not be necessary, all to avoid the possibility that you might need to present a fairly simple "please find my app" dialog. Even worse, having done all that preflight work, you might still be unable to update at which point the user is still stuck without a solution.

If you're going to perform some kind of preflight process, then my suggestion would be that you present it as an ignorable notification or alert, then just let the user run the app wherever they like.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Part 2...

That leads to here:

and how can I make sure my app will be allowed to update when I need it too.

The first thing I'd do here is shift the way you're thinking about this issue from "how do I remove/bypass quarantine" to "how do I make sure my update process always works". You've focussed on quarantine because it's a fairly common edge case; however, the problem with focussing on "solving" it is:

1. The system’s behavior is opaque and undocumented. This means, as you're already experiencing, it's hard to make this work "right" at all, and even if you "succeed", it's very likely that something will change in the future, leaving you back with the same problem.

2. Quarantine is only one failure source of many. For example, what if the actual problem is that the app is being run from a read-only location like a network share?

That second point is what makes this problem so tricky. The "full" range of valid macOS configuration is so large that it's effectively impossible to fully test or predict. This means you end up trapped in a "loop" endlessly trying to tweak or patch your way around edge cases, with frustrated users complaining the entire time.

My recommendation for getting out of this is to split the problem into two different parts:

(1) Build a good "always" experience.

This is the code path that will work no matter what. In practice, this means asking your user "where" your app and, possibly, where they want the new version to be. Strictly speaking, that last step might not be necessary, but if your app cannot write to its current location, then the only option is going to be writing "somewhere else".

It may seem like this is avoiding the actual problem, but what this really does is ensure that you don't leave the user confused and stuck when "something" goes wrong. That leads to #2:

(2) Use heuristics and design choices to avoid #1 when possible.

I don't have any single answer to what that might look like. DiskImages do mitigate some of the quarantine issues:

On the issue of "finding" your app:

Running from a randomized path breaks secondary utilities from updating the app since they can’t know the local location on disk,

I don't have time to test this at the moment, but have you tried having your updater call NSWorkspace.urlForApplication(toOpen:) or NSWorkspace.urlsForApplications(withBundleIdentifier:)? I would have expected either of those to return your app’s true location, though it's possible that the relationship between processes is concealing the true path.

Are there known stability or reliability issues with running an app while it remains translocated?

No, not really.

Any issues if the quarantine attribute is not fully removed?

As I've alluded to above, most of the issues I see are actually apps trying to "fight" with the quarantine system, instead of accepting it as one of many possible failure sources which they can handle "gracefully".

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware