EndpointSecurity AUTH_SIGNAL Handler Causes Dock UI Desync and Activity Monitor Force Quit Failure

Question

Created 2w

Replies 3

Boosts 0

Participants 3

ES_EVENT_TYPE_AUTH_SIGNAL DENY causes Dock icon to disappear and LaunchServices to lose track of the process

Platform: macOS 11.0 (Big Sur) – macOS 15 (Sequoia) Xcode: 16.4 (16F6) Language: Swift, EndpointSecurity framework Testing OS: macOS 15.5 (primary), reproduced on macOS 11.0+

[1]Description

I'm developing a System Extension using the EndpointSecurity framework for a security product. My extension subscribes to ES_EVENT_TYPE_AUTH_SIGNAL to block unauthorized signals sent to protected GUI applications (self-protection feature).

When I respond with ES_AUTH_RESULT_DENY to an AUTH_SIGNAL event targeting a GUI application, the system enters an inconsistent state:

The Dock icon disappears — loginwindow removes the app's UI via its applicationQuit event, even though the process is still running
LaunchServices loses track of the application's PID — it can no longer determine the PID from the LSASN
Activity Monitor's subsequent Force Quit attempts fail silently — no kill() syscall is issued because LaunchServices cannot resolve the PID

The issue only resolves after:

Restarting Activity Monitor (clears its internal cache), or
Relaunching the protected application (re-registers with LaunchServices)

Expected: Signal is denied, the process keeps running, Dock icon remains visible, and Activity Monitor can still force-quit the process normally. Actual: Dock icon disappears after the first blocked signal. Subsequent Force Quit attempts do nothing — no kill() syscall is issued. The process remains alive but is invisible to the system.

[2]Minimal Reproducible Code

Requires System Extension entitlement: com.apple.developer.endpoint-security.client

entitlements.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>com.apple.developer.endpoint-security.client</key>
    <true/>
</dict>
</plist>

SignalBlockingDemo.swift

import EndpointSecurity
import Foundation

var client: OpaquePointer?

es_new_client(&client) { _, message in
    guard message.pointee.event_type == ES_EVENT_TYPE_AUTH_SIGNAL else { return }

    let sig    = message.pointee.event.signal.sig
    let target = message.pointee.event.signal.target.pointee
    let targetPid = audit_token_to_pid(target.audit_token)

    // es_string_token_t does not guarantee null-termination — read via buffer
    let esToken = target.executable.pointee.path
    let targetPath: String
    let count = Int(esToken.length)
    if count > 0, let rawPtr = esToken.data {
        let buf = UnsafeBufferPointer(
            start: UnsafeRawPointer(rawPtr).assumingMemoryBound(to: UInt8.self),
            count: count)
        targetPath = String(decoding: buf, as: UTF8.self)
    } else {
        targetPath = ""
    }

    // Protect a specific GUI app — replace with your target path
    let protectedPath = "/Applications/Numbers.app/Contents/MacOS/Numbers"
    guard targetPath == protectedPath else {
        es_respond_auth_result(client!, message, ES_AUTH_RESULT_ALLOW, false)
        return
    }

    print("[ES] Blocking signal \(sig) -> pid \(targetPid) (\(targetPath))")

    // After this DENY: Dock icon disappears, LaunchServices loses the PID
    es_respond_auth_result(client!, message, ES_AUTH_RESULT_DENY, false)
}

let events: [es_event_type_t] = [ES_EVENT_TYPE_AUTH_SIGNAL]
es_subscribe(client!, events, UInt32(events.count))

print("Signal blocking active. Press Enter to stop.")
_ = readLine()

es_unsubscribe_all(client!)
es_delete_client(client!)

Build & run:

swiftc -o SignalBlockingDemo SignalBlockingDemo.swift
codesign --force --sign - --entitlements entitlements.plist SignalBlockingDemo
sudo ./SignalBlockingDemo

[3]Steps to Reproduce

Build and run SignalBlockingDemo as above (targets Numbers.app)
Launch Numbers.app — note its PID
Open Activity Monitor
In Activity Monitor, select Numbers → click Force Quit (⊗)
Observe: ES extension logs "Blocking signal 15" — signal is denied
Bug: Numbers.app Dock icon disappears, even though the process is alive
Press Enter in the demo terminal to stop signal blocking
In Activity Monitor, click Force Quit again on the Numbers process
Bug: No error shown in Activity Monitor UI, but the process is NOT terminated
In Console.app (filter: LaunchServices), observe: "Unable to determine pid of LSASN:{hi=0x1;lo=0x...}"
Confirm: No kill() syscall is issued — verify with DTrace script below

DTrace verification (trace_kill.d):

syscall::kill:entry
/execname == "Activity Monitor"/
{
    printf("%Y: Activity Monitor calling kill(%d, %d)\n", walltimestamp, arg0, arg1);
}

sudo dtrace -s trace_kill.d

During the broken Force Quit: no output (no kill() issued). After restarting Activity Monitor and retrying: kill() appears and process terminates.

[4 What We've Tried

Allowing ALL signals → Dock icon never disappears, behavior is normal
Subscribing to AUTH_SIGNAL but always returning ALLOW → no issue
Denying signals only on headless daemon processes → no issue observed
Always allowing signals from launchd (PID 1) → does not prevent the Dock issue
Always allowing SIGCHLD, SIGWINCH, SIGCONT → does not prevent the Dock issue

Hypothesis: loginwindow observes the AUTH_SIGNAL event (or a related notification) and proactively removes the Dock UI entry when a termination signal targets a GUI app — regardless of whether the signal was ultimately denied. This seems like a coordination gap between EndpointSecurity's signal interception and loginwindow/LaunchServices' app lifecycle management.

[5] Specific Questions

Is it expected that loginwindow removes the Dock UI entry for a GUI app when AUTH_SIGNAL is received, even if the signal is ultimately denied (ES_AUTH_RESULT_DENY)?
Is there a known coordination mechanism between EndpointSecurity's AUTH_SIGNAL and loginwindow / LaunchServices that we should be aware of when implementing self-protection for GUI apps?
Is there a recommended pattern or API for protecting a GUI app from termination signals via AUTH_SIGNAL without disrupting its Dock presence and LaunchServices registration?
Should we notify loginwindow or LaunchServices to re-register the application after denying a signal, and if so, how?

[6] Additional Context

The issue reproduces on macOS 11.0 through macOS 15.5
Tested with Numbers.app and other GUI applications — all reproduce the same behavior
The issue is NOT reproducible when the protected process is a headless daemon (no Dock presence)
launchd (PID 1) senders are always allowed in our policy
SIGCHLD, SIGWINCH, SIGCONT are excluded from our deny list
DTS Case ID: 19226051
Feedback ID :FB22338746

Answer 1

DTS Engineer OP

Apple

1w

Just a quick admin note: There are a bunch of folks who are happy to help with questions like this, but we rely on your thread’s topic, subtopic, and tags to find relevant questions. So, if you have Endpoint Security questions, it’s important to add the Endpoint Security tag. I’ve fixed that on this thread, but it’s something to keep in mind for the future.

For lots more info on how use forums effectively, see Quinn’s Top Ten DevForums Tips.

As to your technical question, I’m going to ask my colleague Kevin for his input on that.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Answer 2

DTS Engineer OP

Apple

1w

Part 1...

Hypothesis: loginwindow observes the AUTH_SIGNAL event (or a related notification) and proactively removes the Dock UI entry when a termination signal targets a GUI app — regardless of whether the signal was ultimately denied. This seems like a coordination gap between EndpointSecurity's signal interception and loginwindow/LaunchServices' app lifecycle management.

No, that’s what's going on. What's actually going on is that the system has two different systems that track "stuff that's running on the system":

The "low level" system is the POSIX pid-based mechanism you're familiar with. This is also the level the ES system operates at and is the "core" mechanism used to track processes. Every process running on the system "exists" within this system.
The "high level" system is built on top of #1 and is what manages the concept of "Apps". It doesn't have as clear an API layer (much of its API is private), but its origins were the Carbon Process Manager and it's what APIs like LaunchServices and NSWorkspace.runningApplications manage.

That second mechanism is built on top of the first, but is a subset of that system— so every App has a corresponding process, but not every process has a corresponding app. Similarly, the actual definition of "an app" is basically "a process being tracked by #2". Finally, note that it's tied to the current login session (not the broader system), which is why NSWorkspace and NSRunningApplication are part of AppKit. You can't ask for "the list of all running applications", just the list of "applications within the login session I'm in". And, yes, that means your ES client can't/shouldn't be interacting with this level of the system. Interactions with #2 need to be coming from an agent, not a daemon.

That's also why:

This seems like a coordination gap between EndpointSecurity's signal interception and loginwindow/LaunchServices' app lifecycle management.

...well, yes. There isn't any coordination going on between them at all, since ES is a daemon-level API and apps operate at the agent level.

Now, with that context, let me get into some specifics. First off, here:

My extension subscribes to ES_EVENT_TYPE_AUTH_SIGNAL to block unauthorized signals sent to protected GUI applications (self-protection feature).

One thing to be aware of here is that the "modern" mechanism for terminating all apps… is for the system to simply kill the app directly, as described in "Support Sudden Termination". That also leads to a broader point here, which is that it is when your ES client creates issues like this:

Bug: Numbers.app Dock icon disappears, even though the process is alive

...

In Activity Monitor, click Force Quit again on the Numbers process Bug: No error shown in Activity Monitor UI, but the process is NOT terminated

Those are NOT bugs in the system. The ES API operates at a level which allows it to alter and distort the system’s normal operation in ways that the system is incapable of predicting or compensating for, which means it's the ES client’s job to ensure the system functions properly, NOT the "normal" systems.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Answer 3

DTS Engineer OP

Apple

1w

Part 2...

Getting into the sequence here:

The Dock icon disappears — loginwindow removes the app's UI via its applicationQuit event, even though the process is still running

The system was told to "quit the app", so the first step was to remove its "visible" parts, like its dock icon.

LaunchServices loses track of the application's PID — it can no longer determine the PID from the LSASN

The system removed it from system #2, so it "disappears" from LaunchServices.

Now, there's some interesting nuance here:

When I respond with ES_AUTH_RESULT_DENY to an AUTH_SIGNAL event targeting a GUI application, the system enters an inconsistent state:

I don't think that's actually true, at least not on its own. That is, if you send "kill -9" from the terminal, I don't think any of the above would happen. What I'd expect to happen is that your app just "sits there" doing nothing and continues working normally. That's because there are two different ways for an app to "disappear". The first is that the system "notices" the app's process is "gone" and removes the app from LaunchServices (for example, this is what happens when an app crashes). You blocked the "kill" that would have kicked off that process, so what I'd expect is that "nothing" happened.

However, the other flow:

Activity Monitor's subsequent Force Quit attempts fail silently — no kill() syscall is issued because LaunchServices cannot resolve the PID

...is when LaunchServices itself initiates the quit (or force quit). In that case, the normal teardown happens, which would end in a signal, except you blocked it. What actually happened here:

The issue only resolves after: Restarting Activity Monitor (clears its internal cache), or

Activity monitor is "aware" of both the systems above, so relaunching it meant that it picked up your app as a bare process instead of "an app".

Relaunching the protected application (re-registers with LaunchServices)

I'm not sure of exactly what you're describing here, but one of two things happened:

Clicking on the app icon simply relaunched the app as a new process.
The app was still active and clicking on its icon made it "reappear" (in the dock, etc.).

If the second case is happening, then what's going on there is that the app reconnected "itself" when it was opened by the system. On top of the mechanisms above, "apps" can basically "join" themselves into #2 as part of their interaction with AppKit.

All of that leads to here:

My extension subscribes to ES_EVENT_TYPE_AUTH_SIGNAL to block unauthorized signals sent to protected GUI applications (self-protection feature).

and here:

Is there a recommended pattern or API for protecting a GUI app from termination signals via AUTH_SIGNAL without disrupting its Dock presence and LaunchServices registration?

In terms of self-protection, I think you're approaching the issue from the wrong direction. Trying to actively prevent app termination is inherently quite complicated, since:

It's very difficult to try and predict every possible path/circumstance where the system might "legitimately" terminate your app, so it's very likely that you'll end up blocking something you didn't want to block.
Preventing termination doesn't mean the app still works. The app operating environment is extremely complicated, so it's entirely possible that you can end up blocking termination while leaving the app in a state that doesn't actually "work".

Focusing on self-protection, I think the better approach is to use two (or more) components:

A faceless agent that exists to monitor that login sessions context and provide a "gateway" your ES daemon can use to interact with the login session. This component you actively protect by blocking signals, etc.
One or more foreground "apps" that are what the user actually interacts with. You'll monitor the system’s interaction with them, but won't try to actively prevent termination.

If/when #2 quits, #1 can then just relaunch it again whenever you need it.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware