EndpointSecurity AUTH_SIGNAL Handler Causes Dock UI Desync and Activity Monitor Force Quit Failure

Question

Created 2d

Replies 0

Boosts 0

Participants 1

ES_EVENT_TYPE_AUTH_SIGNAL DENY causes Dock icon to disappear and LaunchServices to lose track of the process

Platform: macOS 11.0 (Big Sur) – macOS 15 (Sequoia) Xcode: 16.4 (16F6) Language: Swift, EndpointSecurity framework Testing OS: macOS 15.5 (primary), reproduced on macOS 11.0+

[1]Description

I'm developing a System Extension using the EndpointSecurity framework for a security product. My extension subscribes to ES_EVENT_TYPE_AUTH_SIGNAL to block unauthorized signals sent to protected GUI applications (self-protection feature).

When I respond with ES_AUTH_RESULT_DENY to an AUTH_SIGNAL event targeting a GUI application, the system enters an inconsistent state:

The Dock icon disappears — loginwindow removes the app's UI via its applicationQuit event, even though the process is still running
LaunchServices loses track of the application's PID — it can no longer determine the PID from the LSASN
Activity Monitor's subsequent Force Quit attempts fail silently — no kill() syscall is issued because LaunchServices cannot resolve the PID

The issue only resolves after:

Restarting Activity Monitor (clears its internal cache), or
Relaunching the protected application (re-registers with LaunchServices)

Expected: Signal is denied, the process keeps running, Dock icon remains visible, and Activity Monitor can still force-quit the process normally. Actual: Dock icon disappears after the first blocked signal. Subsequent Force Quit attempts do nothing — no kill() syscall is issued. The process remains alive but is invisible to the system.

[2]Minimal Reproducible Code

Requires System Extension entitlement: com.apple.developer.endpoint-security.client

entitlements.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>com.apple.developer.endpoint-security.client</key>
    <true/>
</dict>
</plist>

SignalBlockingDemo.swift

import EndpointSecurity
import Foundation

var client: OpaquePointer?

es_new_client(&client) { _, message in
    guard message.pointee.event_type == ES_EVENT_TYPE_AUTH_SIGNAL else { return }

    let sig    = message.pointee.event.signal.sig
    let target = message.pointee.event.signal.target.pointee
    let targetPid = audit_token_to_pid(target.audit_token)

    // es_string_token_t does not guarantee null-termination — read via buffer
    let esToken = target.executable.pointee.path
    let targetPath: String
    let count = Int(esToken.length)
    if count > 0, let rawPtr = esToken.data {
        let buf = UnsafeBufferPointer(
            start: UnsafeRawPointer(rawPtr).assumingMemoryBound(to: UInt8.self),
            count: count)
        targetPath = String(decoding: buf, as: UTF8.self)
    } else {
        targetPath = ""
    }

    // Protect a specific GUI app — replace with your target path
    let protectedPath = "/Applications/Numbers.app/Contents/MacOS/Numbers"
    guard targetPath == protectedPath else {
        es_respond_auth_result(client!, message, ES_AUTH_RESULT_ALLOW, false)
        return
    }

    print("[ES] Blocking signal \(sig) -> pid \(targetPid) (\(targetPath))")

    // After this DENY: Dock icon disappears, LaunchServices loses the PID
    es_respond_auth_result(client!, message, ES_AUTH_RESULT_DENY, false)
}

let events: [es_event_type_t] = [ES_EVENT_TYPE_AUTH_SIGNAL]
es_subscribe(client!, events, UInt32(events.count))

print("Signal blocking active. Press Enter to stop.")
_ = readLine()

es_unsubscribe_all(client!)
es_delete_client(client!)

Build & run:

swiftc -o SignalBlockingDemo SignalBlockingDemo.swift
codesign --force --sign - --entitlements entitlements.plist SignalBlockingDemo
sudo ./SignalBlockingDemo

[3]Steps to Reproduce

Build and run SignalBlockingDemo as above (targets Numbers.app)
Launch Numbers.app — note its PID
Open Activity Monitor
In Activity Monitor, select Numbers → click Force Quit (⊗)
Observe: ES extension logs "Blocking signal 15" — signal is denied
Bug: Numbers.app Dock icon disappears, even though the process is alive
Press Enter in the demo terminal to stop signal blocking
In Activity Monitor, click Force Quit again on the Numbers process
Bug: No error shown in Activity Monitor UI, but the process is NOT terminated
In Console.app (filter: LaunchServices), observe: "Unable to determine pid of LSASN:{hi=0x1;lo=0x...}"
Confirm: No kill() syscall is issued — verify with DTrace script below

DTrace verification (trace_kill.d):

syscall::kill:entry
/execname == "Activity Monitor"/
{
    printf("%Y: Activity Monitor calling kill(%d, %d)\n", walltimestamp, arg0, arg1);
}

sudo dtrace -s trace_kill.d

During the broken Force Quit: no output (no kill() issued). After restarting Activity Monitor and retrying: kill() appears and process terminates.

[4 What We've Tried

Allowing ALL signals → Dock icon never disappears, behavior is normal
Subscribing to AUTH_SIGNAL but always returning ALLOW → no issue
Denying signals only on headless daemon processes → no issue observed
Always allowing signals from launchd (PID 1) → does not prevent the Dock issue
Always allowing SIGCHLD, SIGWINCH, SIGCONT → does not prevent the Dock issue

Hypothesis: loginwindow observes the AUTH_SIGNAL event (or a related notification) and proactively removes the Dock UI entry when a termination signal targets a GUI app — regardless of whether the signal was ultimately denied. This seems like a coordination gap between EndpointSecurity's signal interception and loginwindow/LaunchServices' app lifecycle management.

[5] Specific Questions

Is it expected that loginwindow removes the Dock UI entry for a GUI app when AUTH_SIGNAL is received, even if the signal is ultimately denied (ES_AUTH_RESULT_DENY)?
Is there a known coordination mechanism between EndpointSecurity's AUTH_SIGNAL and loginwindow / LaunchServices that we should be aware of when implementing self-protection for GUI apps?
Is there a recommended pattern or API for protecting a GUI app from termination signals via AUTH_SIGNAL without disrupting its Dock presence and LaunchServices registration?
Should we notify loginwindow or LaunchServices to re-register the application after denying a signal, and if so, how?

[6] Additional Context

The issue reproduces on macOS 11.0 through macOS 15.5
Tested with Numbers.app and other GUI applications — all reproduce the same behavior
The issue is NOT reproducible when the protected process is a headless daemon (no Dock presence)
launchd (PID 1) senders are always allowed in our policy
SIGCHLD, SIGWINCH, SIGCONT are excluded from our deny list
DTS Case ID: 19226051
Feedback ID :FB22338746

Boost