Need Inputs on Which Extension to Use

Hi all,

I have a working macOS (Intel) system extension app that currently uses only a Content Filter (NEFilterDataProvider). I need to capture/log HTTP and HTTPS traffic in plain text, and I understand NETransparentProxyProvider is the right extension type for that.

For HTTPS I will need TLS inspection / a MITM proxy — I’m new to that and unsure how complex it will be.

For DNS data (in plain text), can I use the same extension, or do I need a separate extension type such as NEPacketTunnelProvider, NEFilterPacketProvider, or NEDNSProxyProvider?

Current architecture:

Two Xcode targets: MainApp and a SystemExtension target.

The SystemExtension target contains multiple network extension types.

MainApp ↔ SystemExtension communicate via a bidirectional NSXPC connection.

I can already enable two extensions (Content Filter and TransparentProxy). With the NETransparentProxy, I still need to implement HTTPS capture.

Questions I’d appreciate help with:

Can NETransparentProxy capture the DNS fields I need (dns_hostname, dns_query_type, dns_response_code, dns_answer_number, etc.), or do I need an additional extension type to capture DNS in plain text?

If a separate extension is required, is it possible or problematic to include that extension type (Packet Tunnel / DNS Proxy / etc.) in the same SystemExtension Xcode target as the TransparentProxy?

Any recommended resources or guidance on TLS inspection / MITM proxy setup for capturing HTTPS logs?

There are multiple DNS transport types — am I correct that capturing DNS over UDP (port 53) is not necessarily sufficient? Which DNS types should I plan to handle?

I’ve read that TransparentProxy and other extension types (e.g., Packet Tunnel) cannot coexist in the same Xcode target. Is that true?

Best approach for delivering logs from multiple extensions to the main app (is it feasible)? Or what’s the best way to capture logs so an external/independent process (or C/C++ daemon) can consume them?

Required data to capture (not limited to):

All HTTP/HTTPS (request, body, URL, response, etc.)

DNS fields: dns_hostname, dns_query_type, dns_response_code, dns_answer_number, and other DNS data — all in plain text.

I’ve read various resources but remain unclear which extension(s) to use and whether multiple extension types can be combined in one Xcode target. Please ask if you need more details.

Thank you.

Answered by DTS Engineer in 865548022
I’m new to that and unsure how complex it will be.

TLS inspection isn’t too hard, but be aware that it won’t work for all clients. Some system services and apps do their own certificate checks — certificate pinning, public key pinning, and so on — and those are incompatible with TLS inspection.

For DNS data … can I use the same extension

You probably can, but IMO it’s better to use a DNS proxy because that’s specifically designed for this task.

You will need to disable opportunistic secure DNS; see this thread.

is it possible … to include that … in the same SystemExtension … ?

You should be able to include your DNS proxy provider in the same sysex. And the fact that you already have this working with content filter and transparent proxy providers suggests that this’ll actually work in practice (-:

am I correct that capturing DNS over UDP (port 53) is not necessarily sufficient?

Yes. You most definitely need to support DNS-over-TCP on port 53.

You also need to disable opportunistic secure DNS, as I mentioned above.

Having said that, this won’t necessarily work for all clients. Some third-party apps have their own DNS resolvers [1] and in such cases there’s no guarantee that you disable security, avoid DNS-over-HTTP, and so on.

I’ve read that TransparentProxy and other extension types … cannot coexist in the same Xcode target.

I’m not aware of any such limitations for sysexen.

It’s certainly true each appex can only contain a single provider type, but that’s not relevant here.

Best approach for delivering logs

IMO you should log using the system log. I explain why, and how, in Your Friend the System Log.

Using the system log complicates the process capturing logs in your main app, but the benefits outweigh the drawbacks IMO. Specifically, it means your log entries are coordinated with NE’s log entries, which is super useful when debugging hard problems.

Of course, others disagree, and you’re free to write or acquire your own logging library.

Finally, if you’re just getting started, I recommend that you review the links in Network Extension Resources.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] We specifically recommend against that, see TN3151, but folks do it anyway.

I’m new to that and unsure how complex it will be.

TLS inspection isn’t too hard, but be aware that it won’t work for all clients. Some system services and apps do their own certificate checks — certificate pinning, public key pinning, and so on — and those are incompatible with TLS inspection.

For DNS data … can I use the same extension

You probably can, but IMO it’s better to use a DNS proxy because that’s specifically designed for this task.

You will need to disable opportunistic secure DNS; see this thread.

is it possible … to include that … in the same SystemExtension … ?

You should be able to include your DNS proxy provider in the same sysex. And the fact that you already have this working with content filter and transparent proxy providers suggests that this’ll actually work in practice (-:

am I correct that capturing DNS over UDP (port 53) is not necessarily sufficient?

Yes. You most definitely need to support DNS-over-TCP on port 53.

You also need to disable opportunistic secure DNS, as I mentioned above.

Having said that, this won’t necessarily work for all clients. Some third-party apps have their own DNS resolvers [1] and in such cases there’s no guarantee that you disable security, avoid DNS-over-HTTP, and so on.

I’ve read that TransparentProxy and other extension types … cannot coexist in the same Xcode target.

I’m not aware of any such limitations for sysexen.

It’s certainly true each appex can only contain a single provider type, but that’s not relevant here.

Best approach for delivering logs

IMO you should log using the system log. I explain why, and how, in Your Friend the System Log.

Using the system log complicates the process capturing logs in your main app, but the benefits outweigh the drawbacks IMO. Specifically, it means your log entries are coordinated with NE’s log entries, which is super useful when debugging hard problems.

Of course, others disagree, and you’re free to write or acquire your own logging library.

Finally, if you’re just getting started, I recommend that you review the links in Network Extension Resources.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] We specifically recommend against that, see TN3151, but folks do it anyway.

Hi @DTS Engineer , Thank you very much for your responses, really helpful.

For the past two weeks, I've been experimenting with a macOS System Extension application that manages three different Network Extension providers (NEFilterDataProvider, NETransparentProxyProvider, and NEDNSProxyProvider) housed within a single systemext bundle (Xcode target2), all managed and activated by a single main application(Xcode target1).

My goal is to reliably capture comprehensive network activity logs from all three extensions simultaneously, particularly under high-traffic conditions. I've run into a few architectural questions regarding prioritization, concurrency, and logging reliability.

Part 1: Extension Activation Order and Execution Predictability

When multiple Network Extensions are active, how does the system process traffic, and is there a guaranteed order of execution?

Q1: Given that all three extensions are active, is there a predictable order in which the network traffic will hit the providers? For example, should I assume the NEFilterDataProvider processes the socket/flow before the NETransparentProxyProvider or NEDNSProxyProvider handles the connection attempt?

Q2: Is there a recommended or "safe" sequence for activating these three specific managers (NEFilterManager, NETransparentProxyManager, NEDNSProxyManager) from the main application to ensure the system applies them correctly and avoids conflicts?

Q3: When the network is under high load, is there a risk that one provider (e.g., the DNS Proxy) might experience delays or resource starvation, potentially causing it to miss network events or logs while the others continue to process traffic?

Part 2: Bidirectional Inter-Process Communication (IPC/XPC)

I am using a single, bidirectional NSXPCConnection channel (IPCConnection) to send logs from all three providers back to the main application for processing.

My Current IPC Setup (Simplified):

/// App --> Provider IPC (Implemented by Provider)
@objc protocol ProviderCommunication {
    func register(_ completionHandler: @escaping (Bool) -> Void)
}

/// Provider --> App IPC (Implemented by Main App)
@objc protocol AppCommunication {
    func handleLogs(logData: Data)
}

And then in IPCConnection, I call the above function after receiving the logs from extension:


func sendLogs(logData: Data) {
        if let logString = String(data: logData, encoding: .utf8) {
            os_log("IPCConnection: Log entry received: %{public}@", logString)
        }
        
        // Use remoteAppConnection (captured from app via register()) instead of currentConnection
        guard let appProxy = remoteAppConnection else {
            os_log("IPCConnection: No remote app proxy")
            return
        }
        appProxy.handleLogs(logData: logData)
    }

Each extension sends the log to IPCConnection class/file by calling sendLogs function like:

private let ipcConnection: IPCConnection
.......

ipcConnection.sendLogs(logData: logEntry)

or

IPCConnection.shared.sendLogs(logData: logEntry)

Q4: Since all three extensions (which run in the same Xcode target) will be asynchronously calling sendLogs(logData:) simultaneously during high network traffic, is it safe to use a single XPC connection?

Any guidance on the interaction between these three types of network extensions and best practices for robust, concurrent XPC communication would be greatly appreciated.

Please let me know if more clarifications needed from my end.

Thank you

Need Inputs on Which Extension to Use
 
 
Q