NETransparentProxyProvider reset connections upon configuration change

Question

Created 1d

Replies 2

Boosts 0

Participants 2

I'm working on developing a transparent proxy provider extension, and I am trying to figure out how to handle a change in configuration that would result in a different verdict from handleNewFlow()

Consider the following scenario:

The proxy provider is started with configuration A, and a bunch of packet flows get a verdict of NO from handleNewFlow(). These flows are now handled by the system and get routed out to the internet normally.
Some application changes the protocolConfiguration property to configuration B, and the proxy provider detects this change via KVO.
This new configuration changes the verdict that would have been returned from handleNewFlow() to YES, requiring that traffic to be handled by the transparent proxy provider instead of the system.
These flows should be closed (eg: by calling closeReadWithError()) but the proxy provider has no record of them because we previously returned NO

Is there a way that a transparent proxy provider can get the operating system to close the currently open flows so that they can be re-evaluated by handleNewFlow() and directed into the transparent proxy instead?

Answer 1

nkirby OP

1d

Some of the approaches to this problem that I have thought about, but aren't perfect include:

Enumerate the open TCP connections using sysctl() (similar to how netstat does it) and then force-close them using pfctlinput(PRC_UNREACH_PORT, &addr). This seems like it should work, but it doesn't do anything to address UDP flows that ought to be closed.
Restarting the NETransparentProxyProvider seems like it should cause the kernel's flow director to restart, but I can't really find any documentation on how this works and whether it would trigger the existing flows to be closed. It would also be nice if the transparent proxy provider could restart itself in this case.
There is an undocumented syscall: pid_shutdown_sockets() that we could use to force-close sockets for an entire process. But this seems mostly used for suspend/resume operations. And it's probably a bad idea to try and use undocumented APIs.
There is an undocumented method fetchFlowStatesWithCompletionHandler() that I had hoped would return a list of flows that we could close, but this does not seem to return the flows where we returned NO from handleNewFlow().

Answer 2

DTS Engineer OP

Apple

12h

I’m skeptical that there’s a good way to do this, but I have one suggestion for you…

Starting a transparent proxy should close all flows. This was a deliberate design choice to handle cases like yours, where an upcoming proxy wants to make sure that it sees all traffic [1].

An app proxy, and hence a transparent proxy, can stop itself by calling its cancelProxyWithError(_:) method.

So, if you can find a way to convince your transparent proxy to start again, you’ll have your solution. And that’s where things get tricky. It’s possible that an on-demand rule might help with that. However, this isn’t something I’ve played around with.

IMPORTANT You mentioned a bunch of “undocumented” stuff. You have to be careful here. There’s a difference between:

Stuff that’s in the SDK but documentation is lacking
Stuff that’s not in the SDK

Stuff in the second category is not undocumented; rather, it’s an implementation detail. Those can change at any time and it’s not something you should build a product on.

But if you bump into anything in the first category, I’m happy to help out here.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] Curiously, a bunch of transparent proxy folks want the opposite behaviour, that is, for starting the proxy to leave all the flows intact. That’s currently not possible. We already have an ER on file requesting control over this (FB8969320).