NETransparentProxy breaks incoming connections

Hello!

I create NETransparentProxy with the following configuration:

Code Block 
settings.includedNetworkRules = @[[[NENetworkRule alloc] 
		initWithRemoteNetwork:
				[NWHostEndpoint endpointWithHostname:@"2000::" port:@"0"]
		remotePrefix:3
		localNetwork:nil localPrefix:0
		protocol:NENetworkRuleProtocolTCP
		direction:NETrafficDirectionOutbound]];

So, there is wildcard port and IPv6 network 2000::/3 (global unicast)

After this rule is created, any incoming connection from any IPv6 host will be dropped.
And when it is dropped, there is no any activity in AppProxyProvider or and NSLog of Network extension.

Is it a bug? Why it affects incoming traffic while NETransparentProxy may only have outbound rules?

Good old kernel extensions don't break incoming connections when diverting sockets.

I use Catalina but Big Sur users are also reporting this problem.
Up vote post of sfionov Down vote post of sfionov
1.7k views
Posted by
sfionov
Reply
Add a Comment

Replies

Are you seeing the outbound connections here and are you handling them in handleNewFlow?


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com
Posted by
meaton
Up vote reply of meaton Down vote reply of meaton
Add a Comment
Yes, outgoing connections are handled in handleNewFlow (created real connection, opened flow and forwarded data in both directions) and works - there are log records in Console.

But incoming connections from hosts from "intercepted" networks doesn't work while TransparentProxy is running, and no records in Console about them (may be because they are incoming and TransparentProxy have nothing to do with them, but anyway they doesn't work)
Posted by
sfionov
Up vote reply of sfionov Down vote reply of sfionov
Add a Comment

Yes, outgoing connections are handled in handleNewFlow (created real connection, opened flow and forwarded data in both directions) and works - there are log records in Console.

Great. This is the first part I wanted to confirm.


In regards to:

But incoming connections from hosts from "intercepted" networks doesn't work while TransparentProxy is running, and no records in Console about them (may be because they are incoming and TransparentProxy have nothing to do with them, but anyway they doesn't work)

Do you see the incoming connections reaching their local destination? The TransparentProxy in this case has a NENetworkRule setup for NETrafficDirectionOutbound connections.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com
Posted by
meaton
Up vote reply of meaton Down vote reply of meaton
Add a Comment

Do you see the incoming connections reaching their local destination? The TransparentProxy in this case has a NENetworkRule setup for NETrafficDirectionOutbound connections.

No, they doesn't reach my host. Connections to my host on another host just hang.
And nothing in netstat on my host.

If I stop transparent proxy, everything starts working as usual.
If I specify another network in outbound rule, everything starts working as usual too.

Posted by
sfionov
Up vote reply of sfionov Down vote reply of sfionov
Add a Comment

Do you see the incoming connections reaching their local destination? The TransparentProxy in this case has a NENetworkRule setup for NETrafficDirectionOutbound connections.

Sorry, may be I didn't understand question.

Outgoing connections are okay - they works and reaches destinations.
Incoming connections doesn't work when transparent proxy is active (as described in my previous message) and I'm connecting from host that matches NENetworkRule from this topic - connections hang.
Posted by
sfionov
Up vote reply of sfionov Down vote reply of sfionov
Add a Comment
I think the key thing here is that you mention:

If I stop transparent proxy, everything starts working as usual.

This is worth a bug report. Please open a bug report with the following information, along with any logs or sample projects, and respond back with the Feedback ID.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com
Posted by
meaton
Up vote reply of meaton Down vote reply of meaton
Add a Comment

This is worth a bug report. Please open a bug report with the following information, along with any logs or sample projects, and respond back with the Feedback ID.

Reported, FB8486799
Posted by
sfionov
Up vote reply of sfionov Down vote reply of sfionov
Add a Comment
I found another bug in how Safari's outbound socket is created when TransparentProxy is running, and outbound address is IPv6. It is wrongly shown as 0.0.0.0:0 in Inspect -> Network tab, and the same in netstat.

Commented in the same ticket.

There are no such problems with Chrome-based browsers.
Posted by
sfionov
Up vote reply of sfionov Down vote reply of sfionov
Add a Comment
Thank you for providing the Feedback ID. I see it internally and have copied myself on it for further details.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com
Posted by
meaton
Up vote reply of meaton Down vote reply of meaton
Add a Comment
Hello!

Any news on this issues?

Actually there are three separate issues:
1) Incoming connections are broken as in this issue
2) Transparent proxy can't filter localhost (but NKI can)
3) Safari is creating connections with invalid address family when using TransparentProxy and shows 0.0.0.0:0 in filtering log, and tcp4 in netstat, for IPv6 connections

Should I create separate feedback ID for them?

Posted by
sfionov
Up vote reply of sfionov Down vote reply of sfionov
Add a Comment
No news on your Feedback. It looks like 1 and 2 may be covered on your Feedback already. If 3 is not also covered please add it with any additional information or details you may have.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com
Posted by
meaton
Up vote reply of meaton Down vote reply of meaton
Add a Comment
I found another example of bug that application shows 0.0.0.0 instead of IPv6 address.

Code Block 
$ /usr/bin/ssh fa2.lamo.su
The authenticity of host 'fa2.lamo.su (0.0.0.0)' can't be established.
ECDSA key fingerprint is SHA256:dZsSQrUwwPgm/EZ8wGkuijpIEslPTg3NcfIEQgWLqiY.
Are you sure you want to continue connecting (yes/no/[fingerprint])? ^C

Apple ssh also shows 0.0.0.0 instead of IPv6 addresses.

But
Code Block 
$ ~/homebrew/bin/ssh fa2.lamo.su
The authenticity of host 'fa2.lamo.su (2a01:4f8:1c17:4e80::1)' can't be established.
ECDSA key fingerprint is SHA256:dZsSQrUwwPgm/EZ8wGkuijpIEslPTg3NcfIEQgWLqiY.
Are you sure you want to continue connecting (yes/no/[fingerprint])? ^C

Homebrew ssh works as expected.

I guess that both Safari and Apple ssh may use the same network framework, that doesn't work correctly with NETransparentProxy and shows 0.0.0.0 instead of IPv6 addresses.

Posted by
sfionov
Up vote reply of sfionov Down vote reply of sfionov
Add a Comment
Is there any update on the issues posted here?

All three of them seem rather serious. Especially the one about Safari and wrong source address. As far as I understand, it means that every app that uses new networking api will be broken by mere presence of a transparent proxy.

Is there at least a temporary workaround for this bug?
Posted by
ameshkov
Up vote reply of ameshkov Down vote reply of ameshkov
Add a Comment

As far as I understand, it means that every app that uses new networking api will be broken by mere presence of a transparent proxy.

This is simply not the case. As being pointed out on this thread there is a specific case where the author has uncovered an issue related to using 2000::/3 in their NENetworkRule and handling incoming connections with this rule. Also, another case was brought up by the author that a default route v4 address (0.0.0.0) was used instead of v6 address. I do not know the specifics of this case or what NENetworkRule is being used here. Having said that I believe it is not correct to say that using the Transparent Proxy API will break every app.

To get to the bottom of what is happening with your specific issue, please open a TSI and I will dig into this further to and examine your example case and your NENetworkRule's being used so that I can provide further direction.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com
Posted by
meaton
Up vote reply of meaton Down vote reply of meaton
Add a Comment
Hi! Please allow me to chime in since I'm also seeing an issue with macOS's builtin SSH when using a transparent proxy network extension, even without IPv6.

Here's the steps:

  1. Set your network interface's IPv6 settings to link-local-only (effectively disabling IPv6)

  2. Start a network extension (this is a simple TCP forwarder for TCP and IPv4 only) that I've attached in thread/660163 and feedback FB8690357, also available here: https://github.com/ngorskikh/transparentproxyrepro

  3. Try connecting to some SSH server with Apple's default ssh, and OpenSSH from Homebrew. Here's my results:

Code Block 
➜ TransparentProxy git:(master) /usr/bin/ssh fa2.lamo.su   
The authenticity of host 'fa2.lamo.su (0.0.0.0)' can't be established.
ECDSA key fingerprint is SHA256:dZsSQrUwwPgm/EZ8wGkuijpIEslPTg3NcfIEQgWLqiY.
Are you sure you want to continue connecting (yes/no/[fingerprint])? no
Host key verification failed.
➜ TransparentProxy git:(master) /usr/local/Cellar/openssh/8.3p1/bin/ssh fa2.lamo.su
The authenticity of host 'fa2.lamo.su (88.99.32.235)' can't be established.
ECDSA key fingerprint is SHA256:dZsSQrUwwPgm/EZ8wGkuijpIEslPTg3NcfIEQgWLqiY.
Are you sure you want to continue connecting (yes/no/[fingerprint])? no
Host key verification failed.

Apple's SSH sees the wrong remote address for some reason.
Posted by
ngorskikh
Up vote reply of ngorskikh Down vote reply of ngorskikh
Add a Comment