A very serious problem about NetworkExtension NETransparentProxyProvider

and the whole computer could not access the Internet I turn off the VPN. Turn off VPN and then Internet can be access

It sounds like something actually went wrong with your NETransparentProxyProvider. If you put logs in the main.m of your Network System extension, and then you add a constructor to your provider with logs, do you see your provider getting restarted after this issue takes place? If so you hit a crash and that is why the internet is not available.

If your provier is not restarting, look for other log issues in the Console.app or stream out the logs to the terminal using os_log.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

@meaton

Thank you. I followed your suggestion. First, the provider did not restart, it was still running normally, but no new flow could be received.

Then I checked the os_log and found that com.apple.networkextension did report an error below:

@meaton Looks like a system bug?

(1485082327): plugin wrote 4104 bytes
(0): data: failed to write a message: no buffer space available
(1485082327): app receive buffer is full, queueing 2736 bytes
 (0): data: write error: Message too long
2022-02-11 15:13:18.736 [2002:8013] [com.apple.networkextension] (2463085674): Got an error when sending 131072 bytes to the kernel: Message too long
(2463085674): writes are still pending, delaying sending close until all data is sent to the kernel

(0): data: failed to write a message: no buffer space available (1485082327): app receive buffer is full, queueing 2736 bytes (0): data: write error: Message too long

Thank you for attaching this screen shot and these logs. The message that says that there is no buffer space available is the result of the ENOBUFS error. So it looks like data is being retained somehow but ever released. Are you buffering a lot of this data in your provider? I would take a deeper look at this with Instruments.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

@meaton

Are you buffering a lot of this data in your provider? 

No. But I'm not quite sure what you mean. Let me tell you the specific content of my provider.

I create a socket for each TCP/UDP Flow, I mean, each flow has one individual socket(I use the open source library https://github.com/robbiehanson/CocoaAsyncSocket to create socket).

Then, in my provider, each socket will directly forward the traffic in the Flow to the local agent program, and then write the return data of the local agent program back to the TCP/UDP Flow using - (void)writeData:(NSData *)data withCompletionHandler:(void (^)(NSError *error))completionHandler, and every time the socket I created is released correctly.

Is there something wrong with my design? It doesn't look like there should be a problem.

We are worried and hope to get help soon, thank you @meaton.

Is there something wrong with my design? It doesn't look like there should be a problem.

Have you tried using XPC to communicate with your agent instead of a socket?
Or to rule out that sending the data to the agent is the issue, have you tried using your NETransparentProxyProvider as a pass-through proxy instead? For example, if you read / write between the flow and NWTCPConnection / NWUDPSession, then do you have this same issue?

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

@meaton

For example, if you read / write between the flow and NWTCPConnection / NWUDPSession, then do you have this same issue?

I don't know what is wrong with NWTCPConnection. When I use - (NWTCPConnection *)createTCPConnectionToEndpoint:(NWEndpoint *)remoteEndpoint enableTLS:(BOOL)enableTLS TLSParameters:(NWTLSParameters *)TLSParameters delegate:(id)delegate; to create tcp connection and observe the state, I will receive NWTCPConnectionStateConnecting, NWTCPConnectionStateConnected first, and then receive NWTCPConnectionStateDisconnected immediately : ) . At this time, the connection is not real disconnected, and data can still be transmitted, it is funny 😂 .....

At this time, the connection is not real disconnected, and data can still be transmitted

If you have logs of the connection going into NWTCPConnectionStateDisconnected and you have a packet trace of that same connection sending and receiving data on the network then you should open a bug report here.

Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com

I also encountered the same problem and had the same log output。

It seems to be a system bug。 Do you have any latest news。

I also have the same problem and the same log output, do you have a solution?

Hello,

I am working on Network Extension and I have faced with the same issue:

10:33:53.854803+0300 com.my_app.transparentproxy (0): data: write error: Message too long 10:33:53.854840+0300 com.my_app.transparentproxy (0): Aborting the director 10:33:53.854862+0300 com.my_app.transparentproxy (1452558381): Got an error when sending 131072 bytes to the kernel: Message too long

Looks like the issue appears with buffer length 131072 which is equal to 2^17 in all logs above (my and other threads). In my system net.inet.tcp.recvspace is equal to 131072

# sysctl -n net.inet.tcp.recvspace
131072

I solved this issue by restriction of max buffer length (64k) (system call nw_connection_receive). I found the solution by trial and error. In case of using (net.inet.tcp.recvspace -1), I see the same issue. So, it is unclear max value of the maximum_length.

Are there any recommendation?

Thank you in advance, Pavel

I am working on Network Extension and I have faced with the same issue:

I’d like to get a better handle on the context here. My best guess is that:

• You’re implementing a transparent proxy.
• In your proxy you opt in to proxying a TCP flow, by returning true from one of the handle-new-flow methods.
• As part of proxying the flow, you forward traffic coming from the flow to the network and vice versa.
• You’re hitting this issue in the vice versa (inbound) case.
• You read a large chunk of data from the network.
• And then you write the whole chunk to the flow by calling the write(_:withCompletionHandler:) method, or its Swift async equivalent.
• If the chunk is too large, you see this failure.

Is that right?

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"