FilterPacketProvider and outgoing bandwidth dramatically decrease in 10Gbit networks

Hello,

For several versions, we have had a NetworkExtension registered to "FilterDataProvider" for content filtering reasons, with no issues about performance in 10Gigabit networks.

In our latest version, we added registration to "FilterPacketProvider" for packet filtering purposes, and from that moment, we have observed that outgoing bandwidth dramatically decreases by 90% (from 883 MB/s to 140MB/s), simply due to being registered to "FilterPacketProvider," without any processing by us, just only registering and returning ALLOW:

	class FilterPacketProvider: {
	override func startFilter(completionHandler: @escaping (Error?) -> Void) {

		   self.packetHandler = {(_pContext: NEFilterPacketContext, _pIface: OS_nw_interface, _pDirection: NETrafficDirection, _pRawData: UnsafeRawPointer, _pRawDataLength: Int) -> NEFilterPacketProvider.Verdict in       
				return self.handleNewPacket(context: _pContext, interface: _pIface, direction: _pDirection, rawData: _pRawData, rawDataLength: _pRawDataLength)
			}
	}}
					
	func handleNewPacket(context: NEFilterPacketContext, interface: OS_nw_interface, direction: NETrafficDirection, rawData: UnsafeRawPointer, rawDataLength: Int) -> NEFilterPacketProvider.Verdict
	{       
		return .allow
	}	

The most curious thing is that this behaviour only happens with outgoing traffic, while incoming traffic does not experience any performance penalty. Reviewing similar cases, we found a similar post here: https://developer.apple.com/forums/thread/741965

So, our questions are:

1. Could we be doing something wrong when registering to NEFilterPacketProvider?
2. Is there still any known performance bug with outgoing traffic in NEFilterPacketProvider as I read in the post I found and attached?
3. If this bug is confirmed, is there any estimated date for its correction?

Best regards, Asier Gil.

Answered by Systems Engineer in 791572022

In general there is going to be a performance hit when enabling any of the Content Filter Providers. Why is this? When traffic attaches in the system it first decides whether your filter/provider needs to process this flow. If it does then the system halts the flow and a message is sent the provider infrastructure (in this case NetworkExtension) to let it know that your specific provider needs to make a filtering decision about this new flow (because its one you told NetworkExtension that your logic is concerned with a flow of this type). Then the flow is sent to your provider to make a filtering decision. This is the NEFilterDataProvider path for a flow. Now image this same workflow for individual packets on each flow and that would roughly be the workflow for NEFilterPacketProvider. So there is always going to be more of a performance hit when wanting to inspect each packet on a flow (even if you are just allowing it) as opposed to just processing flows with NEFilterDataProvider.

Matt Eaton - Networking

In general there is going to be a performance hit when enabling any of the Content Filter Providers. Why is this? When traffic attaches in the system it first decides whether your filter/provider needs to process this flow. If it does then the system halts the flow and a message is sent the provider infrastructure (in this case NetworkExtension) to let it know that your specific provider needs to make a filtering decision about this new flow (because its one you told NetworkExtension that your logic is concerned with a flow of this type). Then the flow is sent to your provider to make a filtering decision. This is the NEFilterDataProvider path for a flow. Now image this same workflow for individual packets on each flow and that would roughly be the workflow for NEFilterPacketProvider. So there is always going to be more of a performance hit when wanting to inspect each packet on a flow (even if you are just allowing it) as opposed to just processing flows with NEFilterDataProvider.

Matt Eaton - Networking

Hello again,

We fully understand that there will be a performance impact; however, a 90% reduction (from 883 MB/s to 140 MB/s) is unacceptable for our clients, who continuously perform backups over 10 Gbit networks, making it impossible to complete within a day.

We have protection products on other platforms (Windows/Linux), and their network packet engines like NDIS/LWF or WFP (that also have a network stack retention and a verdict systems) only impact about 5% approximately when we register with them.

Another important issue is how could it be explained that, when registered to FilterPacketProvider, only outbound traffic is penalized and not inbound traffic?. The network stack interruption, provider notification and final verdict mechanism is not the same for incoming traffic as it is for outgoing traffic?

Lastly, is there any workaround we can apply, such as registering only for inbound network packet traffic and not for outbound? How would that be implemented?

Best regards, Asier Gil

FilterPacketProvider and outgoing bandwidth dramatically decrease in 10Gbit networks
 
 
Q