(Another) Kernel deadlock/panic by NetworkExtension?

macOS 11.1 fixed RAW socket locking in Network Extension.
However one user reports another kernel deadlock with Network Extension, on macOS 11.1 20C69.

The deadlock seems to be caused by three processes:

Network Extension process:
```
  Thread 0xa36b62    DispatchQueue "NEFilterExtensionProviderContext queue"(66)    1 sample (1)    priority 46 (base 31)
  <thread QoS default (requested default, workloop servicer override default), IO tier 0>
  1  start_wqthread + 15 (libsystem_pthread.dylib + 9319) [0x7fff20342467] 1
    1  _pthread_wqthread + 314 (libsystem_pthread.dylib + 13465) [0x7fff20343499] 1
      1  _dispatch_workloop_worker_thread + 819 (libdispatch.dylib + 80989) [0x7fff201abc5d] 1
        1  _dispatch_lane_invoke + 375 (libdispatch.dylib + 41163) [0x7fff201a20cb] 1
          1  _dispatch_lane_serial_drain + 606 (libdispatch.dylib + 38398) [0x7fff201a15fe] 1
            1  _dispatch_client_callout + 8 (libdispatch.dylib + 14279) [0x7fff2019b7c7] 1
              1  _dispatch_call_block_and_release + 12 (libdispatch.dylib + 9693) [0x7fff2019a5dd] 1
                1  __92-[NEFilterDataExtensionProviderContext handleNewFlow:reply:controlSocket:completionHandler:]_block_invoke_2 + 365 (NetworkExtension + 322118) [0x7fff2dccba46] 1
                  1  -[NEFilterDataSavedMessageHandler enqueueWithFlow:context:] + 183 (NetworkExtension + 343151) [0x7fff2dcd0c6f] 1
                    1  -[NEFilterDataSavedMessageHandler executeWithFlow:context:] + 294 (NetworkExtension + 343479) [0x7fff2dcd0db7] 1
                      1  -[NEFilterDataSavedMessageHandler executeVerdictHandlerWithFlow:verdict:context:] + 305 (NetworkExtension + 343956) [0x7fff2dcd0f94] 1
                        1  __92-[NEFilterDataExtensionProviderContext handleNewFlow:reply:controlSocket:completionHandler:]_block_invoke.287 + 121 (NetworkExtension + 322365) [0x7fff2dccbb3d] 1
                          1  -[NEFilterSocketFlow createNewFlowReply:controlSocket:verdict:context:] + 429 (NetworkExtension + 366248) [0x7fff2dcd66a8] 1
                            1  write + 10 (libsystem_kernel.dylib + 15450) [0x7fff20314c5a] 1
                             *1  hndl_unix_scall64 (in kernel) + 22 [0xffffff800025f1f6] 1
                               *1  unix_syscall64 (in kernel) (systemcalls.c:412) [0xffffff8000968cdb] 1
                                 *1  write_nocancel (in kernel) (sys_generic.c:513) [0xffffff8000876757] 1
                                   *1  dofilewrite (in kernel) (sys_generic.c:623) [0xffffff8000876945] 1
                                     *1  soo_write (in kernel) (sys_socket.c:153) [0xffffff800087bfd8] 1
                                       *1  sosend (in kernel) (uipc_socket.c:2521) [0xffffff80008a78c9] 1
                                         *1  ctl_send (in kernel) (kern_control.c:870) [0xffffff800082d439] 1
                                           *1  cfil_ctl_send (in kernel) (content_filter.c:1976) [0xffffff80006a6b82] 1
                                             *1  cfil_socket_from_sock_id (in kernel) (content_filter.c:1678) [0xffffff80006a96c1] 1
                                               *1  tcp_lock (in kernel) (tcp_subr.c:3125) [0xffffff800071b5fe] 1
                                                 *1  lck_mtx_lock_contended (in kernel) (locks_i386.c:0) [0xffffff80003e3eab] 1
                                                   *1  lck_mtx_lock_wait_x86 (in kernel) (locks_i386.c:3595) [0xffffff80003e4646] 1
                                                     *1  thread_block_reason (in kernel) (sched_prim.c:3170) [0xffffff80002dc3af] 1
                                                       *1  thread_invoke (in kernel) (sched_prim.c:2601) [0xffffff80002dd912] 1
                                                         *1  machine_switch_context (in kernel) (pcb.c:485) [0xffffff80003e768b] (blocked by turnstile waiting for MyNetApp [13174] [unique pid 709951] thread 0xa2f5f5) 1

  Thread 0xa2f5d3    1 sample (1)    priority 31 (base 31)
  <thread QoS default (requested default), IO tier 0>
  1  _dispatch_sig_thread + 53 (libdispatch.dylib + 83227) [0x7fff201ac51b] 1
    1  __sigsuspend_nocancel + 10 (libsystem_kernel.dylib + 26146) [0x7fff20317622] 1
     *1  _sleep_continue (in kernel) (kern_synch.c:65) [0xffffff800085aa20] 1

  Binary Images:
           0x105249000 -                ???  ???                                 <9947A42D-F1D4-3866-91D6-9D663A4EA27E>
        0x7fff20198000 -     0x7fff201dcfff  libdispatch.dylib (1271.40.12)      <CEF1460B-1362-381A-AE69-6BCE2D8C215B>  /usr/lib/system/libdispatch.dylib
        0x7fff20311000 -     0x7fff2033ffff  libsystem_kernel.dylib (7195.60.75) <4BD61365-29AF-3234-8002-D989D295FDBB>  /usr/lib/system/libsystem_kernel.dylib
        0x7fff20340000 -     0x7fff2034bfff  libsystem_pthread.dylib (454.60.1)  <8DD3A0BC-2C92-31E3-BBAB-CE923A4342E4>  /usr/lib/system/libsystem_pthread.dylib
        0x7fff2dc7d000 -     0x7fff2df08fff  NetworkExtension (1295.60.5)        <F7DE2856-7FC5-343B-A118-416C253ED686>  /System/Library/Frameworks/NetworkExtension.framework/Versions/A/NetworkExtension
   *0xffffff8000210000 - 0xffffff8000c0ffff  kernel (7195.60.75)                 <82E2050C-5936-3D24-AD3B-EC4EC5C09E11>  /System/Library/Kernels/kernel
```

Remoted (watchdog timeout)
```
Process:          remoted [15192] [unique pid 711959] (suspended) (zombie)
UUID:             4F9E8344-152E-3508-B21C-99F972F0B6C9
Path:             /usr/libexec/remoted
Architecture:     x86_64
Footprint:        900 KB
Time Since Fork:  121s
Num samples:      1 (1)
Note:             Suspended for 1 sample
Note:             Terminated (zombie) for 1 sample

  Thread 0xa374e4    DispatchQueue "com.apple.main-thread"(1)    1 sample (1)    priority 31 (base 31)
  <thread QoS unspecified (requested default), IO tier 0>
  1  start + 1 (libdyld.dylib + 87585) [0x7fff20361621] 1
    1  ??? (remoted + 30466) [0x10b332702] 1
      1  ??? (remoted + 136197) [0x10b34c405] 1
        1  wd_endpoint_activate + 106 (WatchdogClient + 13866) [0x7fff2ba7b62a] 1
          1  wd_kickoff_ping + 277 (WatchdogClient + 13214) [0x7fff2ba7b39e] 1
            1  ??? (remoted + 136647) [0x10b34c5c7] 1
              1  ??? (remoted + 137186) [0x10b34c7e2] 1
                1  remote_device_copy_unique_of_type + 118 (RemoteServiceDiscovery + 14784) [0x7fff32f069c0] 1
                  1  xpc_connection_send_message_with_reply_sync + 238 (libxpc.dylib + 45484) [0x7fff200891ac] 1
                    1  dispatch_mach_send_with_result_and_wait_for_reply + 50 (libdispatch.dylib + 105326) [0x7fff201b1b6e] 1
                      1  _dispatch_mach_send_and_wait_for_reply + 518 (libdispatch.dylib + 104310) [0x7fff201b1776] 1
                        1  mach_msg_trap + 10 (libsystem_kernel.dylib + 3710) [0x7fff20311e7e] 1
                         *1  thread_terminate_continue (in kernel) (thread.c:363) [0xffffff80002f8d10] (suspended) 1

  Thread 0xa374f1    DispatchQueue "com.apple.remoted"(18)    1 sample (1)    priority 46 (base 31)
  <thread QoS unspecified (requested default, workloop servicer override default), IO tier 0>
  1  start_wqthread + 15 (libsystem_pthread.dylib + 9319) [0x7fff20342467] 1
    1  _pthread_wqthread + 314 (libsystem_pthread.dylib + 13465) [0x7fff20343499] 1
      1  _dispatch_workloop_worker_thread + 819 (libdispatch.dylib + 80989) [0x7fff201abc5d] 1
        1  _dispatch_lane_invoke + 375 (libdispatch.dylib + 41163) [0x7fff201a20cb] 1
          1  _dispatch_lane_serial_drain + 606 (libdispatch.dylib + 38398) [0x7fff201a15fe] 1
            1  _dispatch_client_callout + 8 (libdispatch.dylib + 14279) [0x7fff2019b7c7] 1
              1  _dispatch_call_block_and_release + 12 (libdispatch.dylib + 9693) [0x7fff2019a5dd] 1
                1  __multiverse_device_change_state_block_invoke + 221 (MultiverseSupport + 38924) [0x7fff2ce2180c] 1
                  1  ??? (remoted + 63704) [0x10b33a8d8] 1
                    1  ??? (remoted + 64434) [0x10b33abb2] 1
                      1  ??? (remoted + 65678) [0x10b33b08e] 1
                        1  multiverse_device_connect_with_timeout + 112 (MultiverseSupport + 27532) [0x7fff2ce1eb8c] 1
                          1  socket + 10 (libsystem_kernel.dylib + 16714) [0x7fff2031514a] 1
                           *1  hndl_unix_scall64 (in kernel) + 22 [0xffffff800025f1f6] 1
                             *1  unix_syscall64 (in kernel) (systemcalls.c:412) [0xffffff8000968cdb] 1
                               *1  socket (in kernel) (uipc_syscalls.c:207) [0xffffff80008b6683] 1
                                 *1  socket_common (in kernel) (uipc_syscalls.c:0) [0xffffff80008b67ef] 1
                                   *1  socreate_internal (in kernel) (uipc_socket.c:782) [0xffffff80008a44d0] 1
                                     *1  tcp_usr_attach (in kernel) (tcp_usrreq.c:176) [0xffffff8000721a41] 1
                                       *1  in_pcballoc (in kernel) (in_pcb.c:666) [0xffffff80006dbbf2] 1
                                         *1  lck_rw_lock_exclusive_gen (in kernel) (locks_i386.c:1181) [0xffffff80003e286d] 1
                                           *1  thread_block_reason (in kernel) (sched_prim.c:3170) [0xffffff80002dc3af] 1
                                             *1  thread_invoke (in kernel) (sched_prim.c:2601) [0xffffff80002dd912] 1
                                               *1  machine_switch_context (in kernel) (pcb.c:485) [0xffffff80003e768b] (suspended) 1

  Thread 0xa374f2    1 sample (1)    priority 31 (base 31)
  <thread QoS unspecified (requested default), IO tier 0>
  1  <truncated backtrace> 1
    1  start_wqthread + 0 (libsystem_pthread.dylib + 9304) [0x7fff20342458] 1
     *1  thread_terminate_continue (in kernel) (thread.c:363) [0xffffff80002f8d10] (suspended) 1

  Binary Images:
           0x10b32b000 -        0x10b36afff  remoted (2038.40.38)                <4F9E8344-152E-3508-B21C-99F972F0B6C9>  /usr/libexec/remoted
        0x7fff2007e000 -     0x7fff200b3fff  libxpc.dylib (2038.40.38)           <003A027D-9CE3-3794-A319-88495844662D>  /usr/lib/system/libxpc.dylib
        0x7fff20198000 -     0x7fff201dcfff  libdispatch.dylib (1271.40.12)      <CEF1460B-1362-381A-AE69-6BCE2D8C215B>  /usr/lib/system/libdispatch.dylib
        0x7fff20311000 -     0x7fff2033ffff  libsystem_kernel.dylib (7195.60.75) <4BD61365-29AF-3234-8002-D989D295FDBB>  /usr/lib/system/libsystem_kernel.dylib
        0x7fff20340000 -     0x7fff2034bfff  libsystem_pthread.dylib (454.60.1)  <8DD3A0BC-2C92-31E3-BBAB-CE923A4342E4>  /usr/lib/system/libsystem_pthread.dylib
        0x7fff2034c000 -     0x7fff20386fff  libdyld.dylib (832.7.1)             <2F8A14F5-7CB8-3EDD-85EA-7FA960BBC04E>  /usr/lib/system/libdyld.dylib
        0x7fff2ba78000 -     0x7fff2ba7bfff  WatchdogClient (98.60.1)            <8374BBBB-65CB-3D46-9AD6-0DD1FB99AD88>  /System/Library/PrivateFrameworks/WatchdogClient.framework/Versions/A/WatchdogClient
        0x7fff2ce18000 -     0x7fff2ce27fff  MultiverseSupport (109.40.1)        <D8F8D659-07C4-35B9-83FA-D4A084918F40>  /System/Library/PrivateFrameworks/MultiverseSupport.framework/Versions/A/MultiverseSupport
        0x7fff32f03000 -     0x7fff32f12fff  RemoteServiceDiscovery (2038.40.38) <EFEFC279-E73A-3557-A4AE-BF67E7C96962>  /System/Library/PrivateFrameworks/RemoteServiceDiscovery.framework/Versions/A/RemoteServiceDiscovery
   *0xffffff8000210000 - 0xffffff8000c0ffff  kernel (7195.60.75)                 <82E2050C-5936-3D24-AD3B-EC4EC5C09E11>  /System/Library/Kernels/kernel
```

And MyNetApp
```
  Thread 0xa2f5f5    1 sample (1)    priority 93 (base 20)
  <thread QoS utility (requested default), timers tier 3 (coalesced), IO tier 1>
  1  thread_start + 15 (libsystem_pthread.dylib + 9339) [0x7fff2034247b] 1
    1  _pthread_start + 224 (libsystem_pthread.dylib + 26960) [0x7fff20346950] 1
      // some userspace frames are omitted
        1  __connect + 10 (libsystem_kernel.dylib + 17382) [0x7fff203153e6] 1
         *1  hndl_unix_scall64 (in kernel) + 22 [0xffffff800025f1f6] 1
           *1  unix_syscall64 (in kernel) (systemcalls.c:412) [0xffffff8000968cdb] 1
             *1  connect_nocancel (in kernel) (uipc_syscalls.c:733) [0xffffff80008b7ae9] 1
               *1  soconnectlock (in kernel) (uipc_socket.c:1714) [0xffffff80008a63fb] 1
                 *1  tcp_usr_connect (in kernel) (tcp_usrreq.c:0) [0xffffff8000721e34] 1
                   *1  tcp_connect_complete (in kernel) (tcp_usrreq.c:426) [0xffffff8000724348] 1
                     *1  tcp_output (in kernel) (tcp_output.c:2689) [0xffffff80007126be] 1
                       *1  tcp_ip_output (in kernel) (tcp_output.c:3016) [0xffffff80007135f9] 1
                         *1  ip_output_list (in kernel) (ip_output.c:1067) [0xffffff80006f78ce] 1
                           *1  pf_af_hook (in kernel) (pf_ioctl.c:4586) [0xffffff8000685076] 1
                             *1  pf_test_mbuf (in kernel) (pf.c:9547) [0xffffff80006618b8] 1
                               *1  pf_test (in kernel) (pf.c:0) [0xffffff80006623d3] 1
                                 *1  pf_test_rule (in kernel) (pf.c:0) [0xffffff8000668c69] 1
                                   *1  pf_socket_lookup (in kernel) (pf.c:4096) [0xffffff80006611a0] 1
                                     *1  in_pcblookup_hash_exists (in kernel) (in_pcb.c:2368) [0xffffff80006e0df5] 1
                                       *1  IORWLockRead (in kernel) (locks_i386.c:1514) [0xffffff80003e2469] 1
                                         *1  thread_block_reason (in kernel) (sched_prim.c:3170) [0xffffff80002dc3af] 1
                                           *1  thread_invoke (in kernel) (sched_prim.c:2601) [0xffffff80002dd912] 1
                                             *1  machine_switch_context (in kernel) (pcb.c:485) [0xffffff80003e768b] 1
```


I failed to reproduce this issue, while the user can reproduce it consistently.

I wonder if anyone has also encountered this panic.

FB8968013 for full kernel panics.
(Another) Kernel deadlock/panic by NetworkExtension?
 
 
Q