Bluetooth Permissions denied when trying to access while not logged in

As of Monterrey 12.3, I am getting this error from my authorization plugin. Any ideas for how to get the authorization plugin to get permissions? (I tried having it connection a daemon that has bluetooth permissions via XPC as well as an application, but that didn't work as well). The bluetooth does work when in logged mode on the application.

func peripheralManagerDidUpdateState(_ peripheral: CBPeripheralManager) {
    switch peripheral.state {
    case .unknown:
      Log.i("Bluetooth state is unknown")
      reset()
    case .resetting:
      Log.i("Bluetooth is resetting")
      reset()
    case .unsupported:
      Log.w("Bluetooth is not supported on this device")
      reset()
    case .unauthorized: // I keep getting this response
      Log.w("Bluetooth permission was not granted")
      reset()
    case .poweredOff:
      Log.i("Bluetooth is powered off")
      reset()
    case .poweredOn:
      Log.i("Bluetooth is ready")
      addServiceIfNeeded()
    @unknown default:
      Log.w("Unknown peripheral state", peripheral.state)
    }
  }