AVAudioPlayerNode scheduleBuffer leaks memory

I'm building a streaming app on visionOS that can play sound from audio buffers each frame. The audio format has a bitrate of 48000, and each buffer has 480 samples.

I noticed when calling

audioPlayerNode.scheduleBuffer(audioBuffer)

The memory keeps increasing at the speed of 0.1MB per second And at around 4 minutes, the node seems to be full of buffers and had a hard reset, at which point, the audio is stopped temporary with a memory change. see attached screenshot.

However, if I call

 audioPlayerNode.scheduleBuffer(audioBuffer, at: nil, options: .interrupts)

The memory leak issue is gone, but the audio is broken (sounds like been shortened).

Below is the full code snippet, anyone knows how to fix it?

@Observable
final class MyAudioPlayer {
    private var audioEngine: AVAudioEngine = .init()
    private var audioPlayerNode: AVAudioPlayerNode = .init()
    private var audioFormat: AVAudioFormat?

    

    init() {
        audioEngine.attach(audioPlayerNode)
        audioEngine.connect(audioPlayerNode, to: audioEngine.mainMixerNode, format: nil)

        try? AVAudioSession.sharedInstance().setCategory(.playback, mode: .default)
        try? AVAudioSession.sharedInstance().setActive(true)

        audioEngine.prepare()
        try? audioEngine.start()
        audioPlayerNode.play()
    }


    // more code...


    /// callback every frame
    private func audioFrameCallback_Non_Interleaved(buf: UnsafeMutablePointer<Float>?, samples: Int) { 
        guard let buf,
        let format = AVAudioFormat(commonFormat: .pcmFormatFloat32, sampleRate: 48000, channels: 2, interleaved: false),
        let audioBuffer = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: AVAudioFrameCount(samples))
        else { return }

        audioBuffer.frameLength = AVAudioFrameCount(samples)

        if let data = audioBuffer.floatChannelData {
            for channel in 0 ..< Int(format.channelCount) {
                for frame in 0 ..< Int(audioBuffer.frameLength) {
                    data[channel][frame] = buf[frame * Int(format.channelCount) + channel]
                }
            }
        }

        // memory leak here
        audioPlayerNode.scheduleBuffer(audioBuffer)
    }
}