AudioOutputUnitStart() is taking over 700ms

Our VOIP application is taking around 700msec in the function call  AudioOutputUnitStart(AudioUnit), which is causing the call setup delays. This delay is seen on iPhone11.
Any suggestions to minimize this delay would be highly appreciated.