AVAudioSession setActive throws error 561015905 in Keyboard Extension

I am trying to record and play audio from keyboard extension in Swift, but it throws an error on line recordingSession.setActive(true)

Error: failed to record The operation couldn’t be completed. (OSStatus error 561015905.)

I have already set the key RequestsOpenAccess to true in info.plist and granted full access to the keyboard extension.