Dive into the technical aspects of audio on your device, including codecs, format support, and customization options.

Audio Documentation

Post

Replies

Boosts

Views

Activity

iOS 16 RemoteIO: Input data proc returned inconsistent 2 packets
I am getting an error in iOS 16. This error doesn't appear in previous iOS versions. I am using RemoteIO to playback live audio at 4000 hz. The error is the following: Input data proc returned inconsistent 2 packets for 186 bytes; at 2 bytes per packet, that is actually 93 packets This is how the audio format and the callback is set: // Set the Audio format AudioStreamBasicDescription audioFormat; audioFormat.mSampleRate = 4000; audioFormat.mFormatID = kAudioFormatLinearPCM; audioFormat.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked; audioFormat.mFramesPerPacket = 1; audioFormat.mChannelsPerFrame = 1; audioFormat.mBitsPerChannel = 16; audioFormat.mBytesPerPacket = 2; audioFormat.mBytesPerFrame = 2; AURenderCallbackStruct callbackStruct; // Set output callback callbackStruct.inputProc = playbackCallback; callbackStruct.inputProcRefCon = (__bridge void * _Nullable)(self); status = AudioUnitSetProperty(audioUnit, kAudioUnitProperty_SetRenderCallback, kAudioUnitScope_Global, kOutputBus, &callbackStruct, sizeof(callbackStruct)); Note that the mSampleRate I set is 4000 Hz. In iOS 15 I get 0.02322 seconds of buffer duration (IOBufferDuration) and 93 frames in each callback. This is expected, because: number of frames * buffer duration = sampling rate 93 * 0.02322 = 4000 Hz However, in iOS 16 I am getting the aforementioned error in the callback. Input data proc returned inconsistent 2 packets for 186 bytes; at 2 bytes per packet, that is actually 93 packets Since the number of frames is equal to the number of packets, I am getting 1 or 2 frames in the callback and the buffer duration is of 0.02322 seconds. This didn't affect the playback of the "raw" signal, but it did affect the playback of the "processed" signal. number of frames * buffer duration = sampling rate 2 * 0.02322 = 0.046 Hz That doesn't make any sense. This error appears for different sampling rates (8000, 16000, 32000), but not for 44100. However I would like to keep 4000 as my sampling rate. I have also tried to set the sampling rate by using the setPreferredSampleRate(_:) function of AVAudioSession, but the attempt didn't succeed. The sampling rate was still 44100 after calling that function. Any help on this issue would be appreciated.
10
3
3.9k
Sep ’22
MauiVocalizer: 16038 (Resource load failed)
Hi! Just created a new empty project with just a AVSpeechSynthesizer var and when I try to play a tts string in app console I see: Query for com.apple.MobileAsset.VoiceServices.VoiceResources failed: 2 Unable to list voice folder [AXTTSCommon] MauiVocalizer: 11006 (Can't compile rule): regularExpression=\Oviedo(?=, (\x1b\pause=\d+\)?Florida)\b, message=unrecognized character follows , characterPosition=1 [AXTTSCommon] MauiVocalizer: 16038 (Resource load failed): component=ttt/re, uri=, contentType=application/x-vocalizer-rettt+text, lhError=88602000 Tested with Xcode Version 14.0.1 (14A400) on iPhone 14 Pro device ( iOS 16.0.3 ). More, in case Address Sanitizer is active app crashes: ERROR: AddressSanitizer: heap-buffer-overflow on address 0x000103e5ae75 at pc 0x000100ada758 bp 0x00016ff6e030 sp 0x00016ff6d7e8 READ of size 3062 at 0x000103e5ae75 thread T11 #0 0x100ada754 in wrap_strlen+0x164 (/private/var/containers/Bundle/Application/F72885E1-54FA-4FB9-B32E-320C8A8770A2/TestTTS.app/Frameworks/libclang_rt.asan_ios_dynamic.dylib:arm64e+0x16754)  #1 0x235339d78 in IsThisUrlOrRealPath+0x30 (/System/Library/PrivateFrameworks/TextToSpeechMauiSupport.framework/TextToSpeechMauiSupport:arm64e+0x2fd78) #2 0x235775994 in ve_ttsResourceLoad+0x16c (/System/Library/PrivateFrameworks/TextToSpeechMauiSupport.framework/TextToSpeechMauiSupport:arm64e+0x46b994)
4
1
1.4k
Oct ’22
CoreMediaErrorDomain Code=-12884
I have an AVPlayerViewController in my app playing video and my largest source of errors is NSURLErrorDomain Code=-1008. The underlying error they provide is Error Domain=CoreMediaErrorDomain Code=-12884 "(null)". I couldn't find what the error implies or is caused by - osstatus.com also does not have a reference to it.
1
0
1.4k
Nov ’22
ShazamKit SDK crash on Android
We are using ShazamKit SDK for Android and our application sometimes crashes when performing an audio recognition. We get the following logs: Cause: null pointer dereference backtrace: #00 pc 000000000000806c /data/app/lib/arm64/libsigx.so (SHAZAM_SIGX::reset()) (BuildId: 40e0b3c4250b21f23f7c4ec7d7b88f954606d914) #01 pc 00000000000dc324 /data/app//oat/arm64/base.odex at libsigx.SHAZAM_SIGX::reset()(reset:0) at base.0xdc324(Native Method)
1
2
1.3k
Nov ’22
AVPlayer 'Auto(Recommended)' option for subtitles
I use HLS to play video in AVPlayer. My .mp3u8 file has following content #EXTM3U #EXT-X-VERSION:3 #EXT-X-MEDIA:TYPE=SUBTITLES, DEFAULT=YES,GROUP-ID="subs",NAME="English",LANGUAGE="eng",URI="some/path" .... There is 3 options for subtitles "Off", "Auto (Recommended)", "English" on the screen. By default "Auto" option is chosen in the player, but there is no English subtitles are displayed (phone language is English).I have tried set FORCED attribute to YES but it does not help(subtitle icon become hidden). I have 2 questions: Is it possible to display subtitles when user selects "Auto option"? What criteria is used to determine subtitles language or subtitles visibility?
2
0
832
Jan ’23
AVRoutePickerView displays incorrect output names
Hi Folks, I'm currently working on video conferencing app and use AVRoutePickerView for output device selection. Since iOS 16 release it started to display incorrect names for ear-piece and speaker options. For both of them the name is iPhone (before it was iPhone/Speaker) Output changes works correctly but display names confuse. Here is my audio session configuration:     private func configureSession() {         let configuration = RTCAudioSessionConfiguration.webRTC()         configuration.category = AVAudioSession.Category.playAndRecord.rawValue         configuration.categoryOptions = [.allowBluetooth, .allowBluetoothA2DP, .duckOthers, .mixWithOthers]         let session = RTCAudioSession.sharedInstance()         session.lockForConfiguration()         do {             try session.setConfiguration(configuration)         } catch let error as NSError {             logError("[AudioSessionManager] Unable to configure RTCAudioSession with error: \(error.localizedDescription)")         }         session.unlockForConfiguration()     }          private func overrideOutputPortIfNeeded() {         DispatchQueue.main.async {             guard let currentOutputType = self.session.currentRoute.outputs.first?.portType else { return }                      self.session.lockForConfiguration()             let shouldOverride = [.builtInReceiver, .builtInSpeaker].contains(currentOutputType)             logDebug("[AudioSessionManager] Should override output to speaker? \(shouldOverride)")             if shouldOverride {                 do {                     try self.session.overrideOutputAudioPort(.speaker)                 } catch let error as NSError {                     logError("[AudioSessionManager] Unable to override output to Speaker: \(error.localizedDescription)")                 }             }             self.session.unlockForConfiguration()         }     } Any help appreciated, Thansk!
3
3
1.9k
Jan ’23
Saving plugin presets using fullState in Audio Units
Hi, I'm having trouble saving user presets in the plugin for Audio Units. This works well for saving the user presets in the Host, but I get an error when trying to save them in the plugin. I'm not using a parameter tree, but instead using the fullState's getter and setter for saving and retrieving a dictionary with the state. With some simplified parameters it looks something like this: var gain: Double = 0.0 var frequency: Double = 440.0     private var currentState: [String: Any] = [:] override var fullState: [String: Any]? {     get {       // Save the current state       currentState["gain"] = gain       currentState["frequency"] = frequency       // Return the preset state       return ["myPresetKey": currentState]     }     set {       // Extract the preset state       currentState = newValue?["myPresetKey"] as? [String: Any] ?? [:]       // Set the Audio Unit's properties       gain = currentState["gain"] as? Double ?? 0.0       frequency = currentState["frequency"] as? Double ?? 440.0     }  } This works perfectly well for storing user presets when saved in the host. When trying to save them in the plugin to be able to reuse them across hosts, I get the following error in the interface: "Missing key in preset state map". Note that I am testing mostly in AUM. I could not find any documentation for what the missing key is about and how can I get around this. Any ideas?
2
1
978
Mar ’23
Why does this MP4 video not play on iOS?
Open with the document App, only the file name and file type will be displayed, and the general video can be played directly. So, I switched to AVplayer to play this video with only sound but no picture. mp4info: File: major brand: mp42 minor version: 0 compatible brand: mp42 compatible brand: mp41 fast start: yes Movie: duration: 18842880 (media timescale units) duration: 209365 (ms) time scale: 90000 fragments: no Found 2 Tracks Track 1: flags: 1 ENABLED id: 1 type: Video duration: 209320 ms language: eng media: sample count: 5233 timescale: 25000 duration: 5233000 (media timescale units) duration: 209320 (ms) bitrate (computed): 9592.363 Kbps display width: 1920.000000 display height: 1080.000000 frame rate (computed): 25.000 Sample Description 0 Coding: avc1 (H.264) Codec String: avc1.640029 AVC Profile: 100 (High) AVC Profile Compat: 0 AVC Level: 41 AVC NALU Length Size: 4 AVC SPS: [67640029ac2ca501e0111f7350101014000003000400000300ca10] AVC PPS: [68e881135250] Width: 1920 Height: 1080 Depth: 24 Track 2: flags: 1 ENABLED id: 2 type: Audio duration: 209320 ms language: eng media: sample count: 9814 timescale: 48000 duration: 10049536 (media timescale units) duration: 209365 (ms) bitrate (computed): 317.376 Kbps Sample Description 0 Coding: mp4a (MPEG-4 Audio) Codec String: mp4a.40.2 Stream Type: Audio Object Type: MPEG-4 Audio Max Bitrate: 407250 Avg Bitrate: 317375 Buffer Size: 1536 MPEG-4 Audio Object Type: 2 (AAC Low Complexity) MPEG-4 Audio Decoder Config: Sampling Frequency: 48000 Channels: 2 Sample Rate: 48000 Sample Size: 16 Channels: 2 I used the video conversion tool to convert it to fit the iPhone: File: major brand: isom minor version: 200 compatible brand: isom compatible brand: iso2 compatible brand: avc1 compatible brand: mp41 fast start: yes Movie: duration: 209320 (media timescale units) duration: 209320 (ms) time scale: 1000 fragments: no Found 2 Tracks Track 1: flags: 3 ENABLED IN-MOVIE id: 1 type: Video duration: 209320 ms language: eng media: sample count: 5233 timescale: 12800 duration: 2679296 (media timescale units) duration: 209320 (ms) bitrate (computed): 11996.339 Kbps display width: 1920.000000 display height: 1080.000000 frame rate (computed): 25.000 Sample Description 0 Coding: avc1 (H.264) Codec String: avc1.640033 AVC Profile: 100 (High) AVC Profile Compat: 0 AVC Level: 51 AVC NALU Length Size: 4 AVC SPS: [27640033ac1316c0780227e5c05a8080809818002ee0000bb82f7be0f8442370] AVC PPS: [28ee1f2c] Width: 1920 Height: 1080 Depth: 24 Track 2: flags: 3 ENABLED IN-MOVIE id: 2 type: Audio duration: 209320 ms language: eng media: sample count: 9812 timescale: 48000 duration: 10047360 (media timescale units) duration: 209320 (ms) bitrate (computed): 317.379 Kbps Sample Description 0 Coding: mp4a (MPEG-4 Audio) Codec String: mp4a.40.2 Stream Type: Audio Object Type: MPEG-4 Audio Max Bitrate: 317379 Avg Bitrate: 317379 Buffer Size: 0 MPEG-4 Audio Object Type: 2 (AAC Low Complexity) MPEG-4 Audio Decoder Config: Sampling Frequency: 48000 Channels: 2 Sample Rate: 48000 Sample Size: 16 Channels: 2 The AVC Level and bitrate both higher than the first video, but it plays normally。 So why can't the first video play? I want to know which MP4 videos the iPhone can't play. Apple doesn't have specific specs, but says it supports MP4 and H.264
1
1
1.1k
Apr ’23
v3 AudioUnit, itself an audio unit host cannot see v3 audio units when hosted "out of process"
Back at the start of January this year we filed a bug report that still hasn't been acknowledged. Before we file a code level support ticket, we would love to know if there’s any one else out there who has experienced anything similar. We have read documentation (and again repeatedly) and searched the web over and still found no solution and this issue does look like it could be a bug in the system (or our coding) rather than proper behaviour. The app is a host for a v3 audio unit which itself is a workstation that can host other audio units. The host and audio unit are both working well in all other areas other than this issue. Note: This is not running on catalyst, it is a native Mac OS app. ( not iOS ) The problem is that when an AUv3 is hosted out of process (on the Mac) and then goes to fetch a list of all the other available installed audio units, the list that is returned from the system does not contain any of the other v3 audio units in the system. It only contains v2. We see this issue if we load our AU out of process in our own bespoke host, and also when it loads into Logic Pro which gives no choice but to load out of process. This means that, as it stands at the moment, when we release the app our users will have limited functionality in Logic Pro, and possibly by then, other hosts too. In our own host we can load our hosting AU in-process and then it can find and use all the available units both v2 and v3. So no issue there but sadly when loaded into the only other host that can do anything like this ( Logic Pro at the time of posting) it won't be able to use v3 units which is quite a serious limitation. SUMMARY v3 Audio Unit being hosted out of process. Audio unit fetches the list of available audio units on the system. v3 audio units are not provided in the list. Only v2 are presented. EXPECTED In some ways this seems to be the opposite of the the behaviour we would expect. We would expect to see and host ALL the other units that are installed on the system. “out of process” suggests the safer of the two options and so this looks like it could be related to some kind of sand boxing issue. But sadly we cannot work out a solution hence this report. Is Quin “The Eskimo” still around? Can you help us out here?
3
0
1.2k
Apr ’23
Black frame blinks when calling replaceCurrentItem
I'm building AVPlayerItem from AVComposition and then set to it AVVideoComposition with custom instructions and customVideoCompositorClass. The problem arise when i call player.replaceCurrentItem(with: ). Player view sometimes blinks with black frame. Is there any way to avoid that, for example show previous cached frame instead of black screen? P.S. sometimes it is enough to change fields of currentItem of AVPlayer, no issue here.
2
1
564
Apr ’23
AVSpeechSynthesizer write method get audio buffer and play audio deosn't work
I am using AVSpeechSynthesizer to get audio buffer and play, I am using AVAudioEngine and AVAudioPlayerNode to play the buffer. But I am getting error. [avae] AVAEInternal.h:76 required condition is false: [AVAudioPlayerNode.mm:734:ScheduleBuffer: (_outputFormat.channelCount == buffer.format.channelCount)] 2023-05-02 03:14:35.709020-0700 AudioPlayer[12525:308940] *** Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'required condition is false: _outputFormat.channelCount == buffer.format.channelCount' Can anyone please help me to play the AVAudioBuffer from AVSpeechSynthesizer write method?
2
1
1.2k
May ’23
func channelManager(_ channelManager: PTChannelManager, didActivate audioSession: AVAudioSession) not called in background
I tried to press system Talk button to talk in channel when my APP in background, I see the channelManager(_ : PTChannelManager, _: UUID, didBeginTransmittingFrom _: PTChannelTransmitRequestSource) is called, but the function func channelManager(_ : PTChannelManager, didActivate _: AVAudioSession) never called. When my APP in foreground, everything works fine. I've set the APP's capability Background Modes -> Push to Talk. Anyone can help?
3
0
883
May ’23
Suggested guidance for creating MV-HEVC video files?
After taking a look at the Deliver Video Content for Spatial Experiences session, alongside the Destination Video sample code, I'm a bit unclear on how one might go about creating stereoscopic content that can be bundled up as a MV-HEVC file and played on Vision Pro. I see the ISO Base Media File Format and Apple HEVC Stereo Video format specifications, alongside new mvhevc1440x1440 output presets in AVFoundation, but I'm unclear what sort of camera equipment could be used to create stereoscopic content and how one might be able to create a MV-HEVC using a command-line tool that leverages AVFoundation/VideoToolbox, or something like Final Cut Pro. Is such guidance available on how to film and create this type of file? Thanks!
6
8
4.0k
Jun ’23
IOS obtains microphone audio input data (PCM) and calculates DBSPL, resulting in a significant difference from the actual value
Using the OC framework AVFAudio, obtain PCM data for the 32-bit microphone of the device, and the results obtained are all between -1 and 1 (the actual value of the fixed noise source has exceeded 100 decibels). According to the conversion results, it cannot exceed 100 decibels (20 * log10 (PCM/0.00002)). Do you know what the reason or problem is? Output: -0.82569194 -0.82774025 -0.83398014 -0.87197787 -0.90468484 -0.9037836 0.9085202
0
0
506
Jul ’23
MacOS echo cancellation (AUVoiceProcessing) trouble with device gain/volume
Hi community I'm developing an application for MacOS and i need to capture the mic audio stream. Currently using CoreAudio in Swift i'm able to capture the audio stream using IO Procs and have applied the AUVoiceProcessing for prevent echo from speaker device. I was able to connect the audio unit and perform the echo cancellation. The problem that i'm getting is that when i'm using AUVoiceProcessing the gain of the two devices get reduced and that affects the volume of the two devices (microphone and speaker). I have tried to disable the AGC using the property kAUVoiceIOProperty_VoiceProcessingEnableAGCbut the results are the same. There is any option to disable the gain reduction or there is a better approach to get the echo cancellation working?
3
0
1.5k
Jul ’23
AudioQueue player mix with other sounds has a 0.5s mute
I was play a pcm(24kHz 16bit) file with AudioQueue, and it mix with other sound( 192kHz 24bit) named sound2. Setting for AVAudioSession as: category (AVAudioSessionCategoryPlayback), options (AVAudioSessionCategoryOptionMixWithOthers|AVAudioSessionCategoryOptionDuckOthers) when playing pcm the sound2 should pushed volume lower as setting. BUT, there has a absolutly mute keep 0.5 second when the sound2 become low, and after a 0.5s mute the pushed lower sound came out. It only become in sound2 situation(192kHz, 24bit). if the sound2's quality lower everything is OK.
1
1
1.1k
Jul ’23
enum in NullAudio.c
I like to know NullAudio.c is official SDK sample or not. And the reason of enum and UID is defined in NullAudio.c, not defined in SDK header files. I try to use kObjectID_Mute_Output_Master, but it defined different values on each 3rd party plugin. kObjectID_Mute_Output_Master = 10 // NullAudio.c kObjectID_Mute_Output_Master = 9 // https://github.com/ExistentialAudio/BlackHole kObjectID_Mute_Output_Master = 6 // https://github.com/q-p/SoundPusher I can build BlackHole and SoundPusher, these plugin worked. This enum should be defined SDK header and keep same value on each SDK version. I like to know why 3rd party defined different value. If you know the history of NullAudio.c, please let me know.
0
0
726
Jul ’23