I created a multitimbral sampler based on 16 instances of AVAudioUnitSampler (one per MIDI channel/part). It plays fine when receiving MIDI messages (or using the application virtual keyboard) on a single part. However I tried to play a Midifile using AVAudioSequencer (assiging each AVMusicTrack destination to the corresponding AVAudioUnitSampler instance). It uses 3 parts, one with a pad (samples up to 30s - 100MB total for the aupreset, many simultaneous notes - up to 6 - with many sustain messages), one with a bass sound (a single note at a time, 10MB size aupreset), and one with a lead sound (also one note at a time). Some notes are cut before end or do not play (mainly for the third part), as if there weren't resources left. I'm using the Simulator, and can't try anymore on my real iPad (won't boot since and will need repair or replacement). The Xcode monitoring tab shows only 2 to 3 percent processor used (and 60 MB memory used). However the Simulator runs on an old mac (mid-2010 mac mini - Core2Duo 2,4 Ghz). Is AVAudioUnitSampler suited to be using such way, or should I subclass AVAudioUnitMIDIInstrument (creating an audiounit with kAudioUnitSubTypeMIDISynth subtype as detailed in Gene De Lisa blog post, and loading a soundfont bank using kMusicDevicePropertySoundBankURL) ? Then the only way to change a part instrument would be to send a program change to the AVAudioUnitMIDIInstrument subclass ? I don't know how. Or should I used the kAudioUnitSubType_DLSSynth ?

I have a working AUv3 AUAudioUnit app extension but I had to work around a strange issue: I found that the internalRenderBlock value is fetched and invoked before allocateRenderResources() is called. I have not found any documentation stating that this would be the case, and intuitively it does not make any sense. Is there something I am doing in my code that would be causing this to be the case? Should I *force* a call to allocateRenderResources() if it has not been called before internalRenderBlock is fetched? Thanks! Brad

I am unable to play a private video from AWS S3 using AVPlayer. I have generated a presignedURL and trying to play a video in AVPLayer using that url. Can anyone help me regarding this. please help me if are there any better way to play a private videos of AWS S3 in iOS. Thanks & Regards, Narayanareddy

I'm trying to save h.264 frames that comes to the app from network into a .mov file. Using AVAssetWriter with AVAssetWriterInput appending CMSampleBuffer. Output settings are nil which means that frames will not be encoded during saving: videoWriter = AVAssetWriterInput(mediaType: .video, outputSettings: nil, sourceFormatHint: format) videoWriter.append(buffer) I have the code that saves frames to a file. Tried to run it in iOS app and in macOS app using same frames data iOS result video file plays correctly macOS result video file plays with artefacts I've inspected both result files with MediaInfo and all the data is equal except 1 line (bold line): Video ID : 1 Format : AVC Format/Info : Advanced Video Codec Format profile : High@L4 Format settings : CABAC / 1 Ref Frames Format settings, CABAC : Yes Format settings, Reference frames : 1 frame Format settings, GOP : M=1, N=15 Codec ID : avc1 Codec ID/Info : Advanced Video Coding Duration : 9 s 398 ms Source duration : 9 s 432 ms Bit rate mode : Variable Bit rate : 14.3 Mb/s Width : 1 920 pixels Height : 1 080 pixels Display aspect ratio : 16:9 Frame rate mode : Variable Frame rate : 30.000 FPS Minimum frame rate : 30.000 FPS Maximum frame rate : 31.579 FPS Color space : YUV Chroma subsampling : 4:2:0 Bit depth : 8 bits Scan type : Progressive Bits/(Pixel*Frame) : 0.231 Stream size : 16.1 MiB (100%) Source stream size : 16.1 MiB (100%) Title : Core Media Video Encoded date : UTC 2021-03-04 13:30:21 Tagged date : UTC 2021-03-04 13:30:23 Codec configuration box : avcC iOS result file has this line and macOS result file missing it: Format settings, GOP : M=1, N=15 Is there a way to provide some settings to AVAssetWriter/AVAssetWriterInput to have correct Group of Pictures in result file on macOS? The goal is to save raw H.264 stream to file without additional encoding.

The documentation - https://developer.apple.com/documentation/avfaudio/avaudiosession/capturing_stereo_audio_from_built-in_microphones for the Capturing Stereo Audio from Built-In Microphones sample app mentions: "Starting in iOS 14 and iPadOS 14, you can now capture stereo audio using the built-in microphones on supported devices." And I saw a specific list of devices that support stereo recording here. - https://developer.apple.com/forums/thread/652770 According to the list, an iPhone11 pro is supported. But when I run the sample app on an iPhone11 pro, an iPhone 12, or an iPhone12 mini, the supportedPolarPatterns property for the built-in mic data source does not include the .stereo option. Is there a way to enable it? Or am I using the wrong devices?

I just saw my activity monitor and noticed that there was something named core audio with intel architecture in my apple silicon mac which came pre loaded. Analysis of sampling com.apple.audio.Core-Audio-Driver-Service (pid 11196) every 1 millisecond Process: com.apple.audio.Core-Audio-Driver-Service [11196] Path: /System/Library/Frameworks/CoreAudio.framework/Versions/A/XPCServices/com.apple.audio.Core-Audio-Driver-Service.xpc/Contents/MacOS/com.apple.audio.Core-Audio-Driver-Service Load Address: 0x10023d000 Identifier: com.apple.audio.Core-Audio-Driver-Service Version: 1.0 (1) Code Type: X86-64 (translated) Platform: macOS Parent Process: launchd [1] Date/Time: 2021-03-31 01:30:39.233 +0530 Launch Time: 2021-03-31 01:20:44.850 +0530 OS Version: macOS 11.2.3 (20D91) Report Version: 7 Analysis Tool: /usr/bin/sample I wish I could get an answer for this why it is still in intel architecture and is not yet ported to apple silicon yet.

I Programing Use AVCaptureVideoDataOupput OS Version Big Sur 11.1 M1 - Error Intel - No Error M1 Error Log is "CMIO_Unit_Convertor_VideoToolboxDecompressor.cpp:949:rebuildVideoDecompressor [0x15690bc00] Helpers::VTUtilities::SetUpDecompressionOrTransferSession() returned error -12906!"

I have a USB audio interface that is causing kernel traps and the audio output to "skip" or dropout every few seconds. This behavior occurs with a completely fresh install of Catalina as well as Big Sur with the stock Music app on a 2019 MacBook Pro 16 (full specs below). The Console logs show coreaudiod got an error from a kernel trap, a "USB Sound assertion" in AppleUSBAudio/AppleUSBAudio-401.4/KEXT/AppleUSBAudioDevice.cpp at line 6644, and the Music app "skipping cycle due to overload." I've added a short snippet from Console logs around the time of the audio skip/drop out. The more complete logs are at this gist: https://gist.github.com/djflux/08d9007e2146884e6df1741770de5105 I've also opened a Feedback Assistant ticket (FB9037528): https://feedbackassistant.apple.com/feedback/9037528 Does anyone know what could be causing this issue? Thanks for any help. Cheers, Flux aka Andy. Hardware Overview:  Model Name: MacBook Pro  Model Identifier: MacBookPro16,1  Processor Name: 8-Core Intel Core i9  Processor Speed: 2.4 GHz  Number of Processors: 1  Total Number of Cores: 8  L2 Cache (per Core): 256 KB  L3 Cache: 16 MB  Hyper-Threading Technology: Enabled  Memory: 64 GB  System Firmware Version: 1554.80.3.0.0 (iBridge: 18.16.14347.0.0,0) System Software Overview: System Version: macOS 11.2.3 (20D91)  Kernel Version: Darwin 20.3.0  Boot Volume: Macintosh HD  Boot Mode: Normal  Computer Name: mycomputername  User Name: myusername  Secure Virtual Memory: Enabled  System Integrity Protection: Enabled USB interface: Denon DJ DS1 Snippet of Console logs error 21:07:04.848721-0500 coreaudiod HALS_IOA1Engine::EndWriting: got an error from the kernel trap, Error: 0xE00002D7 default 21:07:04.848855-0500 Music HALC_ProxyIOContext::IOWorkLoop: skipping cycle due to overload default 21:07:04.857903-0500 kernel USB Sound assertion (Resetting engine due to error returned in Read Handler) in /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/AppleUSBAudio/AppleUSBAudio-401.4/KEXT/AppleUSBAudioDevice.cpp at line 6644 ... default 21:07:05.102746-0500 coreaudiod Audio IO Overload inputs: 'private' outputs: 'private' cause: 'Unknown' prewarming: no recovering: no default 21:07:05.102926-0500 coreaudiod   CAReportingClient.mm:508  message {   HostApplicationDisplayID = "com.apple.Music";   cause = Unknown;   deadline = 2615019;   "input_device_source_list" = Unknown;   "input_device_transport_list" = USB;   "input_device_uid_list" = "AppleUSBAudioEngine:Denon DJ:DS1:000:1,2";   "io_buffer_size" = 512;   "io_cycle" = 1;   "is_prewarming" = 0;   "is_recovering" = 0;   "issue_type" = overload;   lateness = "-535";   "output_device_source_list" = Unknown;   "output_device_transport_list" = USB;   "output_device_uid_list" = "AppleUSBAudioEngine:Denon DJ:DS1:000:1,2"; }: (null)

The crash stack as following Thread 0#0 (null) in CFEqual () #1 (null) in CFCachedStringEqual () #2 (null) in CFCachedStringEqual () #3 (null) in _CFXNotificationRegistrarFindName () #4 (null) in CFXNotificationRegistrarFind () #5 (null) in _CFXNotificationPost () #6 (null) in -[NSNotificationCenter postNotificationName:object:userInfo:] () #7 (null) in -[UIScreen _postBrightnessDidChangeNotificationIfAppropriate] () #8 (null) in _UIScreenBacklightLevelDidChangeCallback () #9 (null) in __CFNOTIFICATIONCENTER_IS_CALLING_OUT_TO_AN_OBSERVER__ () #10 (null) in ___CFXRegistrationPost_block_invoke () #11 (null) in _dispatch_call_block_and_release () #12 (null) in _dispatch_client_callout () #13 (null) in _dispatch_main_queue_callback_4CF () #14 (null) in __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ () #15 (null) in __CFRunLoopRun () #16 (null) in CFRunLoopRunSpecific () #17 (null) in GSEventRunModal () #18 (null) in -[UIApplication _run] () #19 (null) in UIApplicationMain () #20 0x0000000102c88fa4 in main at AppDelegate.swift:19 #21 (null) in start () Thread 1 Thread 2 Thread 3 Thread 4 Thread 5 Thread 6 Thread 7 Thread 8 Thread 9 Thread 10 Thread 11 Thread 12 Thread 13 Thread 14 Thread 15 Thread 16 Thread 17 #0 (null) in mach_msg_trap () #1 (null) in mach_msg () #2 (null) in __CFRunLoopServiceMachPort () #3 (null) in __CFRunLoopRun () #4 (null) in CFRunLoopRunSpecific () #5 (null) in -[AVRunLoopCondition _waitInMode:untilDate:] () #6 (null) in -[AVCaptureSession _buildAndRunGraph] () #7 (null) in -[AVCaptureSession _setRunning:] () #8 0x00000001042ff4cc in __27-[CameraKit startCapture]_block_invoke at CameraKit #9 (null) in _dispatch_call_block_and_release () #10 (null) in _dispatch_client_callout () #11 (null) in _dispatch_lane_serial_drain () #12 (null) in _dispatch_lane_invoke () #13 (null) in _dispatch_workloop_worker_thread () #14 (null) in _pthread_wqthread () #15 (null) in start_wqthread () Thread 18 Thread 19 #0 (null) in OUTLINED_FUNCTION_3 () #1 (null) in dispatch_source_cancel.cold.1 () #2 (null) in __FigRPCDisposeServerConnection_block_invoke () #3 (null) in _dispatch_client_callout () #4 (null) in _dispatch_lane_barrier_sync_invoke_and_complete () #5 (null) in FigRPCDisposeServerConnection () #6 (null) in RemoteQueueClientFinalize () #7 (null) in FigBaseObjectFinalize () #8 (null) in _CFRelease () #9 (null) in rqReceiverFinalize () #10 (null) in _CFRelease () #11 (null) in __rqReceiverSetSource_block_invoke_2 () #12 (null) in _dispatch_call_block_and_release () #13 (null) in _dispatch_client_callout () #14 (null) in _dispatch_continuation_pop () #15 (null) in _dispatch_source_invoke () #16 (null) in _dispatch_lane_serial_drain () #17 (null) in _dispatch_lane_invoke () #18 (null) in _dispatch_workloop_worker_thread () #19 (null) in _pthread_wqthread () #20 (null) in start_wqthread () Thread 20 Thread 21 Thread 22 Thread 23 Thread 24 Thread 25 Thread 26

How can I processing on the AudioQueue callback AudioQueueOutputCallback for FFT . Just like this func static void audioQueueOutpuCallBack(void *input, AudioQueueRef inQueue, AudioQueueBufferRef outQueueBuffer) {    SYAudioQueue *aq = (__bridge SYAudioQueue *)input;   dispatch_semaphore_wait(aq-m_mutex, DISPATCH_TIME_FOREVER);   [aq enterQueue:inQueue withBuffer:outQueueBuffer];   dispatch_semaphore_signal(aq-m_mutex); } I know that AVAudioEngine can be processing for FFT under the AVAudioPCMBuffer . or How can I convert AudioQueueBufferRef to AVAudioPCMBuffer

My app is using RemoteIO to record audio. It doesn’t do any playback. RemoteIO seems to be broadly compatible with the new Sound Recognition feature in iOS14, but I’m seeing a glitch when sound recognition is first enabled. If my app is started and I initialise RemoteIO, and then turn on Sound Recognition (say via control centre), the RemoteIO input callback is not called thereafter, until I tear down the audio unit and set it back up. So something like the following: Launch app RemoteIO is initialised and working, can record Turn on Sound Recognition via Settings or control centre widget Start recording with already-set up RemoteIO Recording callback is never again called Though no input callbacks are seen, kAudioOutputUnitProperty_IsRunning is reported as true, so the audio unit thinks it is active Tear down audio unit Set up audio unit again Recording works Buffer size is changed, reflecting some effect on the audio session of the Sound Recognition feature I also noticed that when Sound Recognition is enabled, I see several (usually 3) AVAudioSession.routeChangeNotifications in quick succession. When Sound Recognition is disabled while RemoteIO is set up, I don’t see this problem. I’m allocating my own buffers so it’s not a problem with their size. What could be going on here? Am I not handling a route change properly? There doesn’t seem to be a reliable sequence of events I can catch to know when to reset the audio unit. The only fix I’ve found here is to hack in a timer that checks for callback activity shortly after starting recording, and resets the audio unit if no callback activity is seen. Better than nothing, but not super reliable.

Error Domain=AVFoundationErrorDomain Code=-11800 "这项操作无法完成" UserInfo={NSUnderlyingError=0x1102b4940 {Error Domain=NSOSStatusErrorDomain Code=-16415 "(null)"}, NSLocalizedFailureReason=发生未知错误（-16415）, AVErrorRecordingFailureDomainKey=4, NSLocalizedDescription=这项操作无法完成}

Hello, Our app is using avfoundation to add overlay and merge several video tracks and audio. we are using: AVMutableComposition AVMutableVideoCompositionInstruction AVMutableCompositionTrack AVMutableAudioMix AudioCompositor AVAssetExportSession.exportAsynchronously it works well until iOS 14.5. From iOS 14.5, there are more errors below (very frequently), sometime it crashes too. Error Domain=AVFoundationErrorDomain Code=-11819 "Cannot Complete Action" UserInfo={NSLocalizedRecoverySuggestion=Try again later., NSLocalizedDescription=Cannot Complete Action, NSUnderlyingError=0x283cf3d80 {Error Domain=NSOSStatusErrorDomain Code=-16978 "(null)"}} With our first investigations, we suppose that iOS 14.5 is more strict on memory consumption, but we cant' find any changelog about avfoundation or memory restriction on iOS 14.5 We are in trouble, because our app is completely inusable. Looking forward for your relays, thank you!

Hello Everyone, I am working on an app that will receive some audio files sent from a device over Bluetooth to an iPhone, and the iPhone should play those audio files. The device is currently using A2DP Bluetooth audio profile. Here, the device is the A2DP source and the iPhone should be the A2DP sink. During my research, I found out that Apple doesn't allow iPhone to act as an A2DP sink. My question is, which other Bluetooth audio profiles can be used to make the iPhone act as an audio sink or receiver? Thanks, PC

I want to record both IMU data and Audio data from Airpods Pro. I have tried many times, and I failed. I can successfully record the IMU data and iPhone's microphone data simultaneously. When I choose Airpods Pro's microphone in the setCategory() function, the IMU data collection process stopped. If I change recordingSession.setCategory(.playAndRecord, mode: .default, options: .allowBluetooth) to ecordingSession.setCategory(.playAndRecord, mode: .default), everything is okay except the audio is recorded from the handphone. If I add options: .allowBluetooth, the IMU update will stop. Could you give me some suggestions for this? Below are some parts of my code.   let My_IMU = CMHeadphoneMotionManager()   let My_writer = CSVWriter()   var write_state: Bool = false   func test()   {     recordingSession = AVAudioSession.sharedInstance()     do {       try recordingSession.setCategory(.playAndRecord, mode: .default, options: .allowBluetooth)       try recordingSession.setActive(true)       recordingSession.requestRecordPermission() { [unowned self] allowed in         DispatchQueue.main.async {           if allowed == false {print("failed to record!")}         }       }     } catch {       print("failed to record!")     }                 let audioFilename = getDocumentsDirectory().appendingPathComponent("test_motion_Audio.m4a")     let settings = [       AVFormatIDKey: Int(kAudioFormatMPEG4AAC),       AVSampleRateKey: 8000,       AVNumberOfChannelsKey: 1,       AVEncoderAudioQualityKey: AVAudioQuality.high.rawValue     ]     do     {       audioRecorder = try AVAudioRecorder(url: audioFilename, settings: settings)       audioRecorder.delegate = self       audioRecorder.record()     }     catch     {       print("Fail to record!")       finishRecording()     }           write_state.toggle()     let dir = FileManager.default.urls(      for: .documentDirectory,      in: .userDomainMask     ).first!           let filename = "test_motion_Audio.csv"     let fileUrl = dir.appendingPathComponent(filename)     My_writer.open(fileUrl)           APP.startDeviceMotionUpdates(to: OperationQueue.current!, withHandler: {[weak self] motion, error in       guard let motion = motion, error == nil else { return }       self?.My_writer.write(motion)     })   }

I've got a fairly old app that I first built back in 2014, and have been maintaining over the years. One of the screens in the app uses the camera to scan barcodes. The barcodes we scan are printed on labels, and are in the org.iso.Code128 format. When the app was first developed, I simply set the metadataObjectTypes property of AVCaptureMetadataOutput to all available types. This worked fine. The app scanned our barcodes very quickly with almost no issues. In 2017, we started seeing issues where barcode scanning was becoming slower. I reasoned that having it configured to scan for every possible barcode format might be the issue, so I went in and changed the code to have it scan only for the org.iso.Code128 format. This helped, for a while. Now, we're seeing the problem again in iOS 14 devices. On some devices it is nearly impossible to scan the barcodes. On others, you have to wait 20 or 30 seconds before the barcode is recognized. What could be causing this issue?

Hi, I am interested in decoding multichannel Higher-Order Ambisonics feeds to the Spatial Audio renderer discussed in the "Immerse your app in spatial audio" WWDC21 talk. However, I could not find any documentation about which multichannel audio formats are actually supported by the renderer, and a search for "Ambisonics" in the developer documentation only contains results pertaining to the Audio DriverKit. Can someone please enlighten me? Thank you!

I want to create my own custom audio recognition with ShazamKit, when opening the sample project I found the FoodMath.shazamsignature file. I believe there is a way to generate that file based on my audio collections. How do I create the .shazamsignature file? Thanks.

Where can I find a comprehensive list of all the classes that the built in Sound Classifier model supports?

Would like to play with this example project shown on session wwdc21-10187