Hi! How would you synchronize bpm, pitch and playhead position on 10-20 different devices*, all on the same closed ethernet network? *Mac, iPad and iPhone. A single device is master. Required latency tolerance in sub millisecond range, ideally sample sync. All the devices will play up to 64 channels of audio each. 48khz 24bit wav. I have considered two strategies: Broadcast all user events from master and replicate them on the slaves. Broadcast a continuous stream from master, comparing it on the slaves and slightly increasing / decreasing the corresponding parameter. I have the feeling there are some better options out there as these are neither fail safe nor very accurate. I have looked into the Ableton Link SDK, but it does not support position sync (only beat sync). All the best.

I’m using AVAudioEngine to get a stream of AVAudioPCMBuffers from the device’s microphone using the usual installTap(onBus:) setup. To distribute the audio stream to other parts of the program, I’m sending the buffers to a Combine publisher similar to the following: private let publisher = PassthroughSubject<AVAudioPCMBuffer, Never>() I’m starting to suspect I have some kind of concurrency or memory management issue with the buffers, because when consuming the buffers elsewhere I’m getting a range of crashes that suggest some internal pointer in a buffer is NULL (specifically, I’m seeing crashes in vDSP.convertElements(of:to:) when I try to read samples from the buffer). These crashes are in production and fairly rare — I can’t reproduce them locally. I never modify the audio buffers, only read them for analysis. My question is: should it be possible to put AVAudioPCMBuffers into a Combine pipeline? Does the AVAudioPCMBuffer class not retain/release the underlying AudioBufferList’s memory the way I’m assuming? Is this a fundamentally flawed approach?

I am simply trying to capture the screen, app audio and mic audio. The app audio and mic audio independently work fine but when combined some unknown error is thrown. Following are the methods to start the capture of screen and processSampleBuffer. Method to setup writers for screen capture func startCapture() { _filename = UUID().uuidString let videoPath = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first!.appendingPathComponent("\(_filename).mp4") let writer = try! AVAssetWriter(outputURL: videoPath, fileType: .mp4) let screen = UIScreen.main.bounds let screenBounds = screen.size let videoCompressionPropertys = [ AVVideoAverageBitRateKey: screenBounds.width * screenBounds.height * 10.1 ] let videoSettings: [String: Any] = [ AVVideoCodecKey: AVVideoCodecType.h264, AVVideoWidthKey: screenBounds.width, AVVideoHeightKey: screenBounds.height, AVVideoCompressionPropertiesKey: videoCompressionPropertys ] let input = AVAssetWriterInput(mediaType: .video, outputSettings: videoSettings) input.expectsMediaDataInRealTime = true if writer.canAdd(input) { writer.add(input) } // Add the app audio input var acl = AudioChannelLayout() memset(&acl, 0, MemoryLayout<AudioChannelLayout>.size) acl.mChannelLayoutTag = kAudioChannelLayoutTag_Mono; let audioOutputSettings: [String: Any] = [ AVFormatIDKey: kAudioFormatMPEG4AAC, AVSampleRateKey : 44100, AVNumberOfChannelsKey : 1, AVEncoderBitRateKey : 128000, AVChannelLayoutKey : Data(bytes: &acl, count: MemoryLayout<AudioChannelLayout>.size)] let audioInput = AVAssetWriterInput(mediaType: AVMediaType.audio, outputSettings: audioOutputSettings) audioInput.expectsMediaDataInRealTime = true if (writer.canAdd(audioInput)) { writer.add(audioInput) } // Add the mic audio input let audioOutputSettings1: [String: Any] = [ AVFormatIDKey: kAudioFormatMPEG4AAC, AVSampleRateKey : 24000, AVNumberOfChannelsKey : 1, AVEncoderAudioQualityKey: AVAudioQuality.high.rawValue] let micAudioInput = AVAssetWriterInput(mediaType: AVMediaType.audio, outputSettings: audioOutputSettings1) micAudioInput.expectsMediaDataInRealTime = true if (writer.canAdd(micAudioInput)) { writer.add(micAudioInput) } writer.startWriting() _audioAssetWriterInput = audioInput _micAssetWriterInput = micAudioInput _assetWriterInput = input _assetWriter = writer } processSampleBuffer override func processSampleBuffer(_ sampleBuffer: CMSampleBuffer, with sampleBufferType: RPSampleBufferType) { if startTime == nil { startTime = CMSampleBufferGetPresentationTimeStamp(sampleBuffer) _assetWriter!.startSession(atSourceTime: CMTime.zero) } if sampleBufferType == RPSampleBufferType.video { if let _assetWriterInput = _assetWriterInput { if _assetWriterInput.isReadyForMoreMediaData { let appended = _assetWriterInput.append(sampleBuffer) print(appended) if !appended { let status = _assetWriter?.status let error = _assetWriter?.error print("cannot append video") } } } } if sampleBufferType == RPSampleBufferType.audioApp { if let _assetWriterInput = _audioAssetWriterInput { if _assetWriterInput.isReadyForMoreMediaData { let appended = _assetWriterInput.append(sampleBuffer) print(appended) if !appended { let status = _assetWriter?.status let error = _assetWriter?.error print("cannot append app audio") } } } } if sampleBufferType == RPSampleBufferType.audioMic { if let _assetWriterInput = _micAssetWriterInput { if _assetWriterInput.isReadyForMoreMediaData { let appended = _assetWriterInput.append(sampleBuffer) print(appended) if !appended { let status = _assetWriter?.status let error = _assetWriter?.error print("cannot append mic audio") } } } } if shouldEnd { _finishWriters() } }

Hi, Wondering if anyone has found a solution to the automatic volume reduction on the host computer using the OSX native screen share application. The volume reduction makes it nearly impossible to comfortably continue working on the host computer when there is any audio involved. Is there a way to bypass to this function? It seems to be the same native function that FaceTime uses to reduce the system audio volume to create priority for the application. Please help save my speakers! Thanks.

I receive a buffer from[AVSpeechSynthesizer convertToBuffer:fromBuffer:] and want to schedule it on an AVPlayerNode. The player node's output format need to be something that the next node could handle and as far as I understand most nodes can handle a canonical format. The format provided by AVSpeechSynthesizer is not something thatAVAudioMixerNode supports. So the following:   AVAudioEngine *engine = [[AVAudioEngine alloc] init];   playerNode = [[AVAudioPlayerNode alloc] init];   AVAudioFormat *format = [[AVAudioFormat alloc] initWithSettings:utterance.voice.audioFileSettings];   [engine attachNode:self.playerNode];   [engine connect:self.playerNode to:engine.mainMixerNode format:format]; Throws an exception: Thread 1: "[[busArray objectAtIndexedSubscript:(NSUInteger)element] setFormat:format error:&nsErr]: returned false, error Error Domain=NSOSStatusErrorDomain Code=-10868 \"(null)\"" I am looking for a way to obtain the canonical format for the platform so that I can use AVAudioConverter to convert the buffer. Since different platforms have different canonical formats, I imagine there should be some library way of doing this. Otherwise each developer will have to redefine it for each platform the code will run on (OSX, iOS etc) and keep it updated when it changes. I could not find any constant or function which can make such format, ASDB or settings. The smartest way I could think of, which does not work:   AudioStreamBasicDescription toDesc;   FillOutASBDForLPCM(toDesc, [AVAudioSession sharedInstance].sampleRate,                      2, 16, 16, kAudioFormatFlagIsFloat, kAudioFormatFlagsNativeEndian);   AVAudioFormat *toFormat = [[AVAudioFormat alloc] initWithStreamDescription:&toDesc]; Even the provided example for iPhone, in the documentation linked above, uses kAudioFormatFlagsAudioUnitCanonical and AudioUnitSampleType which are deprecated. So what is the correct way to do this?

Hey I am trying to decode AMR_WB audio on iOS, for this I am using the below settings var asbd = AudioStreamBasicDescription() asbd.mSampleRate = Float64(sampleRate) asbd.mFormatID = kAudioFormatAMR_WB asbd.mFormatFlags = 0 asbd.mFramesPerPacket = 320 asbd.mChannelsPerFrame = UInt32(channels) asbd.mBitsPerChannel = 16 * UInt32(MemoryLayout<UInt8>.size) asbd.mReserved = 0 asbd.mBytesPerFrame = 2 asbd.mBytesPerPacket = asbd.mBytesPerFrame * asbd.mFramesPerPacket let _audioFormat = AVAudioFormat(streamDescription: &asbd)! return _audioFormat But I encounter the error as follows: Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedFailureReason=An unknown error occurred (1885696621), NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x283fb1920 {Error Domain=NSOSStatusErrorDomain Code=1885696621 "(null)" UserInfo={AVErrorFourCharCode='perm'}}} Now as per the documentation found here, it looks to be supported but I am unable what permission to give to the application for this to work. Any help will be appreciated.

Since we have to en/decode the audio stream to/from our audio device anyway and we are using NEON SIMD to do so, we could just convert it into a stream of float on the fly. Since floats are the natural CoreAudio data format we probably can avoid having to involve an additional int-float/float-int conversion by CoreAudio this way. Does this make sense? Thanks, hagen

How do you add a audio player to a swift playgrounds app project? How would you be able to add local audio along in the backend along with a interactive audio player in the frontend??? Thanks!

MacOS CoreAudio buffer playback produces annoying noise between correct sound. I'm interested to play valid .wav data though the buffer. Why I'm playing a .wav? It has valid data. What I'm trying to achieve is to understand how to write correctly to the sound buffer. I'm porting a music engine to MacOS .... #include <string.h> #include <math.h> #include <unistd.h> #include <stdio.h> #include <AudioToolbox/AudioToolbox.h> FILE *fp; typedef struct TwavHeader{ char RIFF[4]; uint32_t RIFFChunkSize; char WAVE[4]; char fmt[4]; uint32_t Subchunk1Size; uint16_t AudioFormat; uint16_t NumOfChan; uint32_t SamplesPerSec; uint32_t bytesPerSec; uint16_t blockAlign; uint16_t bitsPerSample; char Subchunk2ID[4]; uint32_t Subchunk2Size; }TwavHeader; typedef struct SoundState { bool done; }SoundState; void auCallback(void *inUserData, AudioQueueRef queue, AudioQueueBufferRef buffer) { buffer->mAudioDataByteSize = 1024*4; int numToRead = buffer->mAudioDataByteSize / sizeof(float) * 2; void *p = malloc(numToRead); fread(p, numToRead,1,fp); void *myBuf = buffer->mAudioData; for (int i=0; i < numToRead / 2; i++) { uint16_t w = *(uint16_t *)&(p[i*sizeof(uint16_t)]); float f = ((float)w / (float)0x8000) - 1.0; *(float *)&(myBuf[i*sizeof(float)]) = f; } free(p); AudioQueueEnqueueBuffer(queue, buffer, 0, 0); } void checkError(OSStatus error){ if (error != noErr) { printf("Error: %d", error); exit(error); } } int main(int argc, const char * argv[]) { printf("START\n"); TwavHeader theHeader; fp = fopen("/Users/kirillkranz/Documents/mytralala-code/CoreAudioTest/unreal.wav", "r"); fread(&theHeader, sizeof(TwavHeader),1,fp); printf("%i\n",theHeader.bitsPerSample); AudioStreamBasicDescription auDesc = {}; auDesc.mSampleRate = theHeader.SamplesPerSec; auDesc.mFormatID = kAudioFormatLinearPCM; auDesc.mFormatFlags = kLinearPCMFormatFlagIsFloat | kLinearPCMFormatFlagIsPacked; auDesc.mBytesPerPacket = 8; auDesc.mFramesPerPacket = 1; auDesc.mBytesPerFrame = 8; auDesc.mChannelsPerFrame = 2; auDesc.mBitsPerChannel = 32; AudioQueueRef auQueue = 0; AudioQueueBufferRef auBuffers[2] ={}; // our persistent state for sound playback SoundState soundState= {}; soundState.done=false; OSStatus err; // most of the 0 and nullptr params here are for compressed sound formats etc. err = AudioQueueNewOutput(&auDesc, &auCallback, &soundState, 0, 0, 0, &auQueue); checkError(err); // generate buffers holding at most 1/16th of a second of data uint32_t bufferSize = auDesc.mBytesPerFrame * (auDesc.mSampleRate / 16); err = AudioQueueAllocateBuffer(auQueue, bufferSize, &(auBuffers[0])); checkError(err); err = AudioQueueAllocateBuffer(auQueue, bufferSize, &(auBuffers[1])); checkError(err); // prime the buffers auCallback(&soundState, auQueue, auBuffers[0]); auCallback(&soundState, auQueue, auBuffers[1]); // enqueue for playing AudioQueueEnqueueBuffer(auQueue, auBuffers[0], 0, 0); AudioQueueEnqueueBuffer(auQueue, auBuffers[1], 0, 0); // go! AudioQueueStart(auQueue, 0); char rxChar[10]; scanf( "%s", &rxChar); printf("FINISH"); fclose(fp); // be nice even it doesn't really matter at this point if (auQueue) AudioQueueDispose(auQueue, true); } what do I do wrong?

var millisecondsSince1970: Int64 {     Int64((self.timeIntervalSince1970 * 1000.0).rounded())   }       init(milliseconds: Int64) {     self = Date(timeIntervalSince1970: TimeInterval(milliseconds) / 1000)   } } // Today in milliseconds Date().millisecondsSince1970 // 1641554695757 // The Date for 1 000 000 000 000 milliseconds print(Date(milliseconds: 1_000_000_000_000)) // 2001-09-09 01:46:40 +0000