Hey, I am fairly new to working with AVFoundation etc. As far as I could research on my own, if I want to get metadata from let's say a .m4a audio file, I have to get the data and then create an AVAsset. My files are all on local servers and therefore I would not be able to just pass in the URL. The extraction of the metadata works fine - however those AVAssets create a huge overhead in storage consumption. To my knowledge the data instances of each audio file and AVAsset should only live inside the function I call to extract the metadata, however those data/AVAsset instances still live on on storage as I can clearly see that the app's file size increases by multiple Gigabytes (equal to the library size I test with). However, the only data that I purposefully save with SwiftData is the album artwork. Is this normal behavior for AVAssets or am I missing some detail? PS. If I forgot to mention something important, please ask. This is my first ever post, so I'm not too sure what is worth mentioning. Thank you in advance! Denis

The media services used for HLS streaming in an AVPlayer seem to crash if your segments are too large. Anything over 20Mbps seems to cause a crash. I have tried adjusting the segment length to 1 second also and it didn't help. I am remuxing Dolby Vision and HDR video and want to avoid transcoding and losing any metadata. However the segments are too large. Is there a workaround for this? Otherwise it seems AVFoundation is not suited to high bitrate HLS and I should be using MPV or similar.

I have spent the past 2 weeks diving into CoreAudio and have seemingly run into a wall... For my initial test, I am simply trying to create an AUGraph for monitoring input levels from a user chosen Audio Input Device (multi-channel in my case). I was not able to find any way to monitor input levels of a single AUHAL input device - so I decided to create a simple AUGraph for input level monitoring. Graph looks like: [AUHAL Input Device] -> [B1] -> [MatrixMixerAU] -> [B2] -> [AUHAL Output Device] Where B1 is an audio stream consisting of all the input channels available from the input device. The MatrixMixer has metering mode turned on, and level meters are read from the each submix of the MatrixMixer using kMatrixMixerParam_PostAveragePower. B2 is a stereo (2 channel) stream from the MatrixMixerAU to the default audio device - however, since I don't really want to pass audio through to an actual output, I have the volume muted on the MatrixMixerAU output channel. I tried using a GenericOutputAU instread of the default system output, however, the GenericOutputAU never seems to pull date from the ringBuffer (the graph renderProc is never called if a GenericOutputAU is used instead of AUHAL default output device). I have not been able to get this simple graph to work. I do not see any errors when creating the graph and initializing the graph, and I have verified that the inputProc is being called for filling up the ringBuffer - but when I read the level of the MatrixMixer, the levels are always -758 (silence). I have posted my demo project on github in hopes I can find someone with CoreAudio expertise to help with this problem. I am willing to move this to DTS Code Level support if there is someone in DTS with CoreAudio experience. Notes: My App is not sandboxed in this test I have tried with and without hardened runtime with Audio Input checked The multichannel audio device I am using for testing is the Audient iD14 USB-C audio device. It supports 12 input channels and 6 output channels. All input channels have been tested and are working in Ableton Live and Logic Pro. Of particular interest, is that I can't even get the Apple CAPlayThrough demo to work on my system. I see no errors when creating the graph, but all I hear is silence. The MatrixMixerTest from the Apple documentation archives does work - but note, that that demo does not use Audio Input devices - it reads audio into the graph from an audio file. Link to github project page. Diagram of AUGraph for initial test (code that is on github) Once I get audio input level metering to work, my plan is to implement something like in Phase 2 below - with the purpose of capturing a stereo input stream, mixing to mono, and sending to lowpass, bandpass, hipass AUs - and will again use MatrixMixer for level monitoring of the levels out of each filter. I have no plans on passthough audio (sending actual audio out to devices) - I am simple monitoring input levels Diagram of ultimate scope - rendering audio levels of a stereo to mono stream after passing through various filters

I created a locked camera capture extension as explained in Apple's documentation. I'm trying to explore the possibilities of using a bluetooth peripheral from that extension - anybody knows if this is possible? The CBCentralManagerDelegate reports .unsupported in func centralManagerDidUpdateState, even if I have provided all the permissions in Info.plist.

Hello, I have a problem reading a 2D data matrix type code with a camera. In the application, I use AVFoundation to operate the camera and work with 2D codes, and in the vast majority there is no problem with loading. Nothing special. I originally thought it might be a problem in my code, but I got the same result when I tried with the Camera app integrated in IOS. It can be seen that only the LiveText API for text recognition worked. But I am attaching the code with which the camera has a problem, even though the code looks perfectly fine at first glance. A classic handheld 2D code reader will read the code just fine. Can someone please explain to me why the camera, which normally reads these codes at the speed of light, sometimes has a problem with the codes? Thank you [Personal Information Edited by Moderator]

My Camera app is repeatedly opening even though I am not taking any action to open it when I use iPhone. Today during a FaceTime call, the Camera app opened while the phone was unlocked without me touching anything. It didn’t end the FaceTime call, but just put the video on pause for the person I was speaking with. I force closed the Camera app, then it happened again a few minutes later. This has happened while using Google Maps and other apps as well, while the phone is unlocked. This is also happening while the phone is locked, just sitting on a table. All the sudden I look over and the screen is active showing the camera view. Today this has happened at least 20 times. I need to know how to stop it. I am on iOS 18.1 and enrolled in iOS 18 Public Beta. There are no pending software updates.

There are different microphones that can be connected via a 3.5-inch jack or via USB or via Bluetooth, the behavior is the same. There is a code that gets access to the microphone (connected to the 3.5-inch audio jack) and starts an audio capture session. At the same time, the microphone use icon starts to be displayed. The capture of the audio device (microphone) continues for a few seconds, then the session stops, the microphone use icon disappears, then there is a pause of a few seconds, and then a second attempt is made to access the same microphone and start an audio capture session. At the same time, the microphone use icon is displayed again. After a few seconds, access to the microphone stops and the audio capture session stops, after which the microphone access icon disappears. Next, we will try to perform the same actions, but after the first stop of access to the microphone, we will try to pull the microphone plug out of the connector and insert it back before trying to start the second session. In this case, the second attempt to access begins, the running part of the program does not return errors, but the microphone access icon is not displayed, and this is the problem. After the program is completed and restarted, this icon is displayed again. This problem is only the tip of the iceberg, since it manifests itself in the fact that it is not possible to record sound from the audio microphone after reconnecting the microphone until the program is restarted. Is this normal behavior of the AVFoundation framework? Is it possible to somehow make it so that after reconnecting the microphone, access to it occurs correctly and the usage indicator is displayed? What additional actions should the programmer perform in this case? Is there a description of this behavior somewhere in the documentation? Below is the code to demonstrate the described behavior. I am also attaching an example of the microphone usage indicator icon. Computer description: MacBook Pro 13-inch 2020 Intel Core i7 macOS Sequoia 15.1. #include <chrono> #include <condition_variable> #include <iostream> #include <mutex> #include <thread> #include <AVFoundation/AVFoundation.h> #include <Foundation/NSString.h> #include <Foundation/NSURL.h> AVCaptureSession* m_captureSession = nullptr; AVCaptureDeviceInput* m_audioInput = nullptr; AVCaptureAudioDataOutput* m_audioOutput = nullptr; std::condition_variable conditionVariable; std::mutex mutex; bool responseToAccessRequestReceived = false; void receiveResponse() { std::lock_guard<std::mutex> lock(mutex); responseToAccessRequestReceived = true; conditionVariable.notify_one(); } void waitForResponse() { std::unique_lock<std::mutex> lock(mutex); conditionVariable.wait(lock, [] { return responseToAccessRequestReceived; }); } void requestPermissions() { responseToAccessRequestReceived = false; [AVCaptureDevice requestAccessForMediaType:AVMediaTypeAudio completionHandler:^(BOOL granted) { const auto status = [AVCaptureDevice authorizationStatusForMediaType:AVMediaTypeAudio]; std::cout << "Request completion handler granted: " << (int)granted << ", status: " << status << std::endl; receiveResponse(); }]; waitForResponse(); } void timer(int timeSec) { for (auto timeRemaining = timeSec; timeRemaining > 0; --timeRemaining) { std::cout << "Timer, remaining time: " << timeRemaining << "s" << std::endl; std::this_thread::sleep_for(std::chrono::seconds(1)); } } bool updateAudioInput() { [m_captureSession beginConfiguration]; if (m_audioOutput) { AVCaptureConnection *lastConnection = [m_audioOutput connectionWithMediaType:AVMediaTypeAudio]; [m_captureSession removeConnection:lastConnection]; } if (m_audioInput) { [m_captureSession removeInput:m_audioInput]; [m_audioInput release]; m_audioInput = nullptr; } AVCaptureDevice* audioInputDevice = [AVCaptureDevice deviceWithUniqueID: [NSString stringWithUTF8String: "BuiltInHeadphoneInputDevice"]]; if (!audioInputDevice) { std::cout << "Error input audio device creating" << std::endl; return false; } // m_audioInput = [AVCaptureDeviceInput deviceInputWithDevice:audioInputDevice error:nil]; // NSError *error = nil; NSError *error = [[NSError alloc] init]; m_audioInput = [AVCaptureDeviceInput deviceInputWithDevice:audioInputDevice error:&error]; if (error) { const auto code = [error code]; const auto domain = [error domain]; const char* domainC = domain ? [domain UTF8String] : nullptr; std::cout << code << " " << domainC << std::endl; } if (m_audioInput && [m_captureSession canAddInput:m_audioInput]) { [m_audioInput retain]; [m_captureSession addInput:m_audioInput]; } else { std::cout << "Failed to create audio device input" << std::endl; return false; } if (!m_audioOutput) { m_audioOutput = [[AVCaptureAudioDataOutput alloc] init]; if (m_audioOutput && [m_captureSession canAddOutput:m_audioOutput]) { [m_captureSession addOutput:m_audioOutput]; } else { std::cout << "Failed to add audio output" << std::endl; return false; } } [m_captureSession commitConfiguration]; return true; } void start() { std::cout << "Starting..." << std::endl; const bool updatingResult = updateAudioInput(); if (!updatingResult) { std::cout << "Error, while updating audio input" << std::endl; return; } [m_captureSession startRunning]; } void stop() { std::cout << "Stopping..." << std::endl; [m_captureSession stopRunning]; } int main() { requestPermissions(); m_captureSession = [[AVCaptureSession alloc] init]; start(); timer(5); stop(); timer(10); start(); timer(5); stop(); }

I am recording video on iOS using ReplayKit and found that after copying data in the processSampleBuffer:withType: callback using memcpy, the data changes. This occurs particularly frequently when the screen content changes rapidly, making it look like the frames are overlapping. I found that the values starting from byte 672 in the video data on my device often change. Here is the test demo: - (void)processSampleBuffer:(CMSampleBufferRef)sampleBuffer withType:(RPSampleBufferType)sampleBufferType { switch (sampleBufferType) { case RPSampleBufferTypeVideo: { CVPixelBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer); CVPixelBufferLockBaseAddress(pixelBuffer, kCVPixelBufferLock_ReadOnly); int ret = 0; uint8_t *oYData = CVPixelBufferGetBaseAddressOfPlane(pixelBuffer, 0); size_t oYSize = CVPixelBufferGetHeightOfPlane(pixelBuffer, 0) * CVPixelBufferGetBytesPerRowOfPlane(pixelBuffer, 0); uint8_t *oUVData = CVPixelBufferGetBaseAddressOfPlane(pixelBuffer, 1); size_t oUVSize = CVPixelBufferGetHeightOfPlane(pixelBuffer, 1) * CVPixelBufferGetBytesPerRowOfPlane(pixelBuffer, 1); if (oYSize <= 672) { return; } uint8_t tempValue = oYData[672]; uint8_t *tYData = malloc(oYSize); memcpy(tYData, oYData, oYSize); if (tYData[672] != oYData[672]) { NSLog(@"$$$$$$$$$$$$$$$$------ t:%d o:%d temp:%d", tYData[672], oYData[672], tempValue); } free(tYData); CVPixelBufferUnlockBaseAddress(pixelBuffer, kCVPixelBufferLock_ReadOnly); break; } default: { break; } } } Output: $$$$$$$$$$$$$$$$------ t:110 o:124 temp:110 $$$$$$$$$$$$$$$$------ t:111 o:133 temp:111 $$$$$$$$$$$$$$$$------ t:124 o:138 temp:124 $$$$$$$$$$$$$$$$------ t:133 o:144 temp:133 $$$$$$$$$$$$$$$$------ t:138 o:151 temp:138 $$$$$$$$$$$$$$$$------ t:144 o:156 temp:144 $$$$$$$$$$$$$$$$------ t:151 o:135 temp:151 $$$$$$$$$$$$$$$$------ t:156 o:78 temp:156 $$$$$$$$$$$$$$$$------ t:135 o:76 temp:135 $$$$$$$$$$$$$$$$------ t:78 o:77 temp:78 $$$$$$$$$$$$$$$$------ t:76 o:80 temp:76 $$$$$$$$$$$$$$$$------ t:77 o:80 temp:77 $$$$$$$$$$$$$$$$------ t:80 o:79 temp:80 $$$$$$$$$$$$$$$$------ t:79 o:80 temp:79

Hello! I'm trying to create a headphone safety prototype to give warnings if I listen to music too loud, but inputing my headphone's impedance, sensitivity, and wanted SPL level, and all I need is just the data on the amount of power each percentage of volume outputs(I'm assuming the MacBook Pro has 1-100% volume scale). If anyone has this info, or can direct me to someone who has this info, that would be great! Also do I contact apple support for things like this? I'm not too sure... Thanks!!

I’m using AVFoundation in my iPhone application to encode a video in MP4 format with H.264, which can then be shared or exported. Do I need to pay a license for using the H.264 format to MPEG LA? Or are these fees already covered by Apple? I’ve read articles suggesting that Apple covers these fees when encoding is done through its native APIs (or via its dedicated encoding hardware components), but I haven’t found any explicit confirmation of this point in the various documentation or contracts... Did I miss something?

I am building an app for MacOS and I am trying to implement the code to add songs to a library playlist (which is added below). The issue I am having is that if I use Music Kit to load a users library playlists, the ID for the playlist (which is just a string of numbers) does not work with the Add tracks to a Library Playlist endpoint of Apple Music API. If I retrieve the playlists from the Apple Music API and use that playlist ID (which is different than the id I get from MusicKit) my code works fine and adds the song to the playlist. The problem is that when getting a users library playlists from Apple Music API is that it does not give me all of the library playlists that I get when using Music Kit and it also does not give me Artwork for playlists that have the collage of album covers, so I would prefer to use Music Kit to get the playlists. I have also tested trying to retrieve a single playlist using the Apple Music API with the playlist Id from Music Kit and it does not work. I get the error that the resource cannot be found. Since this is a macOs app I cannot use MusicKit to add songs to library playlists. Does anyone know a way to resolve this? Or a possible workaround? Ideally I want to use MusicKit to get the library playlists and have some way to use the playlist Id and add songs to that playlist. Below is my code for adding a song to a playlist using the Apple Music API, which works correctly only if I originally get the library playlist's id value from a playlist retrieved from the Apple Music API. Also, does anyone know why the playlist Id's are not universal and are different when using Music Kit and Apple Music API? For songs and tracks it does not seem to matter if I use music kit or Apple Music API, the Id's are in the correct format for Apple Music API to use and work with my code. Thanks everyone for any and all help! func addToPlaylist(songs: [Track], playlist: Playlist, alert: Binding<AlertItem?>) async { let tracks = AppleMusicPlaylistPostRequestBody(data: songs.compactMap { AppleMusicPlaylistPostRequestItem(id: $0.id.rawValue, type: "songs") // or "library-songs" }) let playlistID = playlist.id // Build the request URL for adding a song to a playlist guard let url = URL(string: "https://api.music.apple.com/v1/me/library/playlists/\(playlistID)/tracks") else { alert.wrappedValue = AlertItem(title: "Error", message: "Invalid URL for the playlist.") return } // Authorization Header guard let musicUserToken = try? await MusicUserTokenProvider().getUserMusicToken() else { alert.wrappedValue = AlertItem(title: "Error", message: "Unable to retrieve Music User Token.") return } do { var request = URLRequest(url: url) request.httpMethod = "POST" request.setValue("Bearer \(musicUserToken)", forHTTPHeaderField: "Authorization") request.setValue("application/json", forHTTPHeaderField: "Content-Type") let encoder = JSONEncoder() let data = try encoder.encode(tracks) request.httpBody = data let musicRequest = MusicDataRequest(urlRequest: request) let musicRequestResponse = try await musicRequest.response() // Check if the request was successful (status 201) if musicRequestResponse.urlResponse.statusCode == 201 { alert.wrappedValue = AlertItem(title: "Success", message: "Song successfully added to the playlist.") } else { print("Status Code: \(musicRequestResponse.urlResponse.statusCode)") print("Response Data: \(String(data: musicRequestResponse.data, encoding: .utf8) ?? "No Data")") // Attempt to decode the error response into the AppleMusicErrorResponse model if let appleMusicError = try? JSONDecoder().decode(AppleMusicErrorResponse.self, from: musicRequestResponse.data) { let errorMessage = appleMusicError.errors.first?.detail ?? "Unknown error occurred." alert.wrappedValue = AlertItem(title: "Error", message: errorMessage) } else { alert.wrappedValue = AlertItem(title: "Error", message: "Failed to add song to the playlist.") } } } catch { alert.wrappedValue = AlertItem(title: "Error", message: "Network error: \(error.localizedDescription)") } }

To get calibration data during video recording, I use AVCaptureDepthDataOutput together with AVCaptureVideoDataOutput, synchronized via AVCaptureDataOutputSynchronizer (using the dataOutputSynchronizer method of CameraController in the example). The issue is that AVCaptureDepthDataOutput can only be used with .builtInLiDARDepthCamera, meaning it isn’t available for devices without LiDAR. Is it possible to obtain calibration data during video recording on devices without LiDAR, such as with .builtInWideAngleCamera? Can lensDistortionLookupTable and lensDistortionCenter be used to undistort the sampleBuffer I receive from AVCaptureVideoDataOutput?

When is playbackBufferEmpty triggered? Why is this property YES observed from the observer method when returning to the foreground from the background even if the buffer is not empty?

Hello, we have HLS stream app and we use AVPlayer for HLS stream. We want to implement dynamic resulotion feature as user's selection. For example, if user want to watch only 1080p user has to watch only 1080p but we have tried to implement "preferredMaximumResolution" and "preferredPeakBitRate" parameters and but AVPlayer does not force it which means that setting preferredMaximumResolution= CGSize(width: 1920, height: 1080) player does not only force to play 1080p profile, player drops resulotion to 720p but we do not want 720p stream if user selected 1080p resulotion. Is there any method to force it even if stream stalls? Thank you in advance

I extracted the gain map info from an image using let url = Bundle.main.url(forResource: "IMG_1181", withExtension: "HEIC") let source = CGImageSourceCreateWithURL(url! as CFURL, nil) let portraitData = CGImageSourceCopyAuxiliaryDataInfoAtIndex(source!, 0, kCGImageAuxiliaryDataTypeHDRGainMap) as! [AnyHashable : Any] let metaData = portraitData[kCGImageAuxiliaryDataInfoMetadata] as! CGImageMetadata Then I printed all the metadata tags func printMetadataProperties(from metadata: CGImageMetadata) { guard let tags = CGImageMetadataCopyTags(metadata) as? [CGImageMetadataTag] else { return } for tag in tags { if let prefix = CGImageMetadataTagCopyPrefix(tag) as String?, let namespace = CGImageMetadataTagCopyNamespace(tag) as String?, let key = CGImageMetadataTagCopyName(tag) as String?, let value = CGImageMetadataTagCopyValue(tag){ print("Namespace: \(namespace), Key: \(key), Prefix: \(prefix), value: \(value)") } else { } } } //Namespace: http://ns.apple.com/ImageIO/1.0/, Key: hasXMP, Prefix: iio, value: True //Namespace: http://ns.apple.com/HDRGainMap/1.0/, Key: HDRGainMapVersion, Prefix: HDRGainMap, value: 131072 //Namespace: http://ns.apple.com/HDRGainMap/1.0/, Key: HDRGainMapHeadroom, Prefix: HDRGainMap, value: 3.586325 I want to create a new CGImageMetadata and tags. But when it comes to the HDR tags. It always fails to add to metadata. let tag = CGImageMetadataTagCreate( "http://ns.apple.com/HDRGainMap/1.0/" as CFString, "HDRGainMap" as CFString, "HDRGainMapHeadroom" as CFString, .default, 3.56 as CFNumber ) let path = "\(HDRGainMap):\(HDRGainMapHeadroom)" as CFString let success = CGImageMetadataSetTagWithPath(metadata, nil, path, tag)// always false The hasXMP works fine. Is HDR a private dict for Apple?

There are significant crash reports coming from iOS 18 users regarding AVKit framework that starts from this line [AVPlayerController _observeValueForKeyPath:oldValue:newValue:] which seems to be coming from iOS internal SDK. There are 2 kinds of crash we found: UI modification on background thread From the stack trace it seems like when AVPictureInPictureController is being deallocated and its view is being removed from superview somehow the code is being executed in background thread because there is this line there _AssertAutoLayoutOnAllowedThreadsOnly highlighted before the crash. But I’ve checked our code that plays around AVPictureInPictureController, in the locations where we would deallocate the object it will always be called on main thread which are insideviewDidLoad and deinit inside UIViewController class. From the log, it seems like the crash happened when user try to open another content when PIP player is active resulting in the current PIP instance will be replaced with a new one. My suspect is the observation logic inside AVPlayerController could be the hint to this issue, probably something broken over there since this issue happened across our app versions on iOS 18 users only. Unfortunately, I was unable to reproduce this issue yet but one of my colleagues reproduced it once but haven’t been able to do it again since. The reports keep raising each day up to 1.3k events in the last 30 days now. Over release object This one has lower reports than the first one but I decided to include it since it might have relevant information regarding the first crash since the starting stack trace is similar. The crash timing seems to be similar to the first one, where we deallocate existing AVPictureInPictureController and later replace it with a new one and also found only in iOS 18 users which also refers to [AVPlayerController _observeValueForKeyPath:oldValue:newValue:]. I also was unable to reproduce this issue so far. Oh, and both of the issues happened on both iPhone and iPad. We’d appreciate any advice on what we can do to avoid this in the future and probably any hint on why it could happened. I have reported this issue with bug number: FB15620734 I also attached one sample crash report for each of the crashes here. non ui thread access.crash over release.crash

Having an issue playing songs fetched with MusicKit but only on release builds, ApplicationMusicPlayer.shared.prepareToPlay() throws "The operation couldn’t be completed. (MPMusicPlayerControllerErrorDomain error 6.)" The authorization status is .authorized The debug version of the app works perfectly. Here is my project for the reference. Thank you. macOS 15.0.1, Xcode 16.0

Our capture application records system audio via HAL plugin, however, with the latest macOS 15 Sequoia, all audio buffer values are zero. I am attaching sample code that replicates the problem. Compile as a Command Line Tool application with Xcode. STEPS TO REPRODUCE Install BlackHole 2ch audio driver: https://existential.audio/blackhole/download/?code=1579271348 Start some system audio, e.g. YouTube. Compile and run the sample application. On macOS up to Sonoma, you will hear audio via loopback and see audio values in the debug/console window. On macOS Sequoia, you will not hear audio and the audio values are 0. #import <AVFoundation/AVFoundation.h> #import <CoreAudio/CoreAudio.h> #define BLACKHOLE_UID @"BlackHole2ch_UID" #define DEFAULT_OUTPUT_UID @"BuiltInSpeakerDevice" @interface AudioCaptureDelegate : NSObject <AVCaptureAudioDataOutputSampleBufferDelegate> @end void setDefaultAudioDevice(NSString *deviceUID); @implementation AudioCaptureDelegate // receive samples from CoreAudio/HAL driver and print amplitute values for testing // this is where samples would normally be copied and passed downstream for further processing which // is not needed in this simple sample application - (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection { // Access the audio data in the sample buffer CMBlockBufferRef blockBuffer = CMSampleBufferGetDataBuffer(sampleBuffer); if (!blockBuffer) { NSLog(@"No audio data in the sample buffer."); return; } size_t length; char *data; CMBlockBufferGetDataPointer(blockBuffer, 0, NULL, &length, &data); // Process the audio samples to calculate the average amplitude int16_t *samples = (int16_t *)data; size_t sampleCount = length / sizeof(int16_t); int64_t sum = 0; for (size_t i = 0; i < sampleCount; i++) { sum += abs(samples[i]); } // Calculate and log the average amplitude float averageAmplitude = (float)sum / sampleCount; NSLog(@"Average Amplitude: %f", averageAmplitude); } @end // set the default audio device to Blackhole while testing or speakers when done // called by main void setDefaultAudioDevice(NSString *deviceUID) { AudioObjectPropertyAddress address; AudioDeviceID deviceID = kAudioObjectUnknown; UInt32 size; CFStringRef uidString = (__bridge CFStringRef)deviceUID; // Gets the device corresponding to the given UID. AudioValueTranslation translation; translation.mInputData = &uidString; translation.mInputDataSize = sizeof(uidString); translation.mOutputData = &deviceID; translation.mOutputDataSize = sizeof(deviceID); size = sizeof(translation); address.mSelector = kAudioHardwarePropertyDeviceForUID; address.mScope = kAudioObjectPropertyScopeGlobal; //???? address.mElement = kAudioObjectPropertyElementMain; OSStatus status = AudioObjectGetPropertyData(kAudioObjectSystemObject, &address, 0, NULL, &size, &translation); if (status != noErr) { NSLog(@"Error: Could not retrieve audio device ID for UID %@. Status code: %d", deviceUID, (int)status); return; } AudioObjectPropertyAddress propertyAddress; propertyAddress.mSelector = kAudioHardwarePropertyDefaultOutputDevice; propertyAddress.mScope = kAudioObjectPropertyScopeGlobal; status = AudioObjectSetPropertyData(kAudioObjectSystemObject, &propertyAddress, 0, NULL, sizeof(AudioDeviceID), &deviceID); if (status == noErr) { NSLog(@"Default audio device set to %@", deviceUID); } else { NSLog(@"Failed to set default audio device: %d", status); } } // sets Blackhole device as default and configures it as AVCatureDeviceInput // sets the speakers as loopback so we can hear what is being captured // sets up queue to receive capture samples // runs session for 30 seconds, then restores speakers as default output int main(int argc, const char * argv[]) { @autoreleasepool { // Create the capture session AVCaptureSession *session = [[AVCaptureSession alloc] init]; // Select the audio device AVCaptureDevice *audioDevice = nil; NSString *audioDriverUID = nil; audioDriverUID = BLACKHOLE_UID; setDefaultAudioDevice(audioDriverUID); audioDevice = [AVCaptureDevice deviceWithUniqueID:audioDriverUID]; if (!audioDevice) { NSLog(@"Audio device %s not found!", [audioDriverUID UTF8String]); return -1; } else { NSLog(@"Using Audio device: %s", [audioDriverUID UTF8String]); } // Configure the audio input with the selected device (Blackhole) NSError *error = nil; AVCaptureDeviceInput *audioInput = [AVCaptureDeviceInput deviceInputWithDevice:audioDevice error:&error]; if (error || !audioInput) { NSLog(@"Failed to create audio input: %@", error); return -1; } [session addInput:audioInput]; // Configure the audio data output AVCaptureAudioDataOutput *audioOutput = [[AVCaptureAudioDataOutput alloc] init]; AudioCaptureDelegate *delegate = [[AudioCaptureDelegate alloc] init]; dispatch_queue_t queue = dispatch_queue_create("AudioCaptureQueue", NULL); [audioOutput setSampleBufferDelegate:delegate queue:queue]; [session addOutput:audioOutput]; // Set audio settings NSDictionary *audioSettings = @{ AVFormatIDKey: @(kAudioFormatLinearPCM), AVSampleRateKey: @48000, AVNumberOfChannelsKey: @2, AVLinearPCMBitDepthKey: @16, AVLinearPCMIsFloatKey: @NO, AVLinearPCMIsNonInterleaved: @NO }; [audioOutput setAudioSettings:audioSettings]; AVCaptureAudioPreviewOutput * loopback_output = nil; loopback_output = [[AVCaptureAudioPreviewOutput alloc] init]; loopback_output.volume = 1.0; loopback_output.outputDeviceUniqueID = DEFAULT_OUTPUT_UID; [session addOutput:loopback_output]; const char *deviceID = loopback_output.outputDeviceUniqueID ? [loopback_output.outputDeviceUniqueID UTF8String] : "nil"; NSLog(@"session addOutput for preview/loopback: %s", deviceID); // Start the session [session startRunning]; NSLog(@"Capturing audio data for 30 seconds..."); [[NSRunLoop currentRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:30.0]]; // Stop the session [session stopRunning]; NSLog(@"Capture session stopped."); setDefaultAudioDevice(DEFAULT_OUTPUT_UID); } return 0; }

Hi. I want to read ADPCM encoded audio data, coming from an external device to my Mac via serial port (/dev/cu.usbserial-0001) as 256 byte chunks, and feed it into an audio player. So far I am using Swift and SwiftSerial (GitHub - mredig/SwiftSerial: A Swift Linux and Mac library for reading and writing to serial ports. 3) to get the data via serialPort.asyncBytes() into a AsyncStream but I am struggling to understand how to feed the stream to a AVAudioPlayer or similar. I am new to Swift and macOS audio development so any help to get me on the right track is greatly appreciated. Thx

Hello, I will use AVFoundation's AVAssetWriter and AVPlayer for H.264 and H.265 encoding and decoding in my app. It will be used commercially, so I would like to know if I need to pay any licensing fees for H.264 and H.265 encoding and decoding.