Dive into the technical aspects of audio on your device, including codecs, format support, and customization options.

Audio Documentation

Posts under Audio subtopic

Post

Replies

Boosts

Views

Activity

Downloading multiple HLS audio files simultaneously fails.
I have been trying to multiple HLS audio files together. The files are encrypted with a key. I| need to fetch the key and store it locally for offline use. When I download a small number (2 or 3 files) of files simultaneously it works fine, but if I start downloading 10-15 files simultaneously most of them fails with the error message-Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedFailureReason=An unknown error occurred (-17377), NSLocalizedDescription=The operation could not be completed}I am getting error message from NSURLErrorDomain as well but they are rare.Below is my download function -class AudioDownloader { var productKey: String var downloadUrl: URL var downloadSession: AVAssetDownloadURLSession? var fakeDownloadUrl: URL? var downloadTask: AVAssetDownloadTask?func downloadAudio() { if downloadSession == nil { let configuration = URLSessionConfiguration.background(withIdentifier: self.productKey) downloadSession = AVAssetDownloadURLSession(configuration: configuration, assetDownloadDelegate: self, delegateQueue: OperationQueue.main) configuration.shouldUseExtendedBackgroundIdleMode = true configuration.httpShouldSetCookies = true configuration.httpShouldUsePipelining = false configuration.allowsCellularAccess = true configuration.isDiscretionary = true } self.fakeDownloadUrl = self.convertToScheme(url: self.downloadUrl, scheme: "fakehttp") let asset = AVURLAsset(url: self.fakeDownloadUrl!) let loader = asset.resourceLoader loader.setDelegate(self, queue: DispatchQueue(label: "dispatch2")) self.downloadTask = downloadSession?.makeAssetDownloadTask(asset: asset, assetTitle: "assetTitle \(self.productKey)", assetArtworkData: nil, options: nil)! self.downloadTask?.taskDescription = self.productKey self.downloadTask?.resume()}}The link of the approach which I am using to fetch the key for offline use is below -https://stackoverflow.com/questions/45670774/playing-offline-hls-with-aes-128-encryption-iosAny help would be appreciated.
3
0
2.4k
Dec ’21
Connecting iOS to MacOS via Audio MIDI Setup & Sending MIDI
Hi! I am trying to develop an application that uses Core MIDI to instantiate a connection to my Macbook via Audio MIDI Setup. I have created a client within my application and that shows up under the directory in Audio MIDI Setup on my macbook. Now I am stuck trying to figure out how to send MIDI from my app to my computer. I have tried MIDISend, and using CreateOutputPort. Both are successful in the sense that I don't get zeros when printing to the console, but nothing changes in the DAW when I set the controller and number values to the exact numbers I created in my app.I have a feeling that I am missing a network connection within my app somehow so that it recognizes my computer as a source, but I have not yet found an effective method to do this.Any information as to how I get midi to send from my app to my DAW on my computer would be greatly appreciated!I am trying to make this for my final project in one of my coding classes.Thanks!-GH
2
0
5.8k
May ’23
Setting AVRoutePickerView icon
Hello,I have recently implemented an AVRoutePickerView. It is working well, but the icon shows as "Airplay Audio" instead of "Airplay Video"https://developer.apple.com/design/human-interface-guidelines/airplay/overview/media-playback/#entering-airplayI would like it to use the "Airplay Video" icon and I cannot figure out how to set it. The closest thing I've found is attempting to set the routePickerButtonStyle but when I do it says "'routePickerButtonStyle' is unavailable".Any help would be appreciated.Thank you.
5
0
4.6k
Mar ’23
WKWebView Autolayout with magnification
When a magnification is altered from default 1.0 (100%) I'm trying to maintain my orignal layout - set in IB like so: func fit(_ childView: NSView, parentView: NSView) { childView.translatesAutoresizingMaskIntoConstraints = false childView.topAnchor.constraint(equalTo: parentView.topAnchor).isActive = true childView.leadingAnchor.constraint(equalTo: parentView.leadingAnchor).isActive = true childView.trailingAnchor.constraint(equalTo: parentView.trailingAnchor).isActive = true childView.bottomAnchor.constraint(equalTo: parentView.bottomAnchor).isActive = true }where the childView is the webView and its parent container view - fit() called from viewDidLoad(). The result is the view is pegged to top left, nice but I'like the window content to match the now smaller or larger webView - using scroll bars if needbe. Any user attempt to resize window, being to skew the layout until they resume default.Can someone suggestt how to go about supporting magnification ?
2
0
2.0k
Apr ’22
Is AudioOutputUnitStop synchronous?
That is, will my render callback ever be called after AudioOutputUnitStop() returns?In other words will it be safe to free resources used by the render callback or do I need to add realtime safe communication between the stopping thread and the callback thread?This question is intended for both macOS HAL Output and iOS Remote IO output units.
5
0
5.6k
Aug ’23
AVAudioPlayerNode crashes on play
I have a very simple setup involving AVAudioEngine, and a player node, AVAudioPlayerNode.The player node is attached to the audio engine, then connected to the engine's main mixer node.Then, to start outputing some sound, I start the engine, and finally play the node :NSError* err = nil; BOOL started = [_audioEngine startAndReturnError:&err]; if(started) { [_playerNode play]; } else { // handle error }It happens sometimes, the app crashes (on the call to the play function) because, as the system says : "player started when engine not running".I can't understand how this can happen, at least in the code above.No error is returned in `err`.Does somebody knows what happen or faced such a situation ?
3
1
3.6k
Nov ’22
Image Capture API to access camera's filesystem?
In this URL there is the following blerb:https://www.apple.com/in/ipados/ipados-preview/features/Image Capture APIThe Image Capture API allows developers to leverage the Camera Connection Kit to import photos directly into their apps.I haven't been able to find in the docs or WWDC sessions where it talks about this. I have an app for our action camera and would love to be able to directly read the camera's filesystem over USB. I see where you can use a a document picker to access a USB filesystem, but that requires the user to properly find/select the correct directory in the camera's file system. I am hoping to be able to detect it and access it without the error prone user selection.
3
0
2.9k
Jun ’23
WKWebView scroll view access on OSX
If I create, via interface builder, a window with a WKWebView, all is nice. I wire it up, constraints and added drop-n-drop support for asset viewing. But upon loading say a movie for example, the window appears to obtain a scroll view. This is easily triggered when resizing the window and make the window smaller - the video will be resized once vertical size is changed, but the addition of a horizontal scrollbar cannot be removed, unless a resize is large enough for the video to fit.I'd like to hide the scroll views' sliders when the mouse leaves the window, but I first need a way to access the scrollView.I have tried to query the WKWebView's enclosingScrollView - but always nil, and on MacOSX, there is no scrollView as on iOS.Any help appreciated, thanks.
3
0
1.6k
Aug ’21
Access to IR camera sensor (no depth image)
Hi everybody,I would like to access the raw data from the iPhone True Depth Camera (IR sensor), in order to do my own image processing.I'm not talking about depth info but the infrared image that the iPhone uses to get those depth information.You can see what I'm talking about below:https://www.youtube.com/watch?v=g4m6StzUcOwHowever, I know that the API allow to get processed depth info but we couldn't find any reference to the raw IR data used to obtain it.Do you know if is it possible to access that information?Thanks in advance.
3
0
9.2k
Feb ’23
AVAssetExportSession fails in compiling .mp4 video and .aac audio together on iOS13 device
Some of my code for combining an .mp4 and an .aac file together into an .mov file worked just fine since its very beginning, it creats an AVMutableComposition with 1 video track and 1 audio track from the input audio and video files. After exportAsynchronously(), the AVAssetExportSession gets AVAssetExportSession.Status.completed. Here is the Swift source code of the key function: func compileAudioAndVideoToMovie(audioInputURL:URL, videoInputURL:URL) { let docPath:String = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true)[0]; let videoOutputURL:URL = URL(fileURLWithPath: docPath).appendingPathComponent("video_output.mov"); do { try FileManager.default.removeItem(at: videoOutputURL); } catch {} let mixComposition = AVMutableComposition(); let videoTrack = mixComposition.addMutableTrack(withMediaType: AVMediaType.video, preferredTrackID: kCMPersistentTrackID_Invalid); let videoInputAsset = AVURLAsset(url: videoInputURL); let audioTrack = mixComposition.addMutableTrack(withMediaType: AVMediaType.audio, preferredTrackID: kCMPersistentTrackID_Invalid); let audioInputAsset = AVURLAsset(url: audioInputURL); do { try videoTrack?.insertTimeRange(CMTimeRangeMake(start: CMTimeMake(value: 0, timescale: 1000), duration: CMTimeMake(value: 3000, timescale: 1000)), of: videoInputAsset.tracks(withMediaType: AVMediaType.video)[0], at: CMTimeMake(value: 0, timescale: 1000));// Insert an 3-second video clip into the video track try audioTrack?.insertTimeRange(CMTimeRangeMake(start: CMTimeMake(value: 0, timescale: 1000), duration: CMTimeMake(value: 3000, timescale: 1000)), of: audioInputAsset.tracks(withMediaType: AVMediaType.audio)[0], at: CMTimeMake(value: 0, timescale: 1000));// Insert an 3-second audio clip into the audio track let assetExporter = AVAssetExportSession(asset: mixComposition, presetName: AVAssetExportPresetPassthrough); assetExporter?.outputFileType = AVFileType.mov; assetExporter?.outputURL = videoOutputURL; assetExporter?.shouldOptimizeForNetworkUse = false; assetExporter?.exportAsynchronously { switch (assetExporter?.status) { case .cancelled: print("Exporting cancelled"); case .completed: print("Exporting completed"); case .exporting: print("Exporting ..."); case .failed: print("Exporting failed"); default: print("Exporting with other result"); } if let error = assetExporter?.error { print("Error:\n\(error)"); } } } catch { print("Exception when compiling movie"); } }However, after I upgraded my iPhone to iOS13(beta), it always ends up with .failed status, and the AVAssetExportSession.error reads: Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedFailureReason=An unknown error occurred (-12735), NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x2815e9bc0 {Error Domain=NSOSStatusErrorDomain Code=-12735 "(null)”}}I've tested this on iPhone6Plus and iPhone7, both given the same result.You can clone my minimal demo project with sample input audio and video files embedded in its bundle from here:https://github.com/chenqiu1024/iOS13VideoRecordingError.git, run it and check the console output.Is there any explanation and suggestion?
4
0
5.6k
Dec ’21
Shooting with 2 rear cameras
I was wondering if there's a way to shoot with two rear cameras simultaneously and recieve two discrete images from them?I saw that with iPhone 11 Pro and iOS 13, some apps like Filmic Pro will have access to two video streams from multiple cameras. Is that available to all developers or only Apple partners? If so, is the API available or will it be made available in the future?I'm looking to take two photos from rear facing cameras and applying post-processing to them seperately before merging them in our app.Also, related but seperate: is there any way to recieve rectification/calibration data from the individual device to account for alignment differences between individual units during manufacturing?
4
0
2.8k
Jan ’22
Midi AUv3 appearing in logic multiple times?
I'm making an Auv3 MIDI plugin (aumi type) which will be a port of one of my iOS ones My basic framework - which at this point is litterally an AUv3 with busses and midi output set and aumi type appears in Logic in the MIDI fx drop down 8 times. I have a couple of other bought pugins (Reaktor and Numerology) which appear in the list only onceAny ideas what I might need to change ?
4
0
3.2k
Oct ’21
AVAudioSession category becomes nil and mediaServicesWereReset, avplayer playback keeps failing
Our team have an app to play back m4a resources online using the avplayer. Recently, there are some users complaining the playback is keep failing and we have no idea the reason behind this.We checked the user log and the avplayer error log are as follow (for multiple failed instance):- avPlayer.currentItem.error = Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedFailureReason=An unknown error occurred (-16155), NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x280e6ef10 {Error Domain=NSOSStatusErrorDomain Code=-16155 "(null)"}}- avPlayer.currentItem.error = Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedFailureReason=An unknown error occurred (606068440), NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x280e9f8d0 {Error Domain=NSOSStatusErrorDomain Code=606068440 "(null)"}}- avPlayer.currentItem.error = Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedFailureReason=An unknown error occurred (1705376704), NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x281ec60d0 {Error Domain=NSOSStatusErrorDomain Code=1705376704 "(null)"}}The normal flow for us to start playing: (Work as expected for majority of user)1. [[AVAudioSession sharedInstance] setActive:YES error:&activationError];2. call [avplayer play]3. the audio started to play successfullyFailed scenario: (For some users, this scenario keeps happening)1. The activation error returns Error Domain=NSOSStatusErrorDomain Code=2003329396 "(null)"2. We logged the [AVAudioSession sharedInstance].category becomes empty3. The mediaServicesWereReset notification is received4. the avplayer failed to play and the one of the above avplayer item error is observedWhen the user failed once, he cannot play any audio resources in our app and the scenario is keep repeating.We would like to know:1. Why would this occur on certain user devices? 2. How to prevent the problem occurs?3. Is there a way to recover from the lost mediaService? so that even if the error occurs once, the user can still play other resources in our app.We cannot produce the fail scenario by ourselves even we tried the Reset Media Services in developer menu, the behaviour is not exactly the same. Look forward to any help from the community and thanks.
1
0
2.8k
Jun ’22
AVAudioEngine thread-safety
I've been wrestling with a problem related to AVAudioEngine. I've posted a couple questions recently to other forums, but haven't had much luck, so I'm guessing not many people are encountering this, or the questions are unclear, or perhaps I'm not asking in the most appropriate subforums. So, I thought I'd try here in the 'Concurrency' forum, as using concurrency would be one way to solve the problem.The specific problem is that AVAudioPlayerNode.play() takes a long time to execute. The execution time seems to be proportional to the value of AVAudioSession.ioBufferDuration, and can be from a few milliseconds for low buffer durations to over 20 milliseconds at the default buffer duration. These execution times can be an issue in real-time applications such as games.An obvious solution would be to move such operations to a background thread using GCD, and I've seen various posts and articles that do this. Here's a code example showing what I mean:import AVFoundation import UIKit class ViewController: UIViewController { private let engine = AVAudioEngine() private let player = AVAudioPlayerNode() private let queue = DispatchQueue(label: "", qos: .userInitiated) override func viewDidLoad() { super.viewDidLoad() engine.attach(player) engine.connect(player, to: engine.mainMixerNode, format: nil) try! engine.start() } override func touchesBegan(_ touches: Set, with event: UIEvent?) { queue.async { if self.player.isPlaying { self.player.stop() } else { self.player.play() } } } }In this scenario all AVAudioEngine-related operations would be serial and never concurrent/parallel. The audio system would never be accessed simultaneously from multiple threads, only serially.My concern is that I don't know whether it's safe to use AVAudioEngine in this way. More generally, I'm not sure what should be assumed about any API for which nothing specific is said about thread safety. In such cases, can it be assumed that access from multiple threads is safe as long as only one thread is active at any given time? (The 'Thread Programming Guide' touches on this, but doesn't appear to address audio frameworks specifically.)The narrowest version of my question is whether it's safe to use GCD with AVAudioEngine, provided all access is serial. The broader question would be what assumptions should or should not be made about APIs and thread safety when it's not specifically addressed in the documentation.Any input on either of these issues would be greatly appreciated.
2
4
3.2k
May ’22
Dismissing AVPlayerViewController doesn't 'kill' the object - it's persisting on lock screen / media controls
I am playing videos that are in my app bundle.They are playing correctly.However, when I call to dismiss the AVPlayerViewController, it visibly is removed from the view hierarchy but, if I turn off the iOS device and turn it back on again, on the lock screen there is a media control showing that video and a 'play' button.If you touch play you only get the audio and no video.My problem is I don't understand why the 'dismiss' is not completely 'killing' the player when I'm done with it.NOTE: The follow code works perfectly in iOS 13 devices, however any device running previous iOS versions, the error appears as stated. This is blocking me from submitting my app, please can someone advise on how to solve this? Here is the presentation code: internal func play(FileName filename: String, FileType type: String) { if self.isAlreadyPlaying == YES { killVideoPlayer() DispatchQueue.main.asyncAfter(deadline: .now() + 1.5, execute: { self.play(FileName: filename, FileType: type) }) return } let audioSession = AVAudioSession.sharedInstance() do { try audioSession.setCategory(AVAudioSession.Category.playback, mode: .moviePlayback, options: [.allowAirPlay, .allowBluetooth, .allowBluetoothA2DP]) try audioSession.setActive(true) } catch { print("Audio session failed") } let path = Bundle.main.path(forResource: filename, ofType: type) let url = NSURL(fileURLWithPath: path!) let player = AVPlayer(url: url as URL) NotificationCenter.default.addObserver(self, selector: #selector(PBFMenuSystemFloatingButtonsViewController.didFinishPlaying(notification:)), name: NSNotification.Name.AVPlayerItemDidPlayToEndTime, object: player.currentItem) self.playerController = AVPlayerViewController() self.menuWindow.videoPlayer = self.playerController self.playerController?.player = player self.playerController?.allowsPictureInPicturePlayback = true self.playerController?.showsPlaybackControls = YES self.playerController?.delegate = self self.playerController?.exitsFullScreenWhenPlaybackEnds = YES self.isAlreadyPlaying = YES self.present(self.playerController!, animated: true, completion : { self.playerController?.player?.play() }) } Here is the dismissal code: private func killVideoPlayer() { self.isAlreadyPlaying = NO self.playerController?.player?.pause() self.playerController?.player = nil let audioSession = AVAudioSession.sharedInstance() do { try audioSession.setActive(false, options: .notifyOthersOnDeactivation) try audioSession.setCategory(.soloAmbient) } catch { print("Audio session failed") } self.playerController?.dismiss(animated: YES, completion: { self.playerController = nil }) } And here's what's remaining in the systemwide media player that is shown on the lock screen / control centre:
1
0
3.5k
Oct ’21
Convert a CMSampleBuffer to AVAudioPCMBuffer
I'm trying to convert a CMSampleBuffer to a AVAudioPCMBuffer instance to be able to perform audio processing in realtime. I wrote an optional initialiser for my extension to pass a CMSampleBuffer reference. Unfortunately I simply don't know how to write to the AVAudioPCMBuffer's data. Here is my code so far:import AVKit extension AVAudioPCMBuffer { static func create(from sampleBuffer: CMSampleBuffer) -> AVAudioPCMBuffer? { guard let description: CMFormatDescription = CMSampleBufferGetFormatDescription(sampleBuffer), let sampleRate: Float64 = description.audioStreamBasicDescription?.mSampleRate, let numberOfChannels: Int = description.audioChannelLayout?.numberOfChannels else { return nil } guard let blockBuffer: CMBlockBuffer = CMSampleBufferGetDataBuffer(sampleBuffer) else { return nil } let length: Int = CMBlockBufferGetDataLength(blockBuffer) let audioFormat = AVAudioFormat(commonFormat: .pcmFormatFloat32, sampleRate: sampleRate, channels: AVAudioChannelCount(numberOfChannels), interleaved: false) let buffer = AVAudioPCMBuffer(pcmFormat: audioFormat!, frameCapacity: AVAudioFrameCount(length))! buffer.frameLength = buffer.frameCapacity for channelIndex in 0...numberOfChannels - 1 { guard let channel: UnsafeMutablePointer<Float> = buffer.floatChannelData?[channelIndex] else { return nil } for pointerIndex in 0...length - 1 { let pointer: UnsafeMutablePointer<Float> = channel.advanced(by: pointerIndex) pointer.pointee = 100 } } return buffer } }Does anyone knows how to convert a CMSampleBuffer to AVAudioPCMBuffer and back again? I assume there is no other way if I want to interact with AVAudioEngine, am I right?
3
0
8.0k
Oct ’22
AVAssetReaderOutput.copyNextSampleBuffer() sometimes hangs forever
I'm using AVAssetReaders with AVSampleBufferDisplayLayers to display multiple videos at once. I'm seeing this issue on iOS 13.1.3, 13.2b2, on various hardware like iPad 10.5 and iPad 12.9.It works well for a while, then a random call to copyNextSampleBuffer never returns, blocking that thread indefinitely and eating up resources.I have tried different threading approaches with no avail:If copyNextSampleBuffer() and reader.cancelReading() are done on the same queue, then copyNextSampleBuffer() gets stuck and the cancelReading() never gets processed because the queue is blocked. If I manually (with the debugger) jump in on that blocked queue and execute cancelReading(), immediately an EXC_BREAKPOINT crashes the appIf copyNextSampleBuffer() and reader.cancelReading() are done on different queues, then copyNextSampleBuffer() crashes with EXC_BAD_ACCESSHere's the stacktrace (same queue approach). I don't understand why it's stuck, my expectation is that copyNextSampleBuffer should always return (ie. with nil in error case).VideoPlayerView: UIView with AVSampleBufferDisplayLayerAVAssetFactory: Singleton with the queue that creates & manages all AVAssetReader / AVAsset* objects* thread #22, queue = 'AVAssetFactory' frame #0: 0x00000001852355f4 libsystem_kernel.dylib`mach_msg_trap + 8 frame #1: 0x0000000185234a60 libsystem_kernel.dylib`mach_msg + 72 frame #2: 0x00000001853dc068 CoreFoundation`__CFRunLoopServiceMachPort + 216 frame #3: 0x00000001853d7188 CoreFoundation`__CFRunLoopRun + 1444 frame #4: 0x00000001853d68bc CoreFoundation`CFRunLoopRunSpecific + 464 frame #5: 0x000000018f42b6ac AVFoundation`-[AVRunLoopCondition _waitInMode:untilDate:] + 400 frame #6: 0x000000018f38f1dc AVFoundation`-[AVAssetReaderOutput copyNextSampleBuffer] + 148 frame #7: 0x000000018f3900f0 AVFoundation`-[AVAssetReaderTrackOutput copyNextSampleBuffer] + 72 * frame #8: 0x0000000103309d98 Photobooth`closure #1 in AVAssetFactory.nextSampleBuffer(reader=0x00000002814016f0, retval=(Swift.Optional<CoreMedia.CMSampleBuffer>, Swift.Optional<AVFoundation.AVAssetReader.Status>) @ 0x000000016dbd1cb8) at AVAssetFactory.swift:108:34 frame #9: 0x0000000102f4f480 Photobooth`thunk for @callee_guaranteed () -> () at <compiler-generated>:0 frame #10: 0x0000000102f4f4a4 Photobooth`thunk for @escaping @callee_guaranteed () -> () at <compiler-generated>:0 frame #11: 0x000000010bfe6c04 libdispatch.dylib`_dispatch_client_callout + 16 frame #12: 0x000000010bff5888 libdispatch.dylib`_dispatch_lane_barrier_sync_invoke_and_complete + 124 frame #13: 0x0000000103309a5c Photobooth`AVAssetFactory.nextSampleBuffer(reader=0x00000002814016f0, self=0x0000000281984f60) at AVAssetFactory.swift:101:20 frame #14: 0x00000001032ab690 Photobooth`closure #1 in VideoPlayerView.setRequestMediaLoop(self=0x000000014b8da1d0, handledCompletion=false) at VideoPlayerView.swift:254:70 frame #15: 0x0000000102dce978 Photobooth`thunk for @escaping @callee_guaranteed () -> () at <compiler-generated>:0 frame #16: 0x000000018f416848 AVFoundation`-[AVMediaDataRequester _requestMediaDataIfReady] + 80 frame #17: 0x000000010bfe5828 libdispatch.dylib`_dispatch_call_block_and_release + 24 frame #18: 0x000000010bfe6c04 libdispatch.dylib`_dispatch_client_callout + 16 frame #19: 0x000000010bfedb74 libdispatch.dylib`_dispatch_lane_serial_drain + 744 frame #20: 0x000000010bfee744 libdispatch.dylib`_dispatch_lane_invoke + 500 frame #21: 0x000000010bff9ae4 libdispatch.dylib`_dispatch_workloop_worker_thread + 1324 frame #22: 0x000000018517bfa4 libsystem_pthread.dylib`_pthread_wqthread + 276I've tried all kinds of other things like making sure the AVAssets and all objects are made on one queue, and stopping the AVAssetReaders from ever deallocing to see if that helps. Nothing works. Any ideas?
4
0
3.1k
Oct ’23
Playback streaming audio with AVAudioengine (iOS)
I want to stream audio on iOS and use for that use-case the AVAudioEngine. So, currently I'm not really sure, what is the best solution for my problem.I get the RTP data from the network and want playback this audio data with AVAudioEngine. I use the iOS Network.Framework to receive the network data. Then first I decode my voice data and want to playback it, now.Here is my receive code:connection.receiveMessage { (data, context, isComplete, error) in if isComplete { // decode the raw network data with Audio codec G711 let decodedData = AudioDecoder.decode(enc: data, frames: 160) // create PCMBuffer for audio data for playback let format = AVAudioFormat(settings: [AVFormatIDKey: NSNumber(value: kAudioFormatALaw), AVSampleRateKey: 8000, AVNumberOfChannelsKey: 1]) // let format = AVAudioFormat(standardFormatWithSampleRate: 8000, channels: 1) let buffer = AVAudioPCMBuffer(pcmFormat: format!, frameCapacity: 160)! buffer.frameLength = buffer.frameCapacity // TODO: now I have to copy the decodedData --> buffer (AVAudioPCMBuffer) if error == nil { // recall receive() for next message self.receive(on: connection) } }?How I have to copy my decoded data into the AVAudioPCMBuffer? Currently, my AVAudioPCMBuffer is created, but not contain any audio data.Background information: My generell approach would be to cash here in the above code (at ToDo-Line) the PCMBuffer in a collection and playback this collection by the AVAudioEngine (in a background thread).My decoded linear data is cashed in an array from type Int16. So the var decodedData is from type [Int16], maybe there is a possibility to consume this data directly? The function scheduleBuffer allows only AVAudioPCMBuffer as input.
2
0
2.1k
Apr ’22
Downloading multiple HLS audio files simultaneously fails.
I have been trying to multiple HLS audio files together. The files are encrypted with a key. I| need to fetch the key and store it locally for offline use. When I download a small number (2 or 3 files) of files simultaneously it works fine, but if I start downloading 10-15 files simultaneously most of them fails with the error message-Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedFailureReason=An unknown error occurred (-17377), NSLocalizedDescription=The operation could not be completed}I am getting error message from NSURLErrorDomain as well but they are rare.Below is my download function -class AudioDownloader { var productKey: String var downloadUrl: URL var downloadSession: AVAssetDownloadURLSession? var fakeDownloadUrl: URL? var downloadTask: AVAssetDownloadTask?func downloadAudio() { if downloadSession == nil { let configuration = URLSessionConfiguration.background(withIdentifier: self.productKey) downloadSession = AVAssetDownloadURLSession(configuration: configuration, assetDownloadDelegate: self, delegateQueue: OperationQueue.main) configuration.shouldUseExtendedBackgroundIdleMode = true configuration.httpShouldSetCookies = true configuration.httpShouldUsePipelining = false configuration.allowsCellularAccess = true configuration.isDiscretionary = true } self.fakeDownloadUrl = self.convertToScheme(url: self.downloadUrl, scheme: "fakehttp") let asset = AVURLAsset(url: self.fakeDownloadUrl!) let loader = asset.resourceLoader loader.setDelegate(self, queue: DispatchQueue(label: "dispatch2")) self.downloadTask = downloadSession?.makeAssetDownloadTask(asset: asset, assetTitle: "assetTitle \(self.productKey)", assetArtworkData: nil, options: nil)! self.downloadTask?.taskDescription = self.productKey self.downloadTask?.resume()}}The link of the approach which I am using to fetch the key for offline use is below -https://stackoverflow.com/questions/45670774/playing-offline-hls-with-aes-128-encryption-iosAny help would be appreciated.
Replies
3
Boosts
0
Views
2.4k
Activity
Dec ’21
Connecting iOS to MacOS via Audio MIDI Setup & Sending MIDI
Hi! I am trying to develop an application that uses Core MIDI to instantiate a connection to my Macbook via Audio MIDI Setup. I have created a client within my application and that shows up under the directory in Audio MIDI Setup on my macbook. Now I am stuck trying to figure out how to send MIDI from my app to my computer. I have tried MIDISend, and using CreateOutputPort. Both are successful in the sense that I don't get zeros when printing to the console, but nothing changes in the DAW when I set the controller and number values to the exact numbers I created in my app.I have a feeling that I am missing a network connection within my app somehow so that it recognizes my computer as a source, but I have not yet found an effective method to do this.Any information as to how I get midi to send from my app to my DAW on my computer would be greatly appreciated!I am trying to make this for my final project in one of my coding classes.Thanks!-GH
Replies
2
Boosts
0
Views
5.8k
Activity
May ’23
Setting AVRoutePickerView icon
Hello,I have recently implemented an AVRoutePickerView. It is working well, but the icon shows as "Airplay Audio" instead of "Airplay Video"https://developer.apple.com/design/human-interface-guidelines/airplay/overview/media-playback/#entering-airplayI would like it to use the "Airplay Video" icon and I cannot figure out how to set it. The closest thing I've found is attempting to set the routePickerButtonStyle but when I do it says "'routePickerButtonStyle' is unavailable".Any help would be appreciated.Thank you.
Replies
5
Boosts
0
Views
4.6k
Activity
Mar ’23
WKWebView Autolayout with magnification
When a magnification is altered from default 1.0 (100%) I'm trying to maintain my orignal layout - set in IB like so: func fit(_ childView: NSView, parentView: NSView) { childView.translatesAutoresizingMaskIntoConstraints = false childView.topAnchor.constraint(equalTo: parentView.topAnchor).isActive = true childView.leadingAnchor.constraint(equalTo: parentView.leadingAnchor).isActive = true childView.trailingAnchor.constraint(equalTo: parentView.trailingAnchor).isActive = true childView.bottomAnchor.constraint(equalTo: parentView.bottomAnchor).isActive = true }where the childView is the webView and its parent container view - fit() called from viewDidLoad(). The result is the view is pegged to top left, nice but I'like the window content to match the now smaller or larger webView - using scroll bars if needbe. Any user attempt to resize window, being to skew the layout until they resume default.Can someone suggestt how to go about supporting magnification ?
Replies
2
Boosts
0
Views
2.0k
Activity
Apr ’22
Is AudioOutputUnitStop synchronous?
That is, will my render callback ever be called after AudioOutputUnitStop() returns?In other words will it be safe to free resources used by the render callback or do I need to add realtime safe communication between the stopping thread and the callback thread?This question is intended for both macOS HAL Output and iOS Remote IO output units.
Replies
5
Boosts
0
Views
5.6k
Activity
Aug ’23
AVAudioPlayerNode crashes on play
I have a very simple setup involving AVAudioEngine, and a player node, AVAudioPlayerNode.The player node is attached to the audio engine, then connected to the engine's main mixer node.Then, to start outputing some sound, I start the engine, and finally play the node :NSError* err = nil; BOOL started = [_audioEngine startAndReturnError:&err]; if(started) { [_playerNode play]; } else { // handle error }It happens sometimes, the app crashes (on the call to the play function) because, as the system says : "player started when engine not running".I can't understand how this can happen, at least in the code above.No error is returned in `err`.Does somebody knows what happen or faced such a situation ?
Replies
3
Boosts
1
Views
3.6k
Activity
Nov ’22
Image Capture API to access camera's filesystem?
In this URL there is the following blerb:https://www.apple.com/in/ipados/ipados-preview/features/Image Capture APIThe Image Capture API allows developers to leverage the Camera Connection Kit to import photos directly into their apps.I haven't been able to find in the docs or WWDC sessions where it talks about this. I have an app for our action camera and would love to be able to directly read the camera's filesystem over USB. I see where you can use a a document picker to access a USB filesystem, but that requires the user to properly find/select the correct directory in the camera's file system. I am hoping to be able to detect it and access it without the error prone user selection.
Replies
3
Boosts
0
Views
2.9k
Activity
Jun ’23
WKWebView scroll view access on OSX
If I create, via interface builder, a window with a WKWebView, all is nice. I wire it up, constraints and added drop-n-drop support for asset viewing. But upon loading say a movie for example, the window appears to obtain a scroll view. This is easily triggered when resizing the window and make the window smaller - the video will be resized once vertical size is changed, but the addition of a horizontal scrollbar cannot be removed, unless a resize is large enough for the video to fit.I'd like to hide the scroll views' sliders when the mouse leaves the window, but I first need a way to access the scrollView.I have tried to query the WKWebView's enclosingScrollView - but always nil, and on MacOSX, there is no scrollView as on iOS.Any help appreciated, thanks.
Replies
3
Boosts
0
Views
1.6k
Activity
Aug ’21
iPhone XS stereo recording
Is there a way to get stereo audio on iPhone XS using RemoteIO unit, or any of the CoreAudio APIs? Need to record audio in stereo format.
Replies
10
Boosts
0
Views
2.7k
Activity
Nov ’21
What do I do with MPMediaItem from MPMediaPickerController?
How do I copy to file system or play with AVAudioPlayer an MPMediaItem that I get from an MPMediaPickerController initialized with type audio? When I try, I get an error that says:The file “item.mp3” couldn’t be opened because URL type ipod-library isn’t supported.
Replies
1
Boosts
0
Views
793
Activity
Jan ’22
Access to IR camera sensor (no depth image)
Hi everybody,I would like to access the raw data from the iPhone True Depth Camera (IR sensor), in order to do my own image processing.I'm not talking about depth info but the infrared image that the iPhone uses to get those depth information.You can see what I'm talking about below:https://www.youtube.com/watch?v=g4m6StzUcOwHowever, I know that the API allow to get processed depth info but we couldn't find any reference to the raw IR data used to obtain it.Do you know if is it possible to access that information?Thanks in advance.
Replies
3
Boosts
0
Views
9.2k
Activity
Feb ’23
AVAssetExportSession fails in compiling .mp4 video and .aac audio together on iOS13 device
Some of my code for combining an .mp4 and an .aac file together into an .mov file worked just fine since its very beginning, it creats an AVMutableComposition with 1 video track and 1 audio track from the input audio and video files. After exportAsynchronously(), the AVAssetExportSession gets AVAssetExportSession.Status.completed. Here is the Swift source code of the key function: func compileAudioAndVideoToMovie(audioInputURL:URL, videoInputURL:URL) { let docPath:String = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true)[0]; let videoOutputURL:URL = URL(fileURLWithPath: docPath).appendingPathComponent("video_output.mov"); do { try FileManager.default.removeItem(at: videoOutputURL); } catch {} let mixComposition = AVMutableComposition(); let videoTrack = mixComposition.addMutableTrack(withMediaType: AVMediaType.video, preferredTrackID: kCMPersistentTrackID_Invalid); let videoInputAsset = AVURLAsset(url: videoInputURL); let audioTrack = mixComposition.addMutableTrack(withMediaType: AVMediaType.audio, preferredTrackID: kCMPersistentTrackID_Invalid); let audioInputAsset = AVURLAsset(url: audioInputURL); do { try videoTrack?.insertTimeRange(CMTimeRangeMake(start: CMTimeMake(value: 0, timescale: 1000), duration: CMTimeMake(value: 3000, timescale: 1000)), of: videoInputAsset.tracks(withMediaType: AVMediaType.video)[0], at: CMTimeMake(value: 0, timescale: 1000));// Insert an 3-second video clip into the video track try audioTrack?.insertTimeRange(CMTimeRangeMake(start: CMTimeMake(value: 0, timescale: 1000), duration: CMTimeMake(value: 3000, timescale: 1000)), of: audioInputAsset.tracks(withMediaType: AVMediaType.audio)[0], at: CMTimeMake(value: 0, timescale: 1000));// Insert an 3-second audio clip into the audio track let assetExporter = AVAssetExportSession(asset: mixComposition, presetName: AVAssetExportPresetPassthrough); assetExporter?.outputFileType = AVFileType.mov; assetExporter?.outputURL = videoOutputURL; assetExporter?.shouldOptimizeForNetworkUse = false; assetExporter?.exportAsynchronously { switch (assetExporter?.status) { case .cancelled: print("Exporting cancelled"); case .completed: print("Exporting completed"); case .exporting: print("Exporting ..."); case .failed: print("Exporting failed"); default: print("Exporting with other result"); } if let error = assetExporter?.error { print("Error:\n\(error)"); } } } catch { print("Exception when compiling movie"); } }However, after I upgraded my iPhone to iOS13(beta), it always ends up with .failed status, and the AVAssetExportSession.error reads: Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedFailureReason=An unknown error occurred (-12735), NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x2815e9bc0 {Error Domain=NSOSStatusErrorDomain Code=-12735 "(null)”}}I've tested this on iPhone6Plus and iPhone7, both given the same result.You can clone my minimal demo project with sample input audio and video files embedded in its bundle from here:https://github.com/chenqiu1024/iOS13VideoRecordingError.git, run it and check the console output.Is there any explanation and suggestion?
Replies
4
Boosts
0
Views
5.6k
Activity
Dec ’21
Shooting with 2 rear cameras
I was wondering if there's a way to shoot with two rear cameras simultaneously and recieve two discrete images from them?I saw that with iPhone 11 Pro and iOS 13, some apps like Filmic Pro will have access to two video streams from multiple cameras. Is that available to all developers or only Apple partners? If so, is the API available or will it be made available in the future?I'm looking to take two photos from rear facing cameras and applying post-processing to them seperately before merging them in our app.Also, related but seperate: is there any way to recieve rectification/calibration data from the individual device to account for alignment differences between individual units during manufacturing?
Replies
4
Boosts
0
Views
2.8k
Activity
Jan ’22
Midi AUv3 appearing in logic multiple times?
I'm making an Auv3 MIDI plugin (aumi type) which will be a port of one of my iOS ones My basic framework - which at this point is litterally an AUv3 with busses and midi output set and aumi type appears in Logic in the MIDI fx drop down 8 times. I have a couple of other bought pugins (Reaktor and Numerology) which appear in the list only onceAny ideas what I might need to change ?
Replies
4
Boosts
0
Views
3.2k
Activity
Oct ’21
AVAudioSession category becomes nil and mediaServicesWereReset, avplayer playback keeps failing
Our team have an app to play back m4a resources online using the avplayer. Recently, there are some users complaining the playback is keep failing and we have no idea the reason behind this.We checked the user log and the avplayer error log are as follow (for multiple failed instance):- avPlayer.currentItem.error = Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedFailureReason=An unknown error occurred (-16155), NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x280e6ef10 {Error Domain=NSOSStatusErrorDomain Code=-16155 "(null)"}}- avPlayer.currentItem.error = Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedFailureReason=An unknown error occurred (606068440), NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x280e9f8d0 {Error Domain=NSOSStatusErrorDomain Code=606068440 "(null)"}}- avPlayer.currentItem.error = Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedFailureReason=An unknown error occurred (1705376704), NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x281ec60d0 {Error Domain=NSOSStatusErrorDomain Code=1705376704 "(null)"}}The normal flow for us to start playing: (Work as expected for majority of user)1. [[AVAudioSession sharedInstance] setActive:YES error:&activationError];2. call [avplayer play]3. the audio started to play successfullyFailed scenario: (For some users, this scenario keeps happening)1. The activation error returns Error Domain=NSOSStatusErrorDomain Code=2003329396 "(null)"2. We logged the [AVAudioSession sharedInstance].category becomes empty3. The mediaServicesWereReset notification is received4. the avplayer failed to play and the one of the above avplayer item error is observedWhen the user failed once, he cannot play any audio resources in our app and the scenario is keep repeating.We would like to know:1. Why would this occur on certain user devices? 2. How to prevent the problem occurs?3. Is there a way to recover from the lost mediaService? so that even if the error occurs once, the user can still play other resources in our app.We cannot produce the fail scenario by ourselves even we tried the Reset Media Services in developer menu, the behaviour is not exactly the same. Look forward to any help from the community and thanks.
Replies
1
Boosts
0
Views
2.8k
Activity
Jun ’22
AVAudioEngine thread-safety
I've been wrestling with a problem related to AVAudioEngine. I've posted a couple questions recently to other forums, but haven't had much luck, so I'm guessing not many people are encountering this, or the questions are unclear, or perhaps I'm not asking in the most appropriate subforums. So, I thought I'd try here in the 'Concurrency' forum, as using concurrency would be one way to solve the problem.The specific problem is that AVAudioPlayerNode.play() takes a long time to execute. The execution time seems to be proportional to the value of AVAudioSession.ioBufferDuration, and can be from a few milliseconds for low buffer durations to over 20 milliseconds at the default buffer duration. These execution times can be an issue in real-time applications such as games.An obvious solution would be to move such operations to a background thread using GCD, and I've seen various posts and articles that do this. Here's a code example showing what I mean:import AVFoundation import UIKit class ViewController: UIViewController { private let engine = AVAudioEngine() private let player = AVAudioPlayerNode() private let queue = DispatchQueue(label: "", qos: .userInitiated) override func viewDidLoad() { super.viewDidLoad() engine.attach(player) engine.connect(player, to: engine.mainMixerNode, format: nil) try! engine.start() } override func touchesBegan(_ touches: Set, with event: UIEvent?) { queue.async { if self.player.isPlaying { self.player.stop() } else { self.player.play() } } } }In this scenario all AVAudioEngine-related operations would be serial and never concurrent/parallel. The audio system would never be accessed simultaneously from multiple threads, only serially.My concern is that I don't know whether it's safe to use AVAudioEngine in this way. More generally, I'm not sure what should be assumed about any API for which nothing specific is said about thread safety. In such cases, can it be assumed that access from multiple threads is safe as long as only one thread is active at any given time? (The 'Thread Programming Guide' touches on this, but doesn't appear to address audio frameworks specifically.)The narrowest version of my question is whether it's safe to use GCD with AVAudioEngine, provided all access is serial. The broader question would be what assumptions should or should not be made about APIs and thread safety when it's not specifically addressed in the documentation.Any input on either of these issues would be greatly appreciated.
Replies
2
Boosts
4
Views
3.2k
Activity
May ’22
Dismissing AVPlayerViewController doesn't 'kill' the object - it's persisting on lock screen / media controls
I am playing videos that are in my app bundle.They are playing correctly.However, when I call to dismiss the AVPlayerViewController, it visibly is removed from the view hierarchy but, if I turn off the iOS device and turn it back on again, on the lock screen there is a media control showing that video and a 'play' button.If you touch play you only get the audio and no video.My problem is I don't understand why the 'dismiss' is not completely 'killing' the player when I'm done with it.NOTE: The follow code works perfectly in iOS 13 devices, however any device running previous iOS versions, the error appears as stated. This is blocking me from submitting my app, please can someone advise on how to solve this? Here is the presentation code: internal func play(FileName filename: String, FileType type: String) { if self.isAlreadyPlaying == YES { killVideoPlayer() DispatchQueue.main.asyncAfter(deadline: .now() + 1.5, execute: { self.play(FileName: filename, FileType: type) }) return } let audioSession = AVAudioSession.sharedInstance() do { try audioSession.setCategory(AVAudioSession.Category.playback, mode: .moviePlayback, options: [.allowAirPlay, .allowBluetooth, .allowBluetoothA2DP]) try audioSession.setActive(true) } catch { print("Audio session failed") } let path = Bundle.main.path(forResource: filename, ofType: type) let url = NSURL(fileURLWithPath: path!) let player = AVPlayer(url: url as URL) NotificationCenter.default.addObserver(self, selector: #selector(PBFMenuSystemFloatingButtonsViewController.didFinishPlaying(notification:)), name: NSNotification.Name.AVPlayerItemDidPlayToEndTime, object: player.currentItem) self.playerController = AVPlayerViewController() self.menuWindow.videoPlayer = self.playerController self.playerController?.player = player self.playerController?.allowsPictureInPicturePlayback = true self.playerController?.showsPlaybackControls = YES self.playerController?.delegate = self self.playerController?.exitsFullScreenWhenPlaybackEnds = YES self.isAlreadyPlaying = YES self.present(self.playerController!, animated: true, completion : { self.playerController?.player?.play() }) } Here is the dismissal code: private func killVideoPlayer() { self.isAlreadyPlaying = NO self.playerController?.player?.pause() self.playerController?.player = nil let audioSession = AVAudioSession.sharedInstance() do { try audioSession.setActive(false, options: .notifyOthersOnDeactivation) try audioSession.setCategory(.soloAmbient) } catch { print("Audio session failed") } self.playerController?.dismiss(animated: YES, completion: { self.playerController = nil }) } And here's what's remaining in the systemwide media player that is shown on the lock screen / control centre:
Replies
1
Boosts
0
Views
3.5k
Activity
Oct ’21
Convert a CMSampleBuffer to AVAudioPCMBuffer
I'm trying to convert a CMSampleBuffer to a AVAudioPCMBuffer instance to be able to perform audio processing in realtime. I wrote an optional initialiser for my extension to pass a CMSampleBuffer reference. Unfortunately I simply don't know how to write to the AVAudioPCMBuffer's data. Here is my code so far:import AVKit extension AVAudioPCMBuffer { static func create(from sampleBuffer: CMSampleBuffer) -> AVAudioPCMBuffer? { guard let description: CMFormatDescription = CMSampleBufferGetFormatDescription(sampleBuffer), let sampleRate: Float64 = description.audioStreamBasicDescription?.mSampleRate, let numberOfChannels: Int = description.audioChannelLayout?.numberOfChannels else { return nil } guard let blockBuffer: CMBlockBuffer = CMSampleBufferGetDataBuffer(sampleBuffer) else { return nil } let length: Int = CMBlockBufferGetDataLength(blockBuffer) let audioFormat = AVAudioFormat(commonFormat: .pcmFormatFloat32, sampleRate: sampleRate, channels: AVAudioChannelCount(numberOfChannels), interleaved: false) let buffer = AVAudioPCMBuffer(pcmFormat: audioFormat!, frameCapacity: AVAudioFrameCount(length))! buffer.frameLength = buffer.frameCapacity for channelIndex in 0...numberOfChannels - 1 { guard let channel: UnsafeMutablePointer<Float> = buffer.floatChannelData?[channelIndex] else { return nil } for pointerIndex in 0...length - 1 { let pointer: UnsafeMutablePointer<Float> = channel.advanced(by: pointerIndex) pointer.pointee = 100 } } return buffer } }Does anyone knows how to convert a CMSampleBuffer to AVAudioPCMBuffer and back again? I assume there is no other way if I want to interact with AVAudioEngine, am I right?
Replies
3
Boosts
0
Views
8.0k
Activity
Oct ’22
AVAssetReaderOutput.copyNextSampleBuffer() sometimes hangs forever
I'm using AVAssetReaders with AVSampleBufferDisplayLayers to display multiple videos at once. I'm seeing this issue on iOS 13.1.3, 13.2b2, on various hardware like iPad 10.5 and iPad 12.9.It works well for a while, then a random call to copyNextSampleBuffer never returns, blocking that thread indefinitely and eating up resources.I have tried different threading approaches with no avail:If copyNextSampleBuffer() and reader.cancelReading() are done on the same queue, then copyNextSampleBuffer() gets stuck and the cancelReading() never gets processed because the queue is blocked. If I manually (with the debugger) jump in on that blocked queue and execute cancelReading(), immediately an EXC_BREAKPOINT crashes the appIf copyNextSampleBuffer() and reader.cancelReading() are done on different queues, then copyNextSampleBuffer() crashes with EXC_BAD_ACCESSHere's the stacktrace (same queue approach). I don't understand why it's stuck, my expectation is that copyNextSampleBuffer should always return (ie. with nil in error case).VideoPlayerView: UIView with AVSampleBufferDisplayLayerAVAssetFactory: Singleton with the queue that creates & manages all AVAssetReader / AVAsset* objects* thread #22, queue = 'AVAssetFactory' frame #0: 0x00000001852355f4 libsystem_kernel.dylib`mach_msg_trap + 8 frame #1: 0x0000000185234a60 libsystem_kernel.dylib`mach_msg + 72 frame #2: 0x00000001853dc068 CoreFoundation`__CFRunLoopServiceMachPort + 216 frame #3: 0x00000001853d7188 CoreFoundation`__CFRunLoopRun + 1444 frame #4: 0x00000001853d68bc CoreFoundation`CFRunLoopRunSpecific + 464 frame #5: 0x000000018f42b6ac AVFoundation`-[AVRunLoopCondition _waitInMode:untilDate:] + 400 frame #6: 0x000000018f38f1dc AVFoundation`-[AVAssetReaderOutput copyNextSampleBuffer] + 148 frame #7: 0x000000018f3900f0 AVFoundation`-[AVAssetReaderTrackOutput copyNextSampleBuffer] + 72 * frame #8: 0x0000000103309d98 Photobooth`closure #1 in AVAssetFactory.nextSampleBuffer(reader=0x00000002814016f0, retval=(Swift.Optional<CoreMedia.CMSampleBuffer>, Swift.Optional<AVFoundation.AVAssetReader.Status>) @ 0x000000016dbd1cb8) at AVAssetFactory.swift:108:34 frame #9: 0x0000000102f4f480 Photobooth`thunk for @callee_guaranteed () -> () at <compiler-generated>:0 frame #10: 0x0000000102f4f4a4 Photobooth`thunk for @escaping @callee_guaranteed () -> () at <compiler-generated>:0 frame #11: 0x000000010bfe6c04 libdispatch.dylib`_dispatch_client_callout + 16 frame #12: 0x000000010bff5888 libdispatch.dylib`_dispatch_lane_barrier_sync_invoke_and_complete + 124 frame #13: 0x0000000103309a5c Photobooth`AVAssetFactory.nextSampleBuffer(reader=0x00000002814016f0, self=0x0000000281984f60) at AVAssetFactory.swift:101:20 frame #14: 0x00000001032ab690 Photobooth`closure #1 in VideoPlayerView.setRequestMediaLoop(self=0x000000014b8da1d0, handledCompletion=false) at VideoPlayerView.swift:254:70 frame #15: 0x0000000102dce978 Photobooth`thunk for @escaping @callee_guaranteed () -> () at <compiler-generated>:0 frame #16: 0x000000018f416848 AVFoundation`-[AVMediaDataRequester _requestMediaDataIfReady] + 80 frame #17: 0x000000010bfe5828 libdispatch.dylib`_dispatch_call_block_and_release + 24 frame #18: 0x000000010bfe6c04 libdispatch.dylib`_dispatch_client_callout + 16 frame #19: 0x000000010bfedb74 libdispatch.dylib`_dispatch_lane_serial_drain + 744 frame #20: 0x000000010bfee744 libdispatch.dylib`_dispatch_lane_invoke + 500 frame #21: 0x000000010bff9ae4 libdispatch.dylib`_dispatch_workloop_worker_thread + 1324 frame #22: 0x000000018517bfa4 libsystem_pthread.dylib`_pthread_wqthread + 276I've tried all kinds of other things like making sure the AVAssets and all objects are made on one queue, and stopping the AVAssetReaders from ever deallocing to see if that helps. Nothing works. Any ideas?
Replies
4
Boosts
0
Views
3.1k
Activity
Oct ’23
Playback streaming audio with AVAudioengine (iOS)
I want to stream audio on iOS and use for that use-case the AVAudioEngine. So, currently I'm not really sure, what is the best solution for my problem.I get the RTP data from the network and want playback this audio data with AVAudioEngine. I use the iOS Network.Framework to receive the network data. Then first I decode my voice data and want to playback it, now.Here is my receive code:connection.receiveMessage { (data, context, isComplete, error) in if isComplete { // decode the raw network data with Audio codec G711 let decodedData = AudioDecoder.decode(enc: data, frames: 160) // create PCMBuffer for audio data for playback let format = AVAudioFormat(settings: [AVFormatIDKey: NSNumber(value: kAudioFormatALaw), AVSampleRateKey: 8000, AVNumberOfChannelsKey: 1]) // let format = AVAudioFormat(standardFormatWithSampleRate: 8000, channels: 1) let buffer = AVAudioPCMBuffer(pcmFormat: format!, frameCapacity: 160)! buffer.frameLength = buffer.frameCapacity // TODO: now I have to copy the decodedData --> buffer (AVAudioPCMBuffer) if error == nil { // recall receive() for next message self.receive(on: connection) } }?How I have to copy my decoded data into the AVAudioPCMBuffer? Currently, my AVAudioPCMBuffer is created, but not contain any audio data.Background information: My generell approach would be to cash here in the above code (at ToDo-Line) the PCMBuffer in a collection and playback this collection by the AVAudioEngine (in a background thread).My decoded linear data is cashed in an array from type Int16. So the var decodedData is from type [Int16], maybe there is a possibility to consume this data directly? The function scheduleBuffer allows only AVAudioPCMBuffer as input.
Replies
2
Boosts
0
Views
2.1k
Activity
Apr ’22