Dive into the technical aspects of audio on your device, including codecs, format support, and customization options.

Audio Documentation

Posts under Audio subtopic

Post

Replies

Boosts

Views

Activity

Crash in VOIP App related to AVAudioSession
Hello, We're having trouble troubleshooting a crash hoping someone can point us in the right direction. Crash Report - https://developer.apple.com/forums/content/attachment/2052e668-bc01-410b-aa60-b6ca24f8328f The crash seems to be related to activities in the app that trigger a haptic feedback (eg toggling a switch, etc), it seems consistent that we can trigger the crash by placing a voip call, ending the call, and then toggling a switch in the UI. Any advice on how to further troubleshoot would be welcome. Thanks, Ben
2
0
2.5k
Aug ’22
AVCaptureSession sometimes too slow to start on iOS 14
On iOS 14, when we have AVCaptureSession setup wit AVCaptureVideoDataOutput, AVCaptureAudioDataOutput and also RemoteIO unit configured, AVCaptureSession sometimes takes upto 3-4 seconds to start especially when also changing AVAudioSession category from playback to playAndRecord. Is this because iOS14 is overall slow and buggy in the initial release or something specific to AVFoundation?
1
1
1.4k
Oct ’22
Save tracks from AudioEngine to file
Hi all, I'm using AVAudioEngine to play multiple nodes at various times (like GarageBand for example). So far I managed to play the various files at the right time using this code : DispatchQueue.global(qos: .background).async {             AudioManager.instance.audioEngine.attach(AudioManager.instance.mixer)             AudioManager.instance.audioEngine.connect(AudioManager.instance.mixer, to: AudioManager.instance.audioEngine.outputNode, format: nil)           // !important - start the engine *before* setting up the player nodes           try! AudioManager.instance.audioEngine.start()                      for audioFile in data {             // Create and attach the audioPlayer node for this file             let audioPlayer = AVAudioPlayerNode()             AudioManager.instance.audioEngine.attach(audioPlayer)             AudioManager.instance.nodes.append(audioPlayer)             // Notice the output is the mixer in this case             AudioManager.instance.audioEngine.connect(audioPlayer, to: AudioManager.instance.mixer, format: nil)             let fileUrl = audioFile.audio.fileUrl             if let file : AVAudioFile = try? AVAudioFile.init(forReading: fileUrl) {                 let time = audioFile.start > 0 ? AudioManager.instance.secondsToAVAudioTime(hostTime: mach_absolute_time(), time: Double(audioFile.start / CGFloat.secondsToPoints)) : nil                 audioPlayer.scheduleFile(file, at: time, completionHandler: nil)                 audioPlayer.play(at: time)             }           }         } Basically my data object contains struct that have a reference to an audio fileURL and the startPosition at which it should begin. That works great. now I would like to export all these tracks mixed into a single file and save it to the Document's directory of the user. How can I achieve this? Thanks for your help.
4
0
2.6k
Dec ’21
What is error -12865 when trying to play a video?
Hello, I'm trying to play a stream video after I downloaded it using a AVAssetDownloadURLSession and aggregateAssetDownloadTask. When I initialise the AVPlayerItem with the AVURLAsset saved locally in the bookmarks it get the -12865 error. What does this code stands for? If I play the video directly from the stream it works fine. This is the code that saves the asset:     func urlSession(_ session: URLSession, task: URLSessionTask, didCompleteWithError error: Error?) {         let userDefaults = UserDefaults.standard         /*          This is the ideal place to begin downloading additional media selections          once the asset itself has finished downloading.          */         guard let task = task as? AVAggregateAssetDownloadTask,             let asset = activeDownloadsMap.removeValue(forKey: task) else { return }         guard let downloadURL = willDownloadToUrlMap.removeValue(forKey: task) else { return }         // Prepare the basic userInfo dictionary that will be posted as part of our notification.         var userInfo = [String: Any]()         userInfo[Asset.Keys.name] = asset.name         if let error = error as NSError? {             switch (error.domain, error.code) {             case (NSURLErrorDomain, NSURLErrorCancelled):                 /*                  This task was canceled, you should perform cleanup using the                  URL saved from AVAssetDownloadDelegate.urlSession(_:assetDownloadTask:didFinishDownloadingTo:).                  */                 guard let localFileLocation = localAsset(withName: asset.name)?.urlAsset.url else { return }                 do {                     try FileManager.default.removeItem(at: localFileLocation)                     userDefaults.removeObject(forKey: asset.name)                 } catch {                     print("An error occured trying to delete the contents on disk for \(asset.name): \(error)")                 }                 userInfo[Asset.Keys.downloadState] = Asset.DownloadState.notDownloaded.rawValue             case (NSURLErrorDomain, NSURLErrorUnknown):                 fatalError("Downloading HLS streams is not supported in the simulator.")             default:                 fatalError("An unexpected error occured \(error.domain)")             }         } else {             do {                 let bookmark = try downloadURL.bookmarkData()                 userDefaults.set(bookmark, forKey: asset.name)             } catch {                 print("Failed to create bookmarkData for download URL.")             }             userInfo[Asset.Keys.downloadState] = Asset.DownloadState.downloaded.rawValue             userInfo[Asset.Keys.downloadSelectionDisplayName] = ""         }         NotificationCenter.default.post(name: .AssetDownloadStateChanged, object: nil, userInfo: userInfo)     }     /// Method called when the an aggregate download task determines the location this asset will be downloaded to.     func urlSession(_ session: URLSession, aggregateAssetDownloadTask: AVAggregateAssetDownloadTask,                     willDownloadTo location: URL) {         /*          This delegate callback should only be used to save the location URL          somewhere in your application. Any additional work should be done in          `URLSessionTaskDelegate.urlSession(_:task:didCompleteWithError:)`.          */         willDownloadToUrlMap[aggregateAssetDownloadTask] = location     } and this one get's the asset back         guard let localFileLocation = userDefaults.value(forKey: name) as? Data else { return nil }                  var bookmarkDataIsStale = false         do {             let url = try URL(resolvingBookmarkData: localFileLocation,                                     bookmarkDataIsStale: &bookmarkDataIsStale)             if bookmarkDataIsStale {                 fatalError("Bookmark data is stale!")             }                          return AVURLAsset(url: url)         } catch {             fatalError("Failed to create URL from bookmark with error: \(error)")         } Thank you
2
0
1.4k
Mar ’23
Why is AVAudioEngine unable to convert the format from my bluetooth headphones Input?
I have been using AVAudioEngine to take audio from the mic and send it out over a WebRTC connection. When I use the iPhone device mic, this seems to work as expected. But if I run the app with bluetooth headphones connected, the engine reports this error when trying to start: [avae]  AVAudioEngine.mm:160   Engine@0x2833e1790: could not initialize, error = -10868 [avae]  AVAEInternal.h:109   [AVAudioEngineGraph.mm:1397:Initialize: (err = AUGraphParser::InitializeActiveNodesInInputChain(ThisGraph, *GetInputNode())): error -10868 Error starting audio engine: The operation couldn’t be completed. (com.apple.coreaudio.avfaudio error -10868.) I see that Error code -10878 is: @constant kAudioUnitErr_FormatNotSupported Returned if an input or output format is not supported ... kAudioUnitErr_FormatNotSupported = -10868 but that doesn't seem like it can be quite correct. I know that the output format is supported because the same format works correctly when my headphones are not attached. And I am pretty sure that the input format is supported because I am able to simply hook up Headphones InputNode -> Mixer -> Headphones OutputNode and correctly hear the audio from the mic. So I can only assume that this means the format conversion is not supported. My Questions: Is this a bug? Is there any way I can work around this? Notes: My full audio graph looks like this, where all the "mixers" are just AVAudioMixerNodes: // InputNode (Mic)  -> Mic Mixer -\ // 																 >-> WebRTC Mixer -> Tap -> WebRTC Framework // AudioPlayer 1 -> Player Mixer  -/ // // AudioPlayer 2 -> Player Mixer -----> LocalOutputMixer -> OutputNode (Device Speakers/Headphones) but the issue still happens even if I simplify down to this: InputNode (Mic)  ->	Mixer -> Tap -> WebRTC Framework Specifically it happens when a single mixer node is connected with an input format and output format as follows: The input format is: (lldb) po audioEngine.inputNode.inputFormat(forBus: 0).streamDescription.pointee ▿ AudioStreamBasicDescription 	- mSampleRate : 16000.0 	- mFormatID : 1819304813 	- mFormatFlags : 41 	- mBytesPerPacket : 4 	- mFramesPerPacket : 1 	- mBytesPerFrame : 4 	- mChannelsPerFrame : 1 	- mBitsPerChannel : 32 	- mReserved : 0 The output format WebRTC expects is: ▿ AudioStreamBasicDescription 	- mSampleRate : 48000.0 	- mFormatID : 1819304813 	- mFormatFlags : 12 	- mBytesPerPacket : 2 	- mFramesPerPacket : 1 	- mBytesPerFrame : 2 	- mChannelsPerFrame : 1 	- mBitsPerChannel : 16 	- mReserved : 0 My headphones are Jaybird Freedom 2.
3
0
3.4k
Dec ’21
Picture in Picture stops audio when phone locked
AVPlayerViewController has the functionality to keep playing the audio when the app is backgrounded or the device is locked. Obviously picture in picture improves this when the app is backgrounded but we lose the functionality when the device is locked as it stops the audio. The user can hit play on the lock-screen to continue playing but doesn't seem to be ideal. Is there any way around this or is the expected behaviour?
3
0
3.3k
May ’23
AVQueuePlayer.remove stutter
I am trying to play an audio clip in a loop until a different audioclip gets selected. When that happens it should wait for the current audio clip to finish and continue looping with the newly selected clip (gapless). These audio clips were made to be played in loops and in succession. However it's important that the next clip doesn't start until some condition has been met. I've tried using AVQueuePlayer which I got working quite well except for one thing, which is that remove(_ item: AVPlayerItem) seems to add a "stutter" to the currently playing clip. Almost like it's buffering or something. Which is strange because all audio files are embedded in the project. The clips are quite short (between 1 and 2 seconds). How can I get this to work without a stutter? Or maybe using something entirely different even? Update: I also tried using AVPlayerLooper, which works great with 1 clip but I can't tell it to continue looping using another clip (without a gap).
4
0
1.1k
Aug ’21
The AudioUnit user presets system has ceased to work as expected in iPadOS 14
The AUAudioUnit in my app uses the user presets system that was introduced with iPadOS/iOS 13. The user presets system worked well in iPadOS 13, and it worked in at least one of the pre-release beta versions of iPadOS 14. However, it has not worked since iPadOS 14.0 was released. The problem is (superficially, at least) that: saveUserPreset no longer writes preset files to the app's Documents folder, but the method doesn't fail either. The userPresets array, which typically contains the presets that were deserialised from the files stored in the Documents folder, is invariably empty. The presets that the array would contain prior to upgrading from iPadOS 13 are no longer accessible. In other words, prior to upgrading to iPadOS 14, my app could save user presets using the saveUserPreset method and subsequently access their contents from the userPresets array. Since upgrading to iPadOS 14, the userPresets array is invariably empty and the saveUserPreset method no longer saves presets to the Documents folder. Has anyone else experienced this problem?
1
0
1.3k
Jun ’22
AVPlayerItem fails with -12889 CoreMediaErrorDomain "No response for media file in 4.096s"
Hi there, We are experiencing intermittent HLS playback failures with the following AVPlayerItemErrorLogEvent: ("https://somewhere.com/ausw/slices/156/e6cf0c55dac249f0a0f72e7c72e6f6cb/1560e743f9d441309eab162aa0c1c95d/H000000E6.ts?pbs=0c1c1ada7e1540dd80c41397753da0d2&cloud=aws&si=0") nil Optional("E0186CE2-D6D7-46A9-8DD8-E92D38F2C64F") -12889 CoreMediaErrorDomain Optional("No response for media file in 4.096s")` This typically occurs when seeking and can be reproduced by repeatedly skipping +10s forward through a video stream. Immediately prior to the playback failure we see a sequence of cancelled (“IO: Stream cancelled by CLIENT”) segment HTTP requests whose durations go from a “normal” few hundred ms to 10.01s which I interpret to be a timeout. The final request appears to retry and timeout ad infinitum. After the error the AVPlayer instance is broken and subsequent attempts to play a new AVPlayerItem instance result in the error immediately recurring. This issue affects iOS 14 and 13, but not iOS 12 or 11. Any suggestions on how to diagnose or fix this are appreciated! Many thanks /R
2
1
4.3k
Nov ’22
How to play .caf file
I tried to change my code from let urlString = Bundle.main.path(forResource: "X", ofType: "mp3")! to let urlString = Bundle.main.path(forResource: "X", ofType: "caf")! after I already uploaded the file. But it throws a fatal error: Unexpectedly found nil while unwrapping an Optional value I double check the filename and it's correct. and the caf file does play.
1
0
2.8k
Sep ’22
iOS 14 - HLS metadata not showing user defined URL frames within id3 tag
Hello, I am using HLS audio stream and I get metadata to show the song info, artist info etc., in the app using AVPlayer. In iOS 14, I see the artwork data (which embeds in WXXX frame of id3 tag) is coming over across along with other metadata. The same stream is working fine and showing artwork data in iOS 13. Does anything changed in iOS 14 which effects this metadata? Please let me know Thank you
4
0
1.8k
Jul ’23
AVAssetWriter leading to huge kernel_task memory usage
I am currently working on a macOS app which will be creating very large video files with up to an hour of content. However, generating the images and adding them to AVAssetWriter leads to VTDecoderXPCService using 16+ GB of memory and the kernel-task using 40+ GB (the max I saw was 105GB). It seems like the generated video is not streamed onto the disk but rather written to memory for it to be written to disk all at once. How can I force it to stream the data to disk while the encoding is happening? Btw. my app itself consistently needs around 300MB of memory, so I don't think I have a memory leak here. Here is the relevant code: func analyse() { 				self.videoWritter = try! AVAssetWriter(outputURL: outputVideo, fileType: AVFileType.mp4) 				let writeSettings: [String: Any] = [ 						AVVideoCodecKey: AVVideoCodecType.h264, 						AVVideoWidthKey: videoSize.width, 						AVVideoHeightKey: videoSize.height, 						AVVideoCompressionPropertiesKey: [ 								AVVideoAverageBitRateKey: 10_000_000, 						] 				] 				self.videoWritter!.movieFragmentInterval = CMTimeMake(value: 60, timescale: 1) 				self.frameInput = AVAssetWriterInput(mediaType: AVMediaType.video, outputSettings: writeSettings) 				self.frameInput?.expectsMediaDataInRealTime = true 				self.videoWritter!.add(self.frameInput!) 				if self.videoWritter!.startWriting() == false { 						print("Could not write file: \(self.videoWritter!.error!)") 						return 				} } func writeFrame(frame: Frame) { 				/* some more code to determine the correct frame presentation time stamp */ 				let newSampleBuffer = self.setTimeStamp(frame.sampleBuffer, newTime: self.nextFrameStartTime!) 				self.frameInput!.append(newSampleBuffer) 				/* some more code */ } func setTimeStamp(_ sample: CMSampleBuffer, newTime: CMTime) -> CMSampleBuffer { 				var timing: CMSampleTimingInfo = CMSampleTimingInfo( 						duration: CMTime.zero, 						presentationTimeStamp: newTime, 						decodeTimeStamp: CMTime.zero 				) 				var newSampleBuffer: CMSampleBuffer? 				CMSampleBufferCreateCopyWithNewTiming( 						allocator: kCFAllocatorDefault, 					 sampleBuffer: sample, 					 sampleTimingEntryCount: 1, 					 sampleTimingArray: &timing, 					 sampleBufferOut: &newSampleBuffer 			 ) 				return	newSampleBuffer! 		} My specs: MacBook Pro 2018 32GB Memory macOS Big Sur 11.1
5
0
2.9k
Nov ’23
AVMIDIPlayer not working properly
AVMIDIPlayer not working properly. Since iOS 14 is it running just a while then begin to stutter and stammer and then silence. Some very simple midi file are running OK but most of them stopped, especially if it contains some crowd of tunes like glissando. There is no crash message, the app in which I’m using AVMIDIPlayer is running father, but with no sound, as if that AVMIDIPlayer were constipated. This happens only on my devices iPhone Xr and iPad 5th gen., in simulator no problem.
3
0
762
May ’22
Adding HTTP headers to a AVURLAsset
Hi, for the implementation of an audio player with signed URL's, I need to be able to set an authorization header to the request for an AVURLAsset. This works but not on Airplay when trying to stream multiple songs in a queue. For each item I do: let headerFields: [String: String] = ["Authorization": getIdToken()!] super.init(url: url, options: ["AVURLAssetHTTPHeaderFieldsKey": headerFields]) But only the first 2 songs in the queue actually get this authorization header sent along, somehow it is removed for subsequent songs. Any ideas on how I can fix this? thanks, Thomas
1
1
1.8k
Oct ’24
Change AudioUnit from VPIO to RemoteIO occurs the volume was reduced (iOS 14 only)
When I change audio unit from VPIO(VoiceProcessingIO) to remoteIO, the volume was reduced and can not restore to normal level. There's a workaround by this post: h ttps://trac.pjsip.org/repos/ticket/1697 I use the workaround in my code and according flowing steps: stop and destory all the running IO AudioUnit (remoteIO and VPIO) run workaround code [[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryRecord error:nil]; [[AVAudioSession sharedInstance] setActive:NO error:nil]; [[AVAudioSession sharedInstance] setActive:YES error:nil]; [[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayAndRecord error:nil]; 3. setup remoteIO and run. and it work perfectly! The volume of remoteIO was go to normally. But sadness happens on iOS 14, the workaround is not work at all.
1
0
2.2k
Feb ’22
AVAssetWriter fail in background
We are using AVAssetWriter to write a movie file. All works fine, but when the app goes into the background and pauses operations, we would expect the writer to come back to life when you bring the app back into the foreground. However, it does not, it fails with this error code. Any hints, workarounds? Error Domain=AVFoundationErrorDomain Code=-11847 "Operation Interrupted" UserInfo={NSLocalizedRecoverySuggestion=Stop other operations and try again., NSLocalizedDescription=Operation Interrupted, NSUnderlyingError=0x2820356b0 {Error Domain=NSOSStatusErrorDomain Code=-16121 "(null)"}}
3
1
2.4k
Sep ’21
AVCaptureMultiCamSession Runtime Error with External headphone
Hi Apple support, I'm using AVMultiCamPip as a project starting base and I have the following setup(sample[1]). It connects without headphones but generate a runtime error(sample[1]) with headphones. I'm running on iPhone Xs with iOS 14.3 The source code I'm using is the sample AVMultiCamPip code. How should I config the external headphone mic to prevent the runtime error to occur? sample[1] 2021-01-23 11:10:54.216702-0500 AVMultiCamPiP[3760:649414] Metal GPU Frame Capture Enabled 2021-01-23 11:10:54.218037-0500 AVMultiCamPiP[3760:649414] Metal API Validation Enabled (lldb) po (notification.object as! AVCaptureMultiCamSession).connections ▿ 6 elements &#9;- 0 : <AVCaptureConnection: 0x282092f80 (AVCaptureDeviceInput: 0x282081280 Back Camera) -> (AVCaptureVideoDataOutput: 0x2820f7840) [type:vide][enabled:1][active:1]> &#9;- 1 : <AVCaptureConnection: 0x282082ba0 (AVCaptureDeviceInput: 0x282081280 Back Camera) -> (AVCaptureVideoPreviewLayer: 0x2820e8060) [type:vide][enabled:1][active:1]> &#9;- 2 : <AVCaptureConnection: 0x282085220 (AVCaptureDeviceInput: 0x282081920 Front Camera) -> (AVCaptureVideoDataOutput: 0x2820f7a80) [type:vide][enabled:1][active:1]> &#9;- 3 : <AVCaptureConnection: 0x2820939e0 (AVCaptureDeviceInput: 0x282081920 Front Camera) -> (AVCaptureVideoPreviewLayer: 0x2820e8200) [type:vide][enabled:1][active:1]> &#9;- 4 : <AVCaptureConnection: 0x28208d460 (AVCaptureDeviceInput: 0x2820843e0 Headphones) -> (AVCaptureAudioDataOutput: 0x2820f7ba0) [type:soun][enabled:1][active:1]> &#9;- 5 : <AVCaptureConnection: 0x28208e480 (AVCaptureDeviceInput: 0x2820843e0 Headphones) -> (AVCaptureAudioDataOutput: 0x2820f7d20) [type:soun][enabled:1][active:1]> (lldb) po (notification.object as! AVCaptureMultiCamSession).hardwareCost 0.58957714 (lldb) po (notification.object as! AVCaptureMultiCamSession).systemPressureCost 0.81041515 (lldb) po (notification.object as! AVCaptureMultiCamSession).outputs ▿ 4 elements &#9;- 0 : <AVCaptureVideoDataOutput: 0x2820f7840> &#9;- 1 : <AVCaptureVideoDataOutput: 0x2820f7a80> &#9;- 2 : <AVCaptureAudioDataOutput: 0x2820f7ba0> &#9;- 3 : <AVCaptureAudioDataOutput: 0x2820f7d20> (lldb) po (notification.object as! AVCaptureMultiCamSession).inputs ▿ 3 elements &#9;- 0 : <AVCaptureDeviceInput: 0x282081280 [Back Camera]> &#9;- 1 : <AVCaptureDeviceInput: 0x282081920 [Front Camera]> &#9;- 2 : <AVCaptureDeviceInput: 0x2820843e0 [Headphones]> (lldb) po notification.userInfo ▿ Optional<Dictionary<AnyHashable, Any>> &#9;▿ some : 1 element &#9;&#9;▿ 0 : 2 elements &#9;&#9;&#9;▿ key : AnyHashable("AVCaptureSessionErrorKey") &#9;&#9;&#9;&#9;- value : "AVCaptureSessionErrorKey" &#9;&#9;&#9;- value : Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedFailureReason=An unknown error occurred (-12780), NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x282e81200 {Error Domain=NSOSStatusErrorDomain Code=-12780 "(null)"}} (lldb)
1
0
1.3k
Aug ’21
Crash in VOIP App related to AVAudioSession
Hello, We're having trouble troubleshooting a crash hoping someone can point us in the right direction. Crash Report - https://developer.apple.com/forums/content/attachment/2052e668-bc01-410b-aa60-b6ca24f8328f The crash seems to be related to activities in the app that trigger a haptic feedback (eg toggling a switch, etc), it seems consistent that we can trigger the crash by placing a voip call, ending the call, and then toggling a switch in the UI. Any advice on how to further troubleshoot would be welcome. Thanks, Ben
Replies
2
Boosts
0
Views
2.5k
Activity
Aug ’22
AVCaptureSession sometimes too slow to start on iOS 14
On iOS 14, when we have AVCaptureSession setup wit AVCaptureVideoDataOutput, AVCaptureAudioDataOutput and also RemoteIO unit configured, AVCaptureSession sometimes takes upto 3-4 seconds to start especially when also changing AVAudioSession category from playback to playAndRecord. Is this because iOS14 is overall slow and buggy in the initial release or something specific to AVFoundation?
Replies
1
Boosts
1
Views
1.4k
Activity
Oct ’22
Save tracks from AudioEngine to file
Hi all, I'm using AVAudioEngine to play multiple nodes at various times (like GarageBand for example). So far I managed to play the various files at the right time using this code : DispatchQueue.global(qos: .background).async {             AudioManager.instance.audioEngine.attach(AudioManager.instance.mixer)             AudioManager.instance.audioEngine.connect(AudioManager.instance.mixer, to: AudioManager.instance.audioEngine.outputNode, format: nil)           // !important - start the engine *before* setting up the player nodes           try! AudioManager.instance.audioEngine.start()                      for audioFile in data {             // Create and attach the audioPlayer node for this file             let audioPlayer = AVAudioPlayerNode()             AudioManager.instance.audioEngine.attach(audioPlayer)             AudioManager.instance.nodes.append(audioPlayer)             // Notice the output is the mixer in this case             AudioManager.instance.audioEngine.connect(audioPlayer, to: AudioManager.instance.mixer, format: nil)             let fileUrl = audioFile.audio.fileUrl             if let file : AVAudioFile = try? AVAudioFile.init(forReading: fileUrl) {                 let time = audioFile.start > 0 ? AudioManager.instance.secondsToAVAudioTime(hostTime: mach_absolute_time(), time: Double(audioFile.start / CGFloat.secondsToPoints)) : nil                 audioPlayer.scheduleFile(file, at: time, completionHandler: nil)                 audioPlayer.play(at: time)             }           }         } Basically my data object contains struct that have a reference to an audio fileURL and the startPosition at which it should begin. That works great. now I would like to export all these tracks mixed into a single file and save it to the Document's directory of the user. How can I achieve this? Thanks for your help.
Replies
4
Boosts
0
Views
2.6k
Activity
Dec ’21
What is error -12865 when trying to play a video?
Hello, I'm trying to play a stream video after I downloaded it using a AVAssetDownloadURLSession and aggregateAssetDownloadTask. When I initialise the AVPlayerItem with the AVURLAsset saved locally in the bookmarks it get the -12865 error. What does this code stands for? If I play the video directly from the stream it works fine. This is the code that saves the asset:     func urlSession(_ session: URLSession, task: URLSessionTask, didCompleteWithError error: Error?) {         let userDefaults = UserDefaults.standard         /*          This is the ideal place to begin downloading additional media selections          once the asset itself has finished downloading.          */         guard let task = task as? AVAggregateAssetDownloadTask,             let asset = activeDownloadsMap.removeValue(forKey: task) else { return }         guard let downloadURL = willDownloadToUrlMap.removeValue(forKey: task) else { return }         // Prepare the basic userInfo dictionary that will be posted as part of our notification.         var userInfo = [String: Any]()         userInfo[Asset.Keys.name] = asset.name         if let error = error as NSError? {             switch (error.domain, error.code) {             case (NSURLErrorDomain, NSURLErrorCancelled):                 /*                  This task was canceled, you should perform cleanup using the                  URL saved from AVAssetDownloadDelegate.urlSession(_:assetDownloadTask:didFinishDownloadingTo:).                  */                 guard let localFileLocation = localAsset(withName: asset.name)?.urlAsset.url else { return }                 do {                     try FileManager.default.removeItem(at: localFileLocation)                     userDefaults.removeObject(forKey: asset.name)                 } catch {                     print("An error occured trying to delete the contents on disk for \(asset.name): \(error)")                 }                 userInfo[Asset.Keys.downloadState] = Asset.DownloadState.notDownloaded.rawValue             case (NSURLErrorDomain, NSURLErrorUnknown):                 fatalError("Downloading HLS streams is not supported in the simulator.")             default:                 fatalError("An unexpected error occured \(error.domain)")             }         } else {             do {                 let bookmark = try downloadURL.bookmarkData()                 userDefaults.set(bookmark, forKey: asset.name)             } catch {                 print("Failed to create bookmarkData for download URL.")             }             userInfo[Asset.Keys.downloadState] = Asset.DownloadState.downloaded.rawValue             userInfo[Asset.Keys.downloadSelectionDisplayName] = ""         }         NotificationCenter.default.post(name: .AssetDownloadStateChanged, object: nil, userInfo: userInfo)     }     /// Method called when the an aggregate download task determines the location this asset will be downloaded to.     func urlSession(_ session: URLSession, aggregateAssetDownloadTask: AVAggregateAssetDownloadTask,                     willDownloadTo location: URL) {         /*          This delegate callback should only be used to save the location URL          somewhere in your application. Any additional work should be done in          `URLSessionTaskDelegate.urlSession(_:task:didCompleteWithError:)`.          */         willDownloadToUrlMap[aggregateAssetDownloadTask] = location     } and this one get's the asset back         guard let localFileLocation = userDefaults.value(forKey: name) as? Data else { return nil }                  var bookmarkDataIsStale = false         do {             let url = try URL(resolvingBookmarkData: localFileLocation,                                     bookmarkDataIsStale: &bookmarkDataIsStale)             if bookmarkDataIsStale {                 fatalError("Bookmark data is stale!")             }                          return AVURLAsset(url: url)         } catch {             fatalError("Failed to create URL from bookmark with error: \(error)")         } Thank you
Replies
2
Boosts
0
Views
1.4k
Activity
Mar ’23
Why is AVAudioEngine unable to convert the format from my bluetooth headphones Input?
I have been using AVAudioEngine to take audio from the mic and send it out over a WebRTC connection. When I use the iPhone device mic, this seems to work as expected. But if I run the app with bluetooth headphones connected, the engine reports this error when trying to start: [avae]  AVAudioEngine.mm:160   Engine@0x2833e1790: could not initialize, error = -10868 [avae]  AVAEInternal.h:109   [AVAudioEngineGraph.mm:1397:Initialize: (err = AUGraphParser::InitializeActiveNodesInInputChain(ThisGraph, *GetInputNode())): error -10868 Error starting audio engine: The operation couldn’t be completed. (com.apple.coreaudio.avfaudio error -10868.) I see that Error code -10878 is: @constant kAudioUnitErr_FormatNotSupported Returned if an input or output format is not supported ... kAudioUnitErr_FormatNotSupported = -10868 but that doesn't seem like it can be quite correct. I know that the output format is supported because the same format works correctly when my headphones are not attached. And I am pretty sure that the input format is supported because I am able to simply hook up Headphones InputNode -> Mixer -> Headphones OutputNode and correctly hear the audio from the mic. So I can only assume that this means the format conversion is not supported. My Questions: Is this a bug? Is there any way I can work around this? Notes: My full audio graph looks like this, where all the "mixers" are just AVAudioMixerNodes: // InputNode (Mic)  -> Mic Mixer -\ // &#9;&#9;&#9;&#9;&#9;&#9;&#9;&#9;&#9;&#9;&#9;&#9;&#9;&#9;&#9;&#9; >-> WebRTC Mixer -> Tap -> WebRTC Framework // AudioPlayer 1 -> Player Mixer  -/ // // AudioPlayer 2 -> Player Mixer -----> LocalOutputMixer -> OutputNode (Device Speakers/Headphones) but the issue still happens even if I simplify down to this: InputNode (Mic)  ->&#9;Mixer -> Tap -> WebRTC Framework Specifically it happens when a single mixer node is connected with an input format and output format as follows: The input format is: (lldb) po audioEngine.inputNode.inputFormat(forBus: 0).streamDescription.pointee ▿ AudioStreamBasicDescription &#9;- mSampleRate : 16000.0 &#9;- mFormatID : 1819304813 &#9;- mFormatFlags : 41 &#9;- mBytesPerPacket : 4 &#9;- mFramesPerPacket : 1 &#9;- mBytesPerFrame : 4 &#9;- mChannelsPerFrame : 1 &#9;- mBitsPerChannel : 32 &#9;- mReserved : 0 The output format WebRTC expects is: ▿ AudioStreamBasicDescription &#9;- mSampleRate : 48000.0 &#9;- mFormatID : 1819304813 &#9;- mFormatFlags : 12 &#9;- mBytesPerPacket : 2 &#9;- mFramesPerPacket : 1 &#9;- mBytesPerFrame : 2 &#9;- mChannelsPerFrame : 1 &#9;- mBitsPerChannel : 16 &#9;- mReserved : 0 My headphones are Jaybird Freedom 2.
Replies
3
Boosts
0
Views
3.4k
Activity
Dec ’21
Picture in Picture stops audio when phone locked
AVPlayerViewController has the functionality to keep playing the audio when the app is backgrounded or the device is locked. Obviously picture in picture improves this when the app is backgrounded but we lose the functionality when the device is locked as it stops the audio. The user can hit play on the lock-screen to continue playing but doesn't seem to be ideal. Is there any way around this or is the expected behaviour?
Replies
3
Boosts
0
Views
3.3k
Activity
May ’23
AVQueuePlayer.remove stutter
I am trying to play an audio clip in a loop until a different audioclip gets selected. When that happens it should wait for the current audio clip to finish and continue looping with the newly selected clip (gapless). These audio clips were made to be played in loops and in succession. However it's important that the next clip doesn't start until some condition has been met. I've tried using AVQueuePlayer which I got working quite well except for one thing, which is that remove(_ item: AVPlayerItem) seems to add a "stutter" to the currently playing clip. Almost like it's buffering or something. Which is strange because all audio files are embedded in the project. The clips are quite short (between 1 and 2 seconds). How can I get this to work without a stutter? Or maybe using something entirely different even? Update: I also tried using AVPlayerLooper, which works great with 1 clip but I can't tell it to continue looping using another clip (without a gap).
Replies
4
Boosts
0
Views
1.1k
Activity
Aug ’21
The AudioUnit user presets system has ceased to work as expected in iPadOS 14
The AUAudioUnit in my app uses the user presets system that was introduced with iPadOS/iOS 13. The user presets system worked well in iPadOS 13, and it worked in at least one of the pre-release beta versions of iPadOS 14. However, it has not worked since iPadOS 14.0 was released. The problem is (superficially, at least) that: saveUserPreset no longer writes preset files to the app's Documents folder, but the method doesn't fail either. The userPresets array, which typically contains the presets that were deserialised from the files stored in the Documents folder, is invariably empty. The presets that the array would contain prior to upgrading from iPadOS 13 are no longer accessible. In other words, prior to upgrading to iPadOS 14, my app could save user presets using the saveUserPreset method and subsequently access their contents from the userPresets array. Since upgrading to iPadOS 14, the userPresets array is invariably empty and the saveUserPreset method no longer saves presets to the Documents folder. Has anyone else experienced this problem?
Replies
1
Boosts
0
Views
1.3k
Activity
Jun ’22
What is the max size of X-Playback-Session-Id
Hi, We have a question on X-Playback-Session-Id which AV player passes on each player request call. Is there any documentation to refer to check what is the max size of the field? Any pointers will be helpful. Thanks
Replies
1
Boosts
0
Views
1.4k
Activity
Oct ’21
AVPlayerItem fails with -12889 CoreMediaErrorDomain "No response for media file in 4.096s"
Hi there, We are experiencing intermittent HLS playback failures with the following AVPlayerItemErrorLogEvent: ("https://somewhere.com/ausw/slices/156/e6cf0c55dac249f0a0f72e7c72e6f6cb/1560e743f9d441309eab162aa0c1c95d/H000000E6.ts?pbs=0c1c1ada7e1540dd80c41397753da0d2&cloud=aws&si=0") nil Optional("E0186CE2-D6D7-46A9-8DD8-E92D38F2C64F") -12889 CoreMediaErrorDomain Optional("No response for media file in 4.096s")` This typically occurs when seeking and can be reproduced by repeatedly skipping +10s forward through a video stream. Immediately prior to the playback failure we see a sequence of cancelled (“IO: Stream cancelled by CLIENT”) segment HTTP requests whose durations go from a “normal” few hundred ms to 10.01s which I interpret to be a timeout. The final request appears to retry and timeout ad infinitum. After the error the AVPlayer instance is broken and subsequent attempts to play a new AVPlayerItem instance result in the error immediately recurring. This issue affects iOS 14 and 13, but not iOS 12 or 11. Any suggestions on how to diagnose or fix this are appreciated! Many thanks /R
Replies
2
Boosts
1
Views
4.3k
Activity
Nov ’22
capturesession set camera out resolution on Mac, but it seems failed
i try to set camera output resolution on Mac, but it seems failed my code as below if ([self.captureSession canSetSessionPreset:AVCaptureSessionPreset640x480]){    [[self captureSession] setSessionPreset:AVCaptureSessionPreset640x480]; } when i debug, the preset is set successfully. but the video output is till 1280*720
Replies
3
Boosts
0
Views
688
Activity
Apr ’22
How to play .caf file
I tried to change my code from let urlString = Bundle.main.path(forResource: "X", ofType: "mp3")! to let urlString = Bundle.main.path(forResource: "X", ofType: "caf")! after I already uploaded the file. But it throws a fatal error: Unexpectedly found nil while unwrapping an Optional value I double check the filename and it's correct. and the caf file does play.
Replies
1
Boosts
0
Views
2.8k
Activity
Sep ’22
iOS 14 - HLS metadata not showing user defined URL frames within id3 tag
Hello, I am using HLS audio stream and I get metadata to show the song info, artist info etc., in the app using AVPlayer. In iOS 14, I see the artwork data (which embeds in WXXX frame of id3 tag) is coming over across along with other metadata. The same stream is working fine and showing artwork data in iOS 13. Does anything changed in iOS 14 which effects this metadata? Please let me know Thank you
Replies
4
Boosts
0
Views
1.8k
Activity
Jul ’23
AVAssetWriter leading to huge kernel_task memory usage
I am currently working on a macOS app which will be creating very large video files with up to an hour of content. However, generating the images and adding them to AVAssetWriter leads to VTDecoderXPCService using 16+ GB of memory and the kernel-task using 40+ GB (the max I saw was 105GB). It seems like the generated video is not streamed onto the disk but rather written to memory for it to be written to disk all at once. How can I force it to stream the data to disk while the encoding is happening? Btw. my app itself consistently needs around 300MB of memory, so I don't think I have a memory leak here. Here is the relevant code: func analyse() { &#9;&#9;&#9;&#9;self.videoWritter = try! AVAssetWriter(outputURL: outputVideo, fileType: AVFileType.mp4) &#9;&#9;&#9;&#9;let writeSettings: [String: Any] = [ &#9;&#9;&#9;&#9;&#9;&#9;AVVideoCodecKey: AVVideoCodecType.h264, &#9;&#9;&#9;&#9;&#9;&#9;AVVideoWidthKey: videoSize.width, &#9;&#9;&#9;&#9;&#9;&#9;AVVideoHeightKey: videoSize.height, &#9;&#9;&#9;&#9;&#9;&#9;AVVideoCompressionPropertiesKey: [ &#9;&#9;&#9;&#9;&#9;&#9;&#9;&#9;AVVideoAverageBitRateKey: 10_000_000, &#9;&#9;&#9;&#9;&#9;&#9;] &#9;&#9;&#9;&#9;] &#9;&#9;&#9;&#9;self.videoWritter!.movieFragmentInterval = CMTimeMake(value: 60, timescale: 1) &#9;&#9;&#9;&#9;self.frameInput = AVAssetWriterInput(mediaType: AVMediaType.video, outputSettings: writeSettings) &#9;&#9;&#9;&#9;self.frameInput?.expectsMediaDataInRealTime = true &#9;&#9;&#9;&#9;self.videoWritter!.add(self.frameInput!) &#9;&#9;&#9;&#9;if self.videoWritter!.startWriting() == false { &#9;&#9;&#9;&#9;&#9;&#9;print("Could not write file: \(self.videoWritter!.error!)") &#9;&#9;&#9;&#9;&#9;&#9;return &#9;&#9;&#9;&#9;} } func writeFrame(frame: Frame) { &#9;&#9;&#9;&#9;/* some more code to determine the correct frame presentation time stamp */ &#9;&#9;&#9;&#9;let newSampleBuffer = self.setTimeStamp(frame.sampleBuffer, newTime: self.nextFrameStartTime!) &#9;&#9;&#9;&#9;self.frameInput!.append(newSampleBuffer) &#9;&#9;&#9;&#9;/* some more code */ } func setTimeStamp(_ sample: CMSampleBuffer, newTime: CMTime) -> CMSampleBuffer { &#9;&#9;&#9;&#9;var timing: CMSampleTimingInfo = CMSampleTimingInfo( &#9;&#9;&#9;&#9;&#9;&#9;duration: CMTime.zero, &#9;&#9;&#9;&#9;&#9;&#9;presentationTimeStamp: newTime, &#9;&#9;&#9;&#9;&#9;&#9;decodeTimeStamp: CMTime.zero &#9;&#9;&#9;&#9;) &#9;&#9;&#9;&#9;var newSampleBuffer: CMSampleBuffer? &#9;&#9;&#9;&#9;CMSampleBufferCreateCopyWithNewTiming( &#9;&#9;&#9;&#9;&#9;&#9;allocator: kCFAllocatorDefault, &#9;&#9;&#9;&#9;&#9; sampleBuffer: sample, &#9;&#9;&#9;&#9;&#9; sampleTimingEntryCount: 1, &#9;&#9;&#9;&#9;&#9; sampleTimingArray: &timing, &#9;&#9;&#9;&#9;&#9; sampleBufferOut: &newSampleBuffer &#9;&#9;&#9; ) &#9;&#9;&#9;&#9;return&#9;newSampleBuffer! &#9;&#9;} My specs: MacBook Pro 2018 32GB Memory macOS Big Sur 11.1
Replies
5
Boosts
0
Views
2.9k
Activity
Nov ’23
Avplayer issue playing video audio from speakers in ios 14.2.1
Hi guys, We implement Avplayer in our app - but it seems to have this odd problem where the audio does not play from the speakers on ios 14.2.1 Seems to work fine on 14.2 Is there some setting or configuration that we can use to get around this? Many thanks
Replies
2
Boosts
0
Views
2.8k
Activity
Dec ’21
AVMIDIPlayer not working properly
AVMIDIPlayer not working properly. Since iOS 14 is it running just a while then begin to stutter and stammer and then silence. Some very simple midi file are running OK but most of them stopped, especially if it contains some crowd of tunes like glissando. There is no crash message, the app in which I’m using AVMIDIPlayer is running father, but with no sound, as if that AVMIDIPlayer were constipated. This happens only on my devices iPhone Xr and iPad 5th gen., in simulator no problem.
Replies
3
Boosts
0
Views
762
Activity
May ’22
Adding HTTP headers to a AVURLAsset
Hi, for the implementation of an audio player with signed URL's, I need to be able to set an authorization header to the request for an AVURLAsset. This works but not on Airplay when trying to stream multiple songs in a queue. For each item I do: let headerFields: [String: String] = ["Authorization": getIdToken()!] super.init(url: url, options: ["AVURLAssetHTTPHeaderFieldsKey": headerFields]) But only the first 2 songs in the queue actually get this authorization header sent along, somehow it is removed for subsequent songs. Any ideas on how I can fix this? thanks, Thomas
Replies
1
Boosts
1
Views
1.8k
Activity
Oct ’24
Change AudioUnit from VPIO to RemoteIO occurs the volume was reduced (iOS 14 only)
When I change audio unit from VPIO(VoiceProcessingIO) to remoteIO, the volume was reduced and can not restore to normal level. There's a workaround by this post: h ttps://trac.pjsip.org/repos/ticket/1697 I use the workaround in my code and according flowing steps: stop and destory all the running IO AudioUnit (remoteIO and VPIO) run workaround code [[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryRecord error:nil]; [[AVAudioSession sharedInstance] setActive:NO error:nil]; [[AVAudioSession sharedInstance] setActive:YES error:nil]; [[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayAndRecord error:nil]; 3. setup remoteIO and run. and it work perfectly! The volume of remoteIO was go to normally. But sadness happens on iOS 14, the workaround is not work at all.
Replies
1
Boosts
0
Views
2.2k
Activity
Feb ’22
AVAssetWriter fail in background
We are using AVAssetWriter to write a movie file. All works fine, but when the app goes into the background and pauses operations, we would expect the writer to come back to life when you bring the app back into the foreground. However, it does not, it fails with this error code. Any hints, workarounds? Error Domain=AVFoundationErrorDomain Code=-11847 "Operation Interrupted" UserInfo={NSLocalizedRecoverySuggestion=Stop other operations and try again., NSLocalizedDescription=Operation Interrupted, NSUnderlyingError=0x2820356b0 {Error Domain=NSOSStatusErrorDomain Code=-16121 "(null)"}}
Replies
3
Boosts
1
Views
2.4k
Activity
Sep ’21
AVCaptureMultiCamSession Runtime Error with External headphone
Hi Apple support, I'm using AVMultiCamPip as a project starting base and I have the following setup(sample[1]). It connects without headphones but generate a runtime error(sample[1]) with headphones. I'm running on iPhone Xs with iOS 14.3 The source code I'm using is the sample AVMultiCamPip code. How should I config the external headphone mic to prevent the runtime error to occur? sample[1] 2021-01-23 11:10:54.216702-0500 AVMultiCamPiP[3760:649414] Metal GPU Frame Capture Enabled 2021-01-23 11:10:54.218037-0500 AVMultiCamPiP[3760:649414] Metal API Validation Enabled (lldb) po (notification.object as! AVCaptureMultiCamSession).connections ▿ 6 elements &#9;- 0 : <AVCaptureConnection: 0x282092f80 (AVCaptureDeviceInput: 0x282081280 Back Camera) -> (AVCaptureVideoDataOutput: 0x2820f7840) [type:vide][enabled:1][active:1]> &#9;- 1 : <AVCaptureConnection: 0x282082ba0 (AVCaptureDeviceInput: 0x282081280 Back Camera) -> (AVCaptureVideoPreviewLayer: 0x2820e8060) [type:vide][enabled:1][active:1]> &#9;- 2 : <AVCaptureConnection: 0x282085220 (AVCaptureDeviceInput: 0x282081920 Front Camera) -> (AVCaptureVideoDataOutput: 0x2820f7a80) [type:vide][enabled:1][active:1]> &#9;- 3 : <AVCaptureConnection: 0x2820939e0 (AVCaptureDeviceInput: 0x282081920 Front Camera) -> (AVCaptureVideoPreviewLayer: 0x2820e8200) [type:vide][enabled:1][active:1]> &#9;- 4 : <AVCaptureConnection: 0x28208d460 (AVCaptureDeviceInput: 0x2820843e0 Headphones) -> (AVCaptureAudioDataOutput: 0x2820f7ba0) [type:soun][enabled:1][active:1]> &#9;- 5 : <AVCaptureConnection: 0x28208e480 (AVCaptureDeviceInput: 0x2820843e0 Headphones) -> (AVCaptureAudioDataOutput: 0x2820f7d20) [type:soun][enabled:1][active:1]> (lldb) po (notification.object as! AVCaptureMultiCamSession).hardwareCost 0.58957714 (lldb) po (notification.object as! AVCaptureMultiCamSession).systemPressureCost 0.81041515 (lldb) po (notification.object as! AVCaptureMultiCamSession).outputs ▿ 4 elements &#9;- 0 : <AVCaptureVideoDataOutput: 0x2820f7840> &#9;- 1 : <AVCaptureVideoDataOutput: 0x2820f7a80> &#9;- 2 : <AVCaptureAudioDataOutput: 0x2820f7ba0> &#9;- 3 : <AVCaptureAudioDataOutput: 0x2820f7d20> (lldb) po (notification.object as! AVCaptureMultiCamSession).inputs ▿ 3 elements &#9;- 0 : <AVCaptureDeviceInput: 0x282081280 [Back Camera]> &#9;- 1 : <AVCaptureDeviceInput: 0x282081920 [Front Camera]> &#9;- 2 : <AVCaptureDeviceInput: 0x2820843e0 [Headphones]> (lldb) po notification.userInfo ▿ Optional<Dictionary<AnyHashable, Any>> &#9;▿ some : 1 element &#9;&#9;▿ 0 : 2 elements &#9;&#9;&#9;▿ key : AnyHashable("AVCaptureSessionErrorKey") &#9;&#9;&#9;&#9;- value : "AVCaptureSessionErrorKey" &#9;&#9;&#9;- value : Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedFailureReason=An unknown error occurred (-12780), NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x282e81200 {Error Domain=NSOSStatusErrorDomain Code=-12780 "(null)"}} (lldb)
Replies
1
Boosts
0
Views
1.3k
Activity
Aug ’21