Dive into the technical aspects of audio on your device, including codecs, format support, and customization options.

Audio Documentation

Posts under Audio subtopic

Post

Replies

Boosts

Views

Activity

Creating ShazamKit signatures from audio files
I was trying to generate a signature from an audio file on-disk. Loading the files work, however generating signatures from them don't seem to work. Here's the code I'm using to generate the signature: func signature(from asset: AVAudioFile) -> SHSignature? {     let format = asset.processingFormat     let frameCount = AVAudioFrameCount(asset.length)           guard let buffer = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: frameCount) else { return nil }     let generator = SHSignatureGenerator()           do {        try asset.read(into: buffer)                let sampleRate = asset.processingFormat.sampleRate                try generator.append(buffer, at: AVAudioTime(sampleTime: .zero, atRate: sampleRate))                return generator.signature()     } catch {        print(error)        return nil     } } There are no errors being thrown in the code, but when I try to use the signatures in an SHSession, the audio is not being recognised. However, using the Shazam library works. I have tried searching ways to do this, however the only examples I've seen are for generating signatures from live audio input. Is there something I'm missing from the code above?
3
0
2.2k
Jul ’22
Adding multiple custom catalogs?
The operation couldn’t be completed. (com.apple.ShazamKit error 300.) I get this error when I try to add multiple shazamcatalogs on a single SHCustomCatalog. The fact that the operation is called add(from:) and not 'load' lets me suggest that it's meant to use that way. Am I wrong? And yes, both work work if I just add one at a time. let catalog = SHCustomCatalog() do { try catalog.add(from: url1) try catalog.add(from: url2) {...} } catch {...} . . Also, fun fact: I was working on a ShazamKit myself before dubDub and it's a really fascinating topic and so awesome to see how well it works, great job! 👏
1
0
1.1k
Sep ’21
ShazamKit constantly fail without an error
In continuation to this question, I am now able to use ShazamKit in my app, but for every video I pass to it didFailWithSignature is always called: func session(_ session: SHSession,        didNotFindMatchFor signature: SHSignature,                  error: Error?) {         DispatchQueue.main.async {             print("match error: \(String(describing: error))")         }     } with the following output: match error: nil This is happening even with videos that are just screen recorded video clips of Ed Sheeran for example, something that Shazam would easily recognized. Any idea why that would happen? What I should investigate? (the full code is also updated on my GitHub
2
0
1.7k
Sep ’21
CMSampleBuffer's audio format changes when coming from the background
Hi, I'm capturing Audio and Video using AVFoundation's AVCaptureSession using the AVCaptureVideoDataOutputSampleBufferDelegate & AVCaptureAudioDataOutputSampleBufferDelegate to collect CMSampleBuffers and write them to file using AVAssetWriter. The issues I'm facing is that while recording if the user goes to the background and returns, the CMSampleBuffer's AudioFormat changes from: this FormatList Array: { Index: 0 ChannelLayoutTag: 0x640001 ASBD: { mSampleRate: 48000.000000 mFormatID: 'lpcm' mFormatFlags: 0xc mBytesPerPacket: 2 mFramesPerPacket: 1 mBytesPerFrame: 2 mChannelsPerFrame: 1 mBitsPerChannel: 16 }} to this: FormatList Array: { Index: 0 ChannelLayoutTag: 0x640001 ASBD: { mSampleRate: 48000.000000 mFormatID: 'lpcm' mFormatFlags: 0xc mBytesPerPacket: 8 mFramesPerPacket: 1 mBytesPerFrame: 8 mChannelsPerFrame: 4 mBitsPerChannel: 16 }} This obviously creates an issue with the AudioInput in the AVAssetWriter and fails the writing Does anyone have an idea what going on? As to the CaptureSession setup: Inputs: Front / back camera & internal microphone outputs: AVCaptureVideoDataOutput & AVCaptureAudioDataOutput I am changing the active format in the videoInput's device when I start the session Thanks, Yoav
2
1
1.6k
Nov ’22
AVAudioEngine - output format has 0 channels after changing device of auAudioUnit of the inputNode
I'm trying to change device of the inputNode of AVAudioEngine. To do so, I'm calling setDeviceID on its auAudioUnit. Although this call doesn't fail, something wrong happens to the output busses. When I ask for its format, it shows a 0Hz and 0 channels format. It makes the app crash when I try to connect the node to the mainMixerNode. Can anyone explain what's wrong with this code? avEngine = AVAudioEngine() print(avEngine.inputNode.auAudioUnit.inputBusses[0].format) // <AVAudioFormat 0x1404b06e0: 2 ch, 44100 Hz, Float32, non-inter> print(avEngine.inputNode.auAudioUnit.outputBusses[0].format) // <AVAudioFormat 0x1404b0a60: 2 ch, 44100 Hz, Float32, inter> // Now, let's change a device from headphone's mic to built-in mic. try! avEngine.inputNode.auAudioUnit.setDeviceID(inputDevice.deviceID) print(avEngine.inputNode.auAudioUnit.inputBusses[0].format) // <AVAudioFormat 0x1404add50: 2 ch, 44100 Hz, Float32, non-inter> print(avEngine.inputNode.auAudioUnit.outputBusses[0].format) // <AVAudioFormat 0x1404adff0: 0 ch, 0 Hz, 'lpcm' (0x00000029) 32-bit little-endian float, deinterleaved> // !!! // Interestingly, 'inputNode' shows a different format than `auAudioUnit` print(avEngine.inputNode.inputFormat(forBus: 0)) // <AVAudioFormat 0x1404af480: 1 ch, 44100 Hz, Float32> print(avEngine.inputNode.outputFormat(forBus: 0)) // <AVAudioFormat 0x1404ade30: 1 ch, 44100 Hz, Float32> Edit: Further debugging revels another puzzling thing. avEngine.inputNode.auAudioUnit == avEngine.outputNode.auAudioUnit // this is true ?! inputNode and outputNode share the same AUAudioUnit. And its deviceID is by default set to the speakers. It's so confusing to me...why would inpudeNode's device be a speaker?
1
0
2.0k
Dec ’21
ShazamKit 202 error
I'm creating a simple song-matching application. My steps are: I create a project in XCode. Set bundle ID. Select my team Create an identifier, in my developer account Inserting my bundle ID from Xcode Selecting my team's application ID Turn on ShazamKit in services And after all that, I still get 202 error Error Domain=com.apple.ShazamKit Code=202 "Please check that you have enabled the ShazamKit App Service for this app identifier" UserInfo={NSDebugDescription=Please check that you have enabled the ShazamKit App Service for this app identifier} What am I doing wrong? Could it have something to do with the code?
2
0
1.9k
Sep ’21
valid replacement for kUTTypeJPEG which is deprecated
I have the following code: let ciImage = filterAndRender(ciImage: inputImage, doCrop: true)         let outCGImage = ciContext.createCGImage(ciImage, from: ciImage.extent)!         let dest = CGImageDestinationCreateWithURL(output.renderedContentURL as CFURL, kUTTypeJPEG, 1, nil)!         CGImageDestinationAddImage(dest, outCGImage, [kCGImageDestinationLossyCompressionQuality as String:1] as CFDictionary)         CGImageDestinationFinalize(dest) I get the following caution: " 'kUTTypeJPEG' was deprecated in iOS 15.0: Use UTTypeJPEG instead." However, when I substitute 'UTTypeJPEG' as directed, I get this error: "Cannot find 'UTTypeJPEG' in scope" What should I use for kUTTypeJPEG instead? Thanks!
5
2
11k
May ’23
ShazamKit during AVCaptureSession - Recognize audio while using camera
Hi, I want to implement ShazamKit in my project. But I have some problems. I use AVCaptureSession to take photos in my app and I'm unable to use ShazamKit. I tried to use three different ways Use an AVAudioEngine during my AVCaptureSession But I didn't obtain any result from Shazam. Try to use ShazamKit after stopping my AvCaptureSession but this causes some problems, and some crashes. Try to use the buffer of my AVCaptureSession to catch audio directly without use AVAudioEngine. This is the code that I use with AVAudioEngine: try! audioSession.setActive(true, options: .notifyOthersOnDeactivation)                 let inputNode = self.audioEngine.inputNode                 let recordingFormat = inputNode.outputFormat(forBus: 0)                                 let audioFormat = recordingFormat //AVAudioFormat(standardFormatWithSampleRate: self.audioEngine.inputNode.outputFormat(forBus: 0).sampleRate,                     //                            channels: 1)                                  inputNode.installTap(onBus: 0, bufferSize: 1024, format: audioFormat) { (buffer: AVAudioPCMBuffer, when: AVAudioTime) in                     try! self.signatureGenerator.append(buffer, at: nil)                                          self.session.matchStreamingBuffer(buffer, at: nil)                 }                              self.audioEngine.prepare()                 try! self.audioEngine.start() I can choose two ways to do this, use AVCaptureSession output to pass it to ShazamKit or use an AVAudioSession after the stop of AVCaptureSession. So I have two questions: Can I use a CMSampleBufferRef from AVCaptureSession buffer in a SHSession? And if the answer is yes how? How can I prevent this error if I want to use an AVAudioSession after I stopped my AVCaptureSession? [aurioc]            AURemoteIO.cpp:1117  failed: -10851 (enable 1, outf< 2 ch,      0 Hz, Float32, deinterleaved> inf< 2 ch,      0 Hz, Float32, deinterleaved>) [avae]            AVAEInternal.h:76    required condition is false: [AVAEGraphNode.mm:834:CreateRecordingTap: (IsFormatSampleRateAndChannelCountValid(format))] *** Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'required condition is false: IsFormatSampleRateAndChannelCountValid(format)' Thanks
1
0
1.5k
Sep ’21
How do music apps separate the input audio channels?
Hello, In my iPadOS app I need to be able to apply AUv3 effects to individual input channels. However, AVAudioEngine's inputNode delivers one output bus with all input channels in that bus. How can I separate these input channels? For example, an audio interface with 6 input channels yields an inputNode with 6 channels on the input bus and output bus. How can I apply an effect to channel 1? AVAudioEngine only lets me connect busses between audio units. AVAudioConnectionPoint is an auxiliary class that allows me to establish a one-to-many routing. It seems that with the V3 API I cannot achieve my goal. What other solutions are there? Use the V2 API and install a callback... exactly where? Or is there some kind of audio format conversion foreseen?
1
0
1.3k
May ’22
CenterStage API
Hi Iam trying to create a video recording page integrating AvFoundation, I want to enable centerStage mode in the app. Is there any API available for enabling the centerStage. Is there any recommended guide to integrate centerstage in the ios App.
2
0
1.6k
Aug ’21
Music preview with MusicKit
Hi, I apologize for asking so many questions. I don't know that I'll ever ship anything but I'm experimenting with an old fashioned radio station like we used to program in the old top forties days. I'd like to allow the music programmers to listen to sets that they're building by skimming through songs - does MusicKit or any part of the API allow us to access the Music preview for a given song? i.e. - I want users with a subscription to be able to hear the 30 s preview that you can find in Music. I may be requesting that a Song have a preview as one of its playing mode options. Thank you, Daniel
9
0
4.6k
Apr ’22
avassetreader convert hdr frames to proper colorized CIImages
there are a lot of examples on how to add HDR video support to your apps, but we want to do is read video frames from an HDR video file and then properly map the colors to CIImages so they don't look washed out. it seems like there is some sort of color transform that needs to be applied. but does anyone know the magic formula? CVPixelBufferRef inputPixelBuffer = CMSampleBufferGetImageBuffer(buffer); CIImage* ciImage = [CIImage imageWithCVPixelBuffer:inputPixelBuffer]; // one vid frame
3
0
1.6k
Oct ’22
AVAudioEngine when connected to Airplay
Background We're writing a small recording app - think Voice Memos for the sake of argument. In our app, users should always record with the built-in iPhone microphone. Our Problem Our setup works fine when using just the speakers or in combination with Bluetooth headsets. However, it doesn't work well with Airplay. One of two things can happen: The app records just silence The app crashes when trying to connect the inputNode to the recorderNode (see code below), complaining that IsFormatSampleRateAndChannelCountValid == false Our testing environment is an iPhone Xs, connected to an Airplay 2 compatible Sonos amp. Code We use the following code to set up the AVAudioSession (simplified, without error handling): let session = AVAudioSession.sharedInstance() try session.setCategory(.playAndRecord, options: [.defaultToSpeaker, .allowBluetoothA2DP, .allowAirPlay]) try AVAudioSession.sharedInstance().setActive(true) Every time we record, we configure the audio session to use the built-in mic, and then create a fresh AVAudioEngine. let session = AVAudioSession.sharedInstance() let builtInMicInput = session.availableInputs!.first(where: { $0.portType == .builtInMic }) try session.setPreferredInput(builtInMicInput) let sampleRate: Double = 44100 let numChannels: AVAudioChannelCount = isStereoEnabled ? 2 : 1 let recordingOutputFormat = AVAudioFormat(commonFormat: .pcmFormatFloat32, sampleRate: sampleRate, channels: numChannels, interleaved: false)! let engine = AVAudioEngine() let recorderNode = AVAudioMixerNode() // This sets the input volume of those nodes in their destination node (mainMixerNode) to 0. // The raw outputVolume of these nodes remains 1, so when you tap them you still get the samples. // If you set outputVolume = 0 instead, the taps would only receives zeros. recorderNode.volume = 0 engine.attach(recorderNode) engine.connect(engine.mainMixerNode,  to: engine.outputNode,   format: engine.outputNode.inputFormat(forBus: 0)) engine.connect(recorderNode,      to: engine.mainMixerNode,  format: recordingOutputFormat) engine.connect(engine.inputNode,    to: recorderNode,      format: engine.inputNode.inputFormat(forBus: 0)) // and later try engine.start() We install a tap on the recorderNode to save the recorded audio into a file. The tap works fine and is out of scope for this question, and thus not included here. Questions How do we route/configure the audio engine correctly to avoid this problem? Do you have any advice on how to debug such issues in the future? Which variables/states should we inspect? Thank you so much in advance!
1
0
1.8k
Aug ’21
AVPlayerItemVideoOutput Converting HDR Video to SDR not work
My app want Converting iphone12 HDR Video to SDR,to edit。 follow the doc Apple-HDR-Convert. My code setting the pixBuffAttributes        [pixBuffAttributes setObject:(id)(kCVImageBufferYCbCrMatrix_ITU_R_709_2) forKey:(id)kCVImageBufferYCbCrMatrixKey];       [pixBuffAttributes setObject:(id)(kCVImageBufferColorPrimaries_ITU_R_709_2) forKey:(id)kCVImageBufferColorPrimariesKey];       [pixBuffAttributes setObject:(id)kCVImageBufferTransferFunction_ITU_R_709_2 forKey:(id)kCVImageBufferTransferFunctionKey];       playerItemOutput = [[AVPlayerItemVideoOutput alloc] initWithPixelBufferAttributes:pixBuffAttributes]; but I get the playerItemOutput's output buffer  CFTypeRef colorAttachments = CVBufferGetAttachment(pixelBuffer, kCVImageBufferYCbCrMatrixKey, NULL);     CFTypeRef colorPrimaries = CVBufferGetAttachment(pixelBuffer, kCVImageBufferColorPrimariesKey, NULL);     CFTypeRef colorTransFunc = CVBufferGetAttachment(pixelBuffer, kCVImageBufferTransferFunctionKey, NULL);      NSLog(@"colorAttachments = %@", colorAttachments);     NSLog(@"colorPrimaries = %@", colorPrimaries);     NSLog(@"colorTransFunc = %@", colorTransFunc); log output: colorAttachments = ITU_R_2020 colorPrimaries = ITU_R_2020 colorTransFunc = ITU_R_2100_HLG pixBuffAttributes setting output format invalid,please help!
2
0
1.4k
Oct ’22
[AVPlayerItemVideoOutput initWithPixelBufferAttributes:] output attributes setting not work
My app want Converting iphone12 HDR Video to SDR,to edit。 follow the doc Apple-HDR-Convert. My code setting the pixBuffAttributes        [pixBuffAttributes setObject:(id)(kCVImageBufferYCbCrMatrix_ITU_R_709_2) forKey:(id)kCVImageBufferYCbCrMatrixKey];       [pixBuffAttributes setObject:(id)(kCVImageBufferColorPrimaries_ITU_R_709_2) forKey:(id)kCVImageBufferColorPrimariesKey];       [pixBuffAttributes setObject:(id)kCVImageBufferTransferFunction_ITU_R_709_2 forKey:(id)kCVImageBufferTransferFunctionKey];       playerItemOutput = [[AVPlayerItemVideoOutput alloc] initWithPixelBufferAttributes:pixBuffAttributes]; but I get the playerItemOutput's output buffer   CFTypeRef colorAttachments = CVBufferGetAttachment(pixelBuffer, kCVImageBufferYCbCrMatrixKey, NULL);     CFTypeRef colorPrimaries = CVBufferGetAttachment(pixelBuffer, kCVImageBufferColorPrimariesKey, NULL);     CFTypeRef colorTransFunc = CVBufferGetAttachment(pixelBuffer, kCVImageBufferTransferFunctionKey, NULL);      NSLog(@"colorAttachments = %@", colorAttachments);     NSLog(@"colorPrimaries = %@", colorPrimaries);     NSLog(@"colorTransFunc = %@", colorTransFunc); log output: colorAttachments = ITU_R_2020 colorPrimaries = ITU_R_2020 colorTransFunc = ITU_R_2100_HLG pixBuffAttributes setting output format invalid,please help!
1
1
886
Nov ’25
What are the conditions under which the play method of AVAudioPlayer returns false?
My application uses AVAudioPlayer to play sound. But lately, when I play a sound, it returns a failure. Restarting the application will fix it temporarily, but it will immediately return false. I read the docs but couldn't figure out the exact reason for returning false. https://developer.apple.com/documentation/avfaudio/avaudioplayer/1387388-play Can anyone please tell me what causes false to be returned?
1
0
671
Jan ’22
MusicKit Sample Code errors
In Xcode 13 beta 4, the Sample Code Using MusicKit to Integrate with Apple Music has compiler errors around the player.play() calls in handleTrackSelected and handlePlayButtonSelected. The error is "async call in a function that does not support concurrency / call can throw, but is not marked with try and the error is not handled" Would it be possible to include a fix for this error in the sample code - or could somebody share a fix here in the forums? Thanks so much!
5
0
1.4k
Aug ’21
rectify images and depth map
In the WWDC 2017 Session 507 - Capturing Depth in iPhone Photography there is a sample app being demonstrated called straightend up. It demonstrates how to make an image rectilinear. Is it possible to have a look at the code of this app. It doesn't seem to be published. Also in this session there is a demo implementation mentioned in AVCameraCalibrationData.h. Is there a way to access this demo implementation in recent Xcode-Versions?
0
0
426
Aug ’21
Stereo Audio API broken on iOS 15
AVAudioSession API for setting stereo orientation on supported devices (iPhone XS and above) is completely broken on iOS 15 betas. Even the 'Stereo Audio Capture' sample code does not work anymore on iOS 15. Even AVCaptureAudioDataOutput fails when setting stereo orientation on AVAudioSession. I am wondering if Apple Engineers are aware of this issue and whether this would be fixed in upcoming betas and most importantly, iOS 15 mainline.
6
0
1.8k
Aug ’21
ShazamKit failing to get matches on Android
I'm trying to test ShazamKit on Android. In the following Kotlin snippet, I load apple-test-wav.wav - a 9 second long, 48khz, 16bit PCM WAV audio snippet of the Food Math example video - and attempt to match it against the FoodMath.shazamsignature custom catalog. This audio is not capture via a microphone, it's a clean sample directly from the video and I can't get it to match. The output is always: NoMatch Am I doing something obviously wrong here? override fun onViewCreated(view: View, savedInstanceState: Bundle?) { super.onViewCreated(view, savedInstanceState) CoroutineScope(Dispatchers.IO).launch { searchCatalog() } } private suspend fun searchCatalog() { val file = context?.assets?.open("apple-test-wav.wav") if (file != null) { val bytes = file.readBytes() file.close() val signatureGenerator = (ShazamKit.createSignatureGenerator(AudioSampleRateInHz.SAMPLE_RATE_48000) as ShazamKitResult.Success).data signatureGenerator.append(bytes, bytes.size, System.currentTimeMillis()) val signature = signatureGenerator.generateSignature() val inputStream: InputStream? = context?.assets?.open("FoodMath.shazamsignature") if (inputStream!= null) { val catalog = ShazamKit.createCustomCatalog().apply { addFromCatalog(inputStream) } val session = (ShazamKit.createSession(catalog) as ShazamKitResult.Success).data val matchResult: MatchResult = session.match(signature) println(matchResult.toString()) } else { println("no input stream") } } }
1
0
1.2k
Oct ’21
Creating ShazamKit signatures from audio files
I was trying to generate a signature from an audio file on-disk. Loading the files work, however generating signatures from them don't seem to work. Here's the code I'm using to generate the signature: func signature(from asset: AVAudioFile) -> SHSignature? {     let format = asset.processingFormat     let frameCount = AVAudioFrameCount(asset.length)           guard let buffer = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: frameCount) else { return nil }     let generator = SHSignatureGenerator()           do {        try asset.read(into: buffer)                let sampleRate = asset.processingFormat.sampleRate                try generator.append(buffer, at: AVAudioTime(sampleTime: .zero, atRate: sampleRate))                return generator.signature()     } catch {        print(error)        return nil     } } There are no errors being thrown in the code, but when I try to use the signatures in an SHSession, the audio is not being recognised. However, using the Shazam library works. I have tried searching ways to do this, however the only examples I've seen are for generating signatures from live audio input. Is there something I'm missing from the code above?
Replies
3
Boosts
0
Views
2.2k
Activity
Jul ’22
Adding multiple custom catalogs?
The operation couldn’t be completed. (com.apple.ShazamKit error 300.) I get this error when I try to add multiple shazamcatalogs on a single SHCustomCatalog. The fact that the operation is called add(from:) and not 'load' lets me suggest that it's meant to use that way. Am I wrong? And yes, both work work if I just add one at a time. let catalog = SHCustomCatalog() do { try catalog.add(from: url1) try catalog.add(from: url2) {...} } catch {...} . . Also, fun fact: I was working on a ShazamKit myself before dubDub and it's a really fascinating topic and so awesome to see how well it works, great job! 👏
Replies
1
Boosts
0
Views
1.1k
Activity
Sep ’21
ShazamKit constantly fail without an error
In continuation to this question, I am now able to use ShazamKit in my app, but for every video I pass to it didFailWithSignature is always called: func session(_ session: SHSession,        didNotFindMatchFor signature: SHSignature,                  error: Error?) {         DispatchQueue.main.async {             print("match error: \(String(describing: error))")         }     } with the following output: match error: nil This is happening even with videos that are just screen recorded video clips of Ed Sheeran for example, something that Shazam would easily recognized. Any idea why that would happen? What I should investigate? (the full code is also updated on my GitHub
Replies
2
Boosts
0
Views
1.7k
Activity
Sep ’21
CMSampleBuffer's audio format changes when coming from the background
Hi, I'm capturing Audio and Video using AVFoundation's AVCaptureSession using the AVCaptureVideoDataOutputSampleBufferDelegate & AVCaptureAudioDataOutputSampleBufferDelegate to collect CMSampleBuffers and write them to file using AVAssetWriter. The issues I'm facing is that while recording if the user goes to the background and returns, the CMSampleBuffer's AudioFormat changes from: this FormatList Array: { Index: 0 ChannelLayoutTag: 0x640001 ASBD: { mSampleRate: 48000.000000 mFormatID: 'lpcm' mFormatFlags: 0xc mBytesPerPacket: 2 mFramesPerPacket: 1 mBytesPerFrame: 2 mChannelsPerFrame: 1 mBitsPerChannel: 16 }} to this: FormatList Array: { Index: 0 ChannelLayoutTag: 0x640001 ASBD: { mSampleRate: 48000.000000 mFormatID: 'lpcm' mFormatFlags: 0xc mBytesPerPacket: 8 mFramesPerPacket: 1 mBytesPerFrame: 8 mChannelsPerFrame: 4 mBitsPerChannel: 16 }} This obviously creates an issue with the AudioInput in the AVAssetWriter and fails the writing Does anyone have an idea what going on? As to the CaptureSession setup: Inputs: Front / back camera & internal microphone outputs: AVCaptureVideoDataOutput & AVCaptureAudioDataOutput I am changing the active format in the videoInput's device when I start the session Thanks, Yoav
Replies
2
Boosts
1
Views
1.6k
Activity
Nov ’22
AVAudioEngine - output format has 0 channels after changing device of auAudioUnit of the inputNode
I'm trying to change device of the inputNode of AVAudioEngine. To do so, I'm calling setDeviceID on its auAudioUnit. Although this call doesn't fail, something wrong happens to the output busses. When I ask for its format, it shows a 0Hz and 0 channels format. It makes the app crash when I try to connect the node to the mainMixerNode. Can anyone explain what's wrong with this code? avEngine = AVAudioEngine() print(avEngine.inputNode.auAudioUnit.inputBusses[0].format) // <AVAudioFormat 0x1404b06e0: 2 ch, 44100 Hz, Float32, non-inter> print(avEngine.inputNode.auAudioUnit.outputBusses[0].format) // <AVAudioFormat 0x1404b0a60: 2 ch, 44100 Hz, Float32, inter> // Now, let's change a device from headphone's mic to built-in mic. try! avEngine.inputNode.auAudioUnit.setDeviceID(inputDevice.deviceID) print(avEngine.inputNode.auAudioUnit.inputBusses[0].format) // <AVAudioFormat 0x1404add50: 2 ch, 44100 Hz, Float32, non-inter> print(avEngine.inputNode.auAudioUnit.outputBusses[0].format) // <AVAudioFormat 0x1404adff0: 0 ch, 0 Hz, 'lpcm' (0x00000029) 32-bit little-endian float, deinterleaved> // !!! // Interestingly, 'inputNode' shows a different format than `auAudioUnit` print(avEngine.inputNode.inputFormat(forBus: 0)) // <AVAudioFormat 0x1404af480: 1 ch, 44100 Hz, Float32> print(avEngine.inputNode.outputFormat(forBus: 0)) // <AVAudioFormat 0x1404ade30: 1 ch, 44100 Hz, Float32> Edit: Further debugging revels another puzzling thing. avEngine.inputNode.auAudioUnit == avEngine.outputNode.auAudioUnit // this is true ?! inputNode and outputNode share the same AUAudioUnit. And its deviceID is by default set to the speakers. It's so confusing to me...why would inpudeNode's device be a speaker?
Replies
1
Boosts
0
Views
2.0k
Activity
Dec ’21
ShazamKit 202 error
I'm creating a simple song-matching application. My steps are: I create a project in XCode. Set bundle ID. Select my team Create an identifier, in my developer account Inserting my bundle ID from Xcode Selecting my team's application ID Turn on ShazamKit in services And after all that, I still get 202 error Error Domain=com.apple.ShazamKit Code=202 "Please check that you have enabled the ShazamKit App Service for this app identifier" UserInfo={NSDebugDescription=Please check that you have enabled the ShazamKit App Service for this app identifier} What am I doing wrong? Could it have something to do with the code?
Replies
2
Boosts
0
Views
1.9k
Activity
Sep ’21
valid replacement for kUTTypeJPEG which is deprecated
I have the following code: let ciImage = filterAndRender(ciImage: inputImage, doCrop: true)         let outCGImage = ciContext.createCGImage(ciImage, from: ciImage.extent)!         let dest = CGImageDestinationCreateWithURL(output.renderedContentURL as CFURL, kUTTypeJPEG, 1, nil)!         CGImageDestinationAddImage(dest, outCGImage, [kCGImageDestinationLossyCompressionQuality as String:1] as CFDictionary)         CGImageDestinationFinalize(dest) I get the following caution: " 'kUTTypeJPEG' was deprecated in iOS 15.0: Use UTTypeJPEG instead." However, when I substitute 'UTTypeJPEG' as directed, I get this error: "Cannot find 'UTTypeJPEG' in scope" What should I use for kUTTypeJPEG instead? Thanks!
Replies
5
Boosts
2
Views
11k
Activity
May ’23
ShazamKit during AVCaptureSession - Recognize audio while using camera
Hi, I want to implement ShazamKit in my project. But I have some problems. I use AVCaptureSession to take photos in my app and I'm unable to use ShazamKit. I tried to use three different ways Use an AVAudioEngine during my AVCaptureSession But I didn't obtain any result from Shazam. Try to use ShazamKit after stopping my AvCaptureSession but this causes some problems, and some crashes. Try to use the buffer of my AVCaptureSession to catch audio directly without use AVAudioEngine. This is the code that I use with AVAudioEngine: try! audioSession.setActive(true, options: .notifyOthersOnDeactivation)                 let inputNode = self.audioEngine.inputNode                 let recordingFormat = inputNode.outputFormat(forBus: 0)                                 let audioFormat = recordingFormat //AVAudioFormat(standardFormatWithSampleRate: self.audioEngine.inputNode.outputFormat(forBus: 0).sampleRate,                     //                            channels: 1)                                  inputNode.installTap(onBus: 0, bufferSize: 1024, format: audioFormat) { (buffer: AVAudioPCMBuffer, when: AVAudioTime) in                     try! self.signatureGenerator.append(buffer, at: nil)                                          self.session.matchStreamingBuffer(buffer, at: nil)                 }                              self.audioEngine.prepare()                 try! self.audioEngine.start() I can choose two ways to do this, use AVCaptureSession output to pass it to ShazamKit or use an AVAudioSession after the stop of AVCaptureSession. So I have two questions: Can I use a CMSampleBufferRef from AVCaptureSession buffer in a SHSession? And if the answer is yes how? How can I prevent this error if I want to use an AVAudioSession after I stopped my AVCaptureSession? [aurioc]            AURemoteIO.cpp:1117  failed: -10851 (enable 1, outf< 2 ch,      0 Hz, Float32, deinterleaved> inf< 2 ch,      0 Hz, Float32, deinterleaved>) [avae]            AVAEInternal.h:76    required condition is false: [AVAEGraphNode.mm:834:CreateRecordingTap: (IsFormatSampleRateAndChannelCountValid(format))] *** Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'required condition is false: IsFormatSampleRateAndChannelCountValid(format)' Thanks
Replies
1
Boosts
0
Views
1.5k
Activity
Sep ’21
How do music apps separate the input audio channels?
Hello, In my iPadOS app I need to be able to apply AUv3 effects to individual input channels. However, AVAudioEngine's inputNode delivers one output bus with all input channels in that bus. How can I separate these input channels? For example, an audio interface with 6 input channels yields an inputNode with 6 channels on the input bus and output bus. How can I apply an effect to channel 1? AVAudioEngine only lets me connect busses between audio units. AVAudioConnectionPoint is an auxiliary class that allows me to establish a one-to-many routing. It seems that with the V3 API I cannot achieve my goal. What other solutions are there? Use the V2 API and install a callback... exactly where? Or is there some kind of audio format conversion foreseen?
Replies
1
Boosts
0
Views
1.3k
Activity
May ’22
CenterStage API
Hi Iam trying to create a video recording page integrating AvFoundation, I want to enable centerStage mode in the app. Is there any API available for enabling the centerStage. Is there any recommended guide to integrate centerstage in the ios App.
Replies
2
Boosts
0
Views
1.6k
Activity
Aug ’21
Music preview with MusicKit
Hi, I apologize for asking so many questions. I don't know that I'll ever ship anything but I'm experimenting with an old fashioned radio station like we used to program in the old top forties days. I'd like to allow the music programmers to listen to sets that they're building by skimming through songs - does MusicKit or any part of the API allow us to access the Music preview for a given song? i.e. - I want users with a subscription to be able to hear the 30 s preview that you can find in Music. I may be requesting that a Song have a preview as one of its playing mode options. Thank you, Daniel
Replies
9
Boosts
0
Views
4.6k
Activity
Apr ’22
avassetreader convert hdr frames to proper colorized CIImages
there are a lot of examples on how to add HDR video support to your apps, but we want to do is read video frames from an HDR video file and then properly map the colors to CIImages so they don't look washed out. it seems like there is some sort of color transform that needs to be applied. but does anyone know the magic formula? CVPixelBufferRef inputPixelBuffer = CMSampleBufferGetImageBuffer(buffer); CIImage* ciImage = [CIImage imageWithCVPixelBuffer:inputPixelBuffer]; // one vid frame
Replies
3
Boosts
0
Views
1.6k
Activity
Oct ’22
AVAudioEngine when connected to Airplay
Background We're writing a small recording app - think Voice Memos for the sake of argument. In our app, users should always record with the built-in iPhone microphone. Our Problem Our setup works fine when using just the speakers or in combination with Bluetooth headsets. However, it doesn't work well with Airplay. One of two things can happen: The app records just silence The app crashes when trying to connect the inputNode to the recorderNode (see code below), complaining that IsFormatSampleRateAndChannelCountValid == false Our testing environment is an iPhone Xs, connected to an Airplay 2 compatible Sonos amp. Code We use the following code to set up the AVAudioSession (simplified, without error handling): let session = AVAudioSession.sharedInstance() try session.setCategory(.playAndRecord, options: [.defaultToSpeaker, .allowBluetoothA2DP, .allowAirPlay]) try AVAudioSession.sharedInstance().setActive(true) Every time we record, we configure the audio session to use the built-in mic, and then create a fresh AVAudioEngine. let session = AVAudioSession.sharedInstance() let builtInMicInput = session.availableInputs!.first(where: { $0.portType == .builtInMic }) try session.setPreferredInput(builtInMicInput) let sampleRate: Double = 44100 let numChannels: AVAudioChannelCount = isStereoEnabled ? 2 : 1 let recordingOutputFormat = AVAudioFormat(commonFormat: .pcmFormatFloat32, sampleRate: sampleRate, channels: numChannels, interleaved: false)! let engine = AVAudioEngine() let recorderNode = AVAudioMixerNode() // This sets the input volume of those nodes in their destination node (mainMixerNode) to 0. // The raw outputVolume of these nodes remains 1, so when you tap them you still get the samples. // If you set outputVolume = 0 instead, the taps would only receives zeros. recorderNode.volume = 0 engine.attach(recorderNode) engine.connect(engine.mainMixerNode,  to: engine.outputNode,   format: engine.outputNode.inputFormat(forBus: 0)) engine.connect(recorderNode,      to: engine.mainMixerNode,  format: recordingOutputFormat) engine.connect(engine.inputNode,    to: recorderNode,      format: engine.inputNode.inputFormat(forBus: 0)) // and later try engine.start() We install a tap on the recorderNode to save the recorded audio into a file. The tap works fine and is out of scope for this question, and thus not included here. Questions How do we route/configure the audio engine correctly to avoid this problem? Do you have any advice on how to debug such issues in the future? Which variables/states should we inspect? Thank you so much in advance!
Replies
1
Boosts
0
Views
1.8k
Activity
Aug ’21
AVPlayerItemVideoOutput Converting HDR Video to SDR not work
My app want Converting iphone12 HDR Video to SDR,to edit。 follow the doc Apple-HDR-Convert. My code setting the pixBuffAttributes        [pixBuffAttributes setObject:(id)(kCVImageBufferYCbCrMatrix_ITU_R_709_2) forKey:(id)kCVImageBufferYCbCrMatrixKey];       [pixBuffAttributes setObject:(id)(kCVImageBufferColorPrimaries_ITU_R_709_2) forKey:(id)kCVImageBufferColorPrimariesKey];       [pixBuffAttributes setObject:(id)kCVImageBufferTransferFunction_ITU_R_709_2 forKey:(id)kCVImageBufferTransferFunctionKey];       playerItemOutput = [[AVPlayerItemVideoOutput alloc] initWithPixelBufferAttributes:pixBuffAttributes]; but I get the playerItemOutput's output buffer  CFTypeRef colorAttachments = CVBufferGetAttachment(pixelBuffer, kCVImageBufferYCbCrMatrixKey, NULL);     CFTypeRef colorPrimaries = CVBufferGetAttachment(pixelBuffer, kCVImageBufferColorPrimariesKey, NULL);     CFTypeRef colorTransFunc = CVBufferGetAttachment(pixelBuffer, kCVImageBufferTransferFunctionKey, NULL);      NSLog(@"colorAttachments = %@", colorAttachments);     NSLog(@"colorPrimaries = %@", colorPrimaries);     NSLog(@"colorTransFunc = %@", colorTransFunc); log output: colorAttachments = ITU_R_2020 colorPrimaries = ITU_R_2020 colorTransFunc = ITU_R_2100_HLG pixBuffAttributes setting output format invalid,please help!
Replies
2
Boosts
0
Views
1.4k
Activity
Oct ’22
[AVPlayerItemVideoOutput initWithPixelBufferAttributes:] output attributes setting not work
My app want Converting iphone12 HDR Video to SDR,to edit。 follow the doc Apple-HDR-Convert. My code setting the pixBuffAttributes        [pixBuffAttributes setObject:(id)(kCVImageBufferYCbCrMatrix_ITU_R_709_2) forKey:(id)kCVImageBufferYCbCrMatrixKey];       [pixBuffAttributes setObject:(id)(kCVImageBufferColorPrimaries_ITU_R_709_2) forKey:(id)kCVImageBufferColorPrimariesKey];       [pixBuffAttributes setObject:(id)kCVImageBufferTransferFunction_ITU_R_709_2 forKey:(id)kCVImageBufferTransferFunctionKey];       playerItemOutput = [[AVPlayerItemVideoOutput alloc] initWithPixelBufferAttributes:pixBuffAttributes]; but I get the playerItemOutput's output buffer   CFTypeRef colorAttachments = CVBufferGetAttachment(pixelBuffer, kCVImageBufferYCbCrMatrixKey, NULL);     CFTypeRef colorPrimaries = CVBufferGetAttachment(pixelBuffer, kCVImageBufferColorPrimariesKey, NULL);     CFTypeRef colorTransFunc = CVBufferGetAttachment(pixelBuffer, kCVImageBufferTransferFunctionKey, NULL);      NSLog(@"colorAttachments = %@", colorAttachments);     NSLog(@"colorPrimaries = %@", colorPrimaries);     NSLog(@"colorTransFunc = %@", colorTransFunc); log output: colorAttachments = ITU_R_2020 colorPrimaries = ITU_R_2020 colorTransFunc = ITU_R_2100_HLG pixBuffAttributes setting output format invalid,please help!
Replies
1
Boosts
1
Views
886
Activity
Nov ’25
What are the conditions under which the play method of AVAudioPlayer returns false?
My application uses AVAudioPlayer to play sound. But lately, when I play a sound, it returns a failure. Restarting the application will fix it temporarily, but it will immediately return false. I read the docs but couldn't figure out the exact reason for returning false. https://developer.apple.com/documentation/avfaudio/avaudioplayer/1387388-play Can anyone please tell me what causes false to be returned?
Replies
1
Boosts
0
Views
671
Activity
Jan ’22
MusicKit Sample Code errors
In Xcode 13 beta 4, the Sample Code Using MusicKit to Integrate with Apple Music has compiler errors around the player.play() calls in handleTrackSelected and handlePlayButtonSelected. The error is "async call in a function that does not support concurrency / call can throw, but is not marked with try and the error is not handled" Would it be possible to include a fix for this error in the sample code - or could somebody share a fix here in the forums? Thanks so much!
Replies
5
Boosts
0
Views
1.4k
Activity
Aug ’21
rectify images and depth map
In the WWDC 2017 Session 507 - Capturing Depth in iPhone Photography there is a sample app being demonstrated called straightend up. It demonstrates how to make an image rectilinear. Is it possible to have a look at the code of this app. It doesn't seem to be published. Also in this session there is a demo implementation mentioned in AVCameraCalibrationData.h. Is there a way to access this demo implementation in recent Xcode-Versions?
Replies
0
Boosts
0
Views
426
Activity
Aug ’21
Stereo Audio API broken on iOS 15
AVAudioSession API for setting stereo orientation on supported devices (iPhone XS and above) is completely broken on iOS 15 betas. Even the 'Stereo Audio Capture' sample code does not work anymore on iOS 15. Even AVCaptureAudioDataOutput fails when setting stereo orientation on AVAudioSession. I am wondering if Apple Engineers are aware of this issue and whether this would be fixed in upcoming betas and most importantly, iOS 15 mainline.
Replies
6
Boosts
0
Views
1.8k
Activity
Aug ’21
ShazamKit failing to get matches on Android
I'm trying to test ShazamKit on Android. In the following Kotlin snippet, I load apple-test-wav.wav - a 9 second long, 48khz, 16bit PCM WAV audio snippet of the Food Math example video - and attempt to match it against the FoodMath.shazamsignature custom catalog. This audio is not capture via a microphone, it's a clean sample directly from the video and I can't get it to match. The output is always: NoMatch Am I doing something obviously wrong here? override fun onViewCreated(view: View, savedInstanceState: Bundle?) { super.onViewCreated(view, savedInstanceState) CoroutineScope(Dispatchers.IO).launch { searchCatalog() } } private suspend fun searchCatalog() { val file = context?.assets?.open("apple-test-wav.wav") if (file != null) { val bytes = file.readBytes() file.close() val signatureGenerator = (ShazamKit.createSignatureGenerator(AudioSampleRateInHz.SAMPLE_RATE_48000) as ShazamKitResult.Success).data signatureGenerator.append(bytes, bytes.size, System.currentTimeMillis()) val signature = signatureGenerator.generateSignature() val inputStream: InputStream? = context?.assets?.open("FoodMath.shazamsignature") if (inputStream!= null) { val catalog = ShazamKit.createCustomCatalog().apply { addFromCatalog(inputStream) } val session = (ShazamKit.createSession(catalog) as ShazamKitResult.Success).data val matchResult: MatchResult = session.match(signature) println(matchResult.toString()) } else { println("no input stream") } } }
Replies
1
Boosts
0
Views
1.2k
Activity
Oct ’21