Explore the integration of media technologies within your app. Discuss working with audio, video, camera, and other media functionalities.

All subtopics
Posts under Media Technologies topic

Post

Replies

Boosts

Views

Activity

Can't create a key for MusicKit
I'm trying to create an app that can access a users Apple Music library to play music (the app will do far more than that when finished) but I am unable to generate a key. When I go to Register a New Key, it says: "There are no identifiers available that can be associated with the key" under Media Services (MusicKit, ShazamKit, Apple Music Feed). MusicKit is already checked under App Services in Certificates, Identifiers & Profiles. I've been stuck on this for two days. Please help!
0
0
1
22m
SpeechTranscriber Faster Results
I am experimenting with SpeechTranscriber and am curious if I can get quicker results when using buffered audio, rather than a file. The use case is a voice ordering experience for a restaurant. When I've been playing with it, it takes about 3 seconds for faster results and 7-8 seconds for accurate results. Is there any way to bring this down a bit? In this WWDC demo, the results appear nearly instantaneously. I'm curious how to replicate this in my app. I presume DicationTranscriber is faster, but how is siri detecting when the user stops speaking? Is it custom code, or is it using SpeechDetector? I tried using SpeechDetector with SpeechTranscriber but the detector didn't emit any results and seemed to slow down the results of SpeechTranscriber. I also assumed SpeechTranscriber makes more sense than DictationTranscriber in this use case, but want to confirm.
1
0
34
4h
ABR switching behavior with short (1–2s) segments in live HLS
To reduce latency in our live HLS service, we are experimenting with short segment durations (1–2 seconds). We have noticed that ABR switching occurs more frequently than expected. Looking at the console logs, variants appear to enter and leave what is referred to as a "penalty box" under conditions such as -12889 (a segment returning 0 bytes after 1.0 × target_duration) and -12888 (a playlist remaining unchanged beyond 1.5 × target_duration). Are there recommended practices to reduce this excessive switching? Specifically: Is there an API to control this penalty box behavior? Would preparing an LL-HLS stream with 1-second parts settle this down? Alternatively, would 4-second segments with a slightly smaller configuredTimeOffsetFromLive help stabilize this?
1
2
83
14h
How to detect if a HDR10+ variant is currently playing
Hi, We currently deliver HDR10+ content within a composite HLS stream where Dolby Vision is signaled as a supplemental codec alongside HDR10+. Our testing on Apple TV 4K (3rd generation) indicates that playback succeeds, but we have not found a reliable way to determine whether playback is occurring in Dolby Vision or HDR10+ mode when connected to a display that supports both formats. We have also verified that the same stream plays correctly on displays that support HDR10+ but do not support Dolby Vision. However, on displays that support both HDR10+ and Dolby Vision, we are unable to determine which dynamic range format AVPlayer ultimately selects. Is there a recommended Apple API or mechanism to determine whether AVPlayer is rendering content as HDR10+, Dolby Vision, or another HDR format at runtime? For reference, below is an example variant from our master playlist: #EXT-X-STREAM-INF:BANDWIDTH=2851276,AVERAGE-BANDWIDTH=2030750,CODECS="hvc1.2.4.L90.90,mp4a.40.2",SUPPLEMENTAL-CODECS="dvh1.08.01/dv1p,hvc1.2.4.L90.90/cdm4",RESOLUTION=854x480,FRAME-RATE=24,VIDEO-RANGE=PQ,HDCP-LEVEL=TYPE-1,CHARACTERISTICS="com.dss.cbcs.hdr.hd",AUDIO="aac-128k",SUBTITLES="sub-main" Specifically, we would like to understand: How AVPlayer chooses between Dolby Vision and HDR10+ when both are available in the same variant. Whether there is a supported way to determine the selected HDR format during playback. Whether any other APIs expose this information. Any guidance would be greatly appreciated. Thank you.
1
1
65
14h
Voice Processing
Why voice processing enabled on AVAudioInputNode makes output audio noticable lower than without it and how to overcome it using voice processing enabled
4
0
90
16h
AirPods Custom EQ interaction
In iOS 27, when a user has configured a custom AirPods EQ (three-band lows/mids/highs), does that EQ apply on top of audio output from AVAudioEngine in my app, or only to media playback through AVPlayer/MPMusicPlayerController? My app generates therapeutic frequencies (e.g. 528 Hz pure tones) where precise frequency output matters - is there any API to detect or opt out of the system AirPods EQ?
1
0
28
17h
Real-time synthesis vs. files for long background sessions
For a sleep app running 8–12 hours in background, is AVAudioSourceNode with a real-time render block more power-efficient than looping a pre-encoded audio file via AVAudioPlayerNode? I want to migrate from files to procedural synthesis but not at the cost of battery. What does Instruments / Energy Log show as the typical CPU overhead difference, and is there Apple guidance on this trade-off?
1
0
34
17h
BlietoothHFP to MFi hearingaids
Is it possible to port a bluetooth microphone (BluetoothHFP) to MFi hearing aids connected to an iPhone. When I try to do that the BluetoothHFP grabs input and output. When I pull the output away (BluetoothA2DP) it gives up the input.
5
0
90
18h
Voice Isolation Suggestions
We are working on a voice app that uses ASR/TTS on the backend and run into some difficulties in noisy environments. We have compiled the DeepFilterNet3 library into an XCFramework and are using that on the app, but it's sometimes a bit ambitious with trimming out noise and removes some of the voice. Is there any way to make use of Apple's on-device voice isolation mic mode? I see that we can detect the user's mic mode, but we cannot programmatically set it. While we could prompt the user to enable it manually, this adds a bit of friction to the user experience. Do you have any suggestions for enabling voice isolation, or for performing denoising in general?
1
0
37
19h
Acoustic Echo Cancellation doesn't initially work
We are using AEC in our voice app and it mostly works. However, when the experience begins we play a greeting through the speaker, and the initial few hundred milliseconds of the greeting are being captured by the inputNode. This is throwing off our ASR/TTS. For now, we've disabled audio capture while playing audio, but would prefer to be able to capture all audio with echo cancellation working. Below is some relevant code snippets. Do you have any suggestions to get AEC working more quickly? I've tried a few things like enabling voice processing before setting the audio session to active. public init() { recorderNode = engine.inputNode speakerNode = engine.outputNode mainMixerNode = engine.mainMixerNode engine.attach(audioPlayer) engine.connect( audioPlayer, to: mainMixerNode, format: nil ) playbackFormat = mainMixerNode.outputFormat(forBus: 0) } public func setupAudioSession() async throws(AudioError) { do { let audioSession = AVAudioSession.sharedInstance() try audioSession.setCategory( .playAndRecord, mode: .voiceChat, policy: .default, options: [ .defaultToSpeaker, .allowBluetoothHFP, ] ) try audioSession.setActive(true) } catch { throw .audioSessionSetupFailed(error) } do { try recorderNode.setVoiceProcessingEnabled(true) try speakerNode.setVoiceProcessingEnabled(true) } catch { throw .enableVoiceProcessingFailed(error) } }
1
0
25
19h
Acoustic Echo Cancellation does not work initially
I have a voice app that is both playing and recording audio. I have enabled voice processing and am setting AVAudioSession.Category to .playAndRecord and AVAudioSession.Mode to .voiceChat. When the experience first launches, we play a greeting. The first few hundred milliseconds of that greeting are being captured by the inputNode before AEC seems to start working. Is there any way to get AEC working the entire time? For now we've had to disable recording while we're playing audio, but would prefer to both play and record simultaneously. Here's some code snippets: public init(denoiseModelPath: URL? = nil) { noiseReducer = denoiseModelPath.flatMap { NoiseReducer(modelPath: $0) } recorderNode = engine.inputNode speakerNode = engine.outputNode mainMixerNode = engine.mainMixerNode engine.attach(audioPlayer) engine.connect( audioPlayer, to: mainMixerNode, format: nil ) playbackFormat = mainMixerNode.outputFormat(forBus: 0) } public func setupAudioSession() async throws(AudioError) { do { let audioSession = AVAudioSession.sharedInstance() try audioSession.setCategory( .playAndRecord, mode: .voiceChat, policy: .default, options: [ .defaultToSpeaker, .allowBluetoothHFP, ] ) try audioSession.setActive(true) } catch { throw .audioSessionSetupFailed(error) } do { try recorderNode.setVoiceProcessingEnabled(true) try speakerNode.setVoiceProcessingEnabled(true) } catch { throw .enableVoiceProcessingFailed(error) } }
1
0
19
20h
Voice Isolation Suggestions
We are working on a voice app that uses ASR/TTS on the backend and run into some difficulties in noisy environments. We have compiled the DeepFilterNet3 library into an XCFramework and are using that on the app, but it's sometimes a bit ambitious with trimming out noise and removes some of the voice. Is there any way to make use of Apple's on-device voice isolation mic mode? I see that we can detect the user's mic mode, but we cannot programmatically set it. While we could prompt the user to enable it manually, this adds a bit of friction to the user experience. Do you have any suggestions for enabling voice isolation, or for performing denoising in general?
2
0
64
20h
IOS27beta screenshot error
After taking a screenshot, click the picture to crop and save it. Only the uncropped content will be displayed, not the cropped content.
Replies
0
Boosts
0
Views
7
Activity
9m
Can't create a key for MusicKit
I'm trying to create an app that can access a users Apple Music library to play music (the app will do far more than that when finished) but I am unable to generate a key. When I go to Register a New Key, it says: "There are no identifiers available that can be associated with the key" under Media Services (MusicKit, ShazamKit, Apple Music Feed). MusicKit is already checked under App Services in Certificates, Identifiers & Profiles. I've been stuck on this for two days. Please help!
Replies
0
Boosts
0
Views
1
Activity
22m
SpeechTranscriber Faster Results
I am experimenting with SpeechTranscriber and am curious if I can get quicker results when using buffered audio, rather than a file. The use case is a voice ordering experience for a restaurant. When I've been playing with it, it takes about 3 seconds for faster results and 7-8 seconds for accurate results. Is there any way to bring this down a bit? In this WWDC demo, the results appear nearly instantaneously. I'm curious how to replicate this in my app. I presume DicationTranscriber is faster, but how is siri detecting when the user stops speaking? Is it custom code, or is it using SpeechDetector? I tried using SpeechDetector with SpeechTranscriber but the detector didn't emit any results and seemed to slow down the results of SpeechTranscriber. I also assumed SpeechTranscriber makes more sense than DictationTranscriber in this use case, but want to confirm.
Replies
1
Boosts
0
Views
34
Activity
4h
AVPictureInPictureController doesn't work with AVSampleBufferDisplayLayer on tvOS
When we try to use AVPictureInPictureController with a AVSampleBufferDisplayLayer on tvOS 15 through tvOS 26 isPictureInPicturePossible never becomes true. The code works great on iOS, but it has never worked on tvOS. You can check this out for a repro: https://github.com/jazzychad/PiPBugDemo Feedbacks: FB9751461, FB14158567, FB14037110 DTS Case: 7999337
Replies
6
Boosts
0
Views
153
Activity
13h
test post
This is a test test test
Replies
0
Boosts
1
Views
18
Activity
14h
ABR switching behavior with short (1–2s) segments in live HLS
To reduce latency in our live HLS service, we are experimenting with short segment durations (1–2 seconds). We have noticed that ABR switching occurs more frequently than expected. Looking at the console logs, variants appear to enter and leave what is referred to as a "penalty box" under conditions such as -12889 (a segment returning 0 bytes after 1.0 × target_duration) and -12888 (a playlist remaining unchanged beyond 1.5 × target_duration). Are there recommended practices to reduce this excessive switching? Specifically: Is there an API to control this penalty box behavior? Would preparing an LL-HLS stream with 1-second parts settle this down? Alternatively, would 4-second segments with a slightly smaller configuredTimeOffsetFromLive help stabilize this?
Replies
1
Boosts
2
Views
83
Activity
14h
How to detect if a HDR10+ variant is currently playing
Hi, We currently deliver HDR10+ content within a composite HLS stream where Dolby Vision is signaled as a supplemental codec alongside HDR10+. Our testing on Apple TV 4K (3rd generation) indicates that playback succeeds, but we have not found a reliable way to determine whether playback is occurring in Dolby Vision or HDR10+ mode when connected to a display that supports both formats. We have also verified that the same stream plays correctly on displays that support HDR10+ but do not support Dolby Vision. However, on displays that support both HDR10+ and Dolby Vision, we are unable to determine which dynamic range format AVPlayer ultimately selects. Is there a recommended Apple API or mechanism to determine whether AVPlayer is rendering content as HDR10+, Dolby Vision, or another HDR format at runtime? For reference, below is an example variant from our master playlist: #EXT-X-STREAM-INF:BANDWIDTH=2851276,AVERAGE-BANDWIDTH=2030750,CODECS="hvc1.2.4.L90.90,mp4a.40.2",SUPPLEMENTAL-CODECS="dvh1.08.01/dv1p,hvc1.2.4.L90.90/cdm4",RESOLUTION=854x480,FRAME-RATE=24,VIDEO-RANGE=PQ,HDCP-LEVEL=TYPE-1,CHARACTERISTICS="com.dss.cbcs.hdr.hd",AUDIO="aac-128k",SUBTITLES="sub-main" Specifically, we would like to understand: How AVPlayer chooses between Dolby Vision and HDR10+ when both are available in the same variant. Whether there is a supported way to determine the selected HDR format during playback. Whether any other APIs expose this information. Any guidance would be greatly appreciated. Thank you.
Replies
1
Boosts
1
Views
65
Activity
14h
Real-time audio level monitoring improvements
Have there been any changes in macOS 27 or iOS 27 that improve real-time audio level monitoring, WidgetKit updates, Live Activities, or audio route change handling for professional monitoring applications?
Replies
0
Boosts
0
Views
10
Activity
16h
Voice Processing
Why voice processing enabled on AVAudioInputNode makes output audio noticable lower than without it and how to overcome it using voice processing enabled
Replies
4
Boosts
0
Views
90
Activity
16h
AirPods Custom EQ interaction
In iOS 27, when a user has configured a custom AirPods EQ (three-band lows/mids/highs), does that EQ apply on top of audio output from AVAudioEngine in my app, or only to media playback through AVPlayer/MPMusicPlayerController? My app generates therapeutic frequencies (e.g. 528 Hz pure tones) where precise frequency output matters - is there any API to detect or opt out of the system AirPods EQ?
Replies
1
Boosts
0
Views
28
Activity
17h
Real-time synthesis vs. files for long background sessions
For a sleep app running 8–12 hours in background, is AVAudioSourceNode with a real-time render block more power-efficient than looping a pre-encoded audio file via AVAudioPlayerNode? I want to migrate from files to procedural synthesis but not at the cost of battery. What does Instruments / Energy Log show as the typical CPU overhead difference, and is there Apple guidance on this trade-off?
Replies
1
Boosts
0
Views
34
Activity
17h
Bluetooth mic in, live listen out
Is it possible to link a HFBluetooth input device with the output iPhone speaker while Live Listen is active?
Replies
10
Boosts
0
Views
326
Activity
17h
BlietoothHFP to MFi hearingaids
Is it possible to port a bluetooth microphone (BluetoothHFP) to MFi hearing aids connected to an iPhone. When I try to do that the BluetoothHFP grabs input and output. When I pull the output away (BluetoothA2DP) it gives up the input.
Replies
5
Boosts
0
Views
90
Activity
18h
Voice Isolation Suggestions
We are working on a voice app that uses ASR/TTS on the backend and run into some difficulties in noisy environments. We have compiled the DeepFilterNet3 library into an XCFramework and are using that on the app, but it's sometimes a bit ambitious with trimming out noise and removes some of the voice. Is there any way to make use of Apple's on-device voice isolation mic mode? I see that we can detect the user's mic mode, but we cannot programmatically set it. While we could prompt the user to enable it manually, this adds a bit of friction to the user experience. Do you have any suggestions for enabling voice isolation, or for performing denoising in general?
Replies
1
Boosts
0
Views
37
Activity
19h
Acoustic Echo Cancellation doesn't initially work
We are using AEC in our voice app and it mostly works. However, when the experience begins we play a greeting through the speaker, and the initial few hundred milliseconds of the greeting are being captured by the inputNode. This is throwing off our ASR/TTS. For now, we've disabled audio capture while playing audio, but would prefer to be able to capture all audio with echo cancellation working. Below is some relevant code snippets. Do you have any suggestions to get AEC working more quickly? I've tried a few things like enabling voice processing before setting the audio session to active. public init() { recorderNode = engine.inputNode speakerNode = engine.outputNode mainMixerNode = engine.mainMixerNode engine.attach(audioPlayer) engine.connect( audioPlayer, to: mainMixerNode, format: nil ) playbackFormat = mainMixerNode.outputFormat(forBus: 0) } public func setupAudioSession() async throws(AudioError) { do { let audioSession = AVAudioSession.sharedInstance() try audioSession.setCategory( .playAndRecord, mode: .voiceChat, policy: .default, options: [ .defaultToSpeaker, .allowBluetoothHFP, ] ) try audioSession.setActive(true) } catch { throw .audioSessionSetupFailed(error) } do { try recorderNode.setVoiceProcessingEnabled(true) try speakerNode.setVoiceProcessingEnabled(true) } catch { throw .enableVoiceProcessingFailed(error) } }
Replies
1
Boosts
0
Views
25
Activity
19h
MusicUnderstanding and Apple Music / MusicKit
Am I correct in my understanding that the new MusicUnderstanding is not intended to be able to support analysis of Apple Music streams via MusicKit or in other ways?
Replies
1
Boosts
0
Views
29
Activity
19h
iOS and macOS support of Bluetooth LC3
Do iOS and/or macOS support Bluetooth LC3? If yes, since what iOS/macOS version? is there any limitation and/or caveat?
Replies
2
Boosts
0
Views
36
Activity
20h
Acoustic Echo Cancellation does not work initially
I have a voice app that is both playing and recording audio. I have enabled voice processing and am setting AVAudioSession.Category to .playAndRecord and AVAudioSession.Mode to .voiceChat. When the experience first launches, we play a greeting. The first few hundred milliseconds of that greeting are being captured by the inputNode before AEC seems to start working. Is there any way to get AEC working the entire time? For now we've had to disable recording while we're playing audio, but would prefer to both play and record simultaneously. Here's some code snippets: public init(denoiseModelPath: URL? = nil) { noiseReducer = denoiseModelPath.flatMap { NoiseReducer(modelPath: $0) } recorderNode = engine.inputNode speakerNode = engine.outputNode mainMixerNode = engine.mainMixerNode engine.attach(audioPlayer) engine.connect( audioPlayer, to: mainMixerNode, format: nil ) playbackFormat = mainMixerNode.outputFormat(forBus: 0) } public func setupAudioSession() async throws(AudioError) { do { let audioSession = AVAudioSession.sharedInstance() try audioSession.setCategory( .playAndRecord, mode: .voiceChat, policy: .default, options: [ .defaultToSpeaker, .allowBluetoothHFP, ] ) try audioSession.setActive(true) } catch { throw .audioSessionSetupFailed(error) } do { try recorderNode.setVoiceProcessingEnabled(true) try speakerNode.setVoiceProcessingEnabled(true) } catch { throw .enableVoiceProcessingFailed(error) } }
Replies
1
Boosts
0
Views
19
Activity
20h
Voice Isolation Suggestions
We are working on a voice app that uses ASR/TTS on the backend and run into some difficulties in noisy environments. We have compiled the DeepFilterNet3 library into an XCFramework and are using that on the app, but it's sometimes a bit ambitious with trimming out noise and removes some of the voice. Is there any way to make use of Apple's on-device voice isolation mic mode? I see that we can detect the user's mic mode, but we cannot programmatically set it. While we could prompt the user to enable it manually, this adds a bit of friction to the user experience. Do you have any suggestions for enabling voice isolation, or for performing denoising in general?
Replies
2
Boosts
0
Views
64
Activity
20h
Why are none of my posts showing up in the Audio Q&A?
Why are none of my posts showing up in the Audio Q&A? There are no errors or any useful messaging... my posts just aren't showing.
Replies
2
Boosts
0
Views
35
Activity
20h