Our team conducted security testing and found one vulnerability with fairplay license acquisition.
Our QA engineer manually changed the device's system date and time (setting it 4 days into the future) and was able to successfully obtain a license response and initiate playback on an iOS device. However, on an Android device, the license acquisition failed.
Can you please tell us if Time Manipulation Detection is available in FairPlay SDK?
Explore the integration of media technologies within your app. Discuss working with audio, video, camera, and other media functionalities.
Selecting any option will automatically load the page
Post
Replies
Boosts
Views
Activity
It sounds simple but searching for the name "Favorite Songs" is a non-starter because it's called different names in different countries, even if I specify "&l=en_us" on the query.
So is there another property, relationship or combination thereof which I can use to tell me when I've found the right playlist?
Properties I've looked at so far:
canEdit: will always be false so narrows things down a little
inFavorites: not helpful as it depends on whether the user has favourite the favourites playlist, so not relevant
hasCatalog: seems always true so again may narrow things down a bit
isPublic: doesn't help
Adding the catalog relationship doesn't seem to show anything immediately useful either.
Can anyone help?
Ideally I'd like to see this as a "kind" or "type" as it has different properties to other playlists, but frankly I'll take anything at this point.
We encounter issue with avplayer in case of EXT-X-DISCONTINUITY misalignment between audio and video produced after insertion of gaps.
The initial objective is to introduce an EXT-X-DISCONTINUITY in audio playlist after some missing segments (EXT-X-GAP) which durations are aligned to video segments durations, to handle irregular audio durations.
Please find below an example of corresponding video and audio playlists:
video:
#EXTM3U
#EXT-X-VERSION:7
#EXT-X-MEDIA-SEQUENCE:872524632
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-TARGETDURATION:2
#USP-X-TIMESTAMP-MAP:MPEGTS=7096045027,LOCAL=2025-05-09T12:38:32.369100Z
#EXT-X-MAP:URI="hls/StreamingBasic-video=979200.m4s"
#EXT-X-PROGRAM-DATE-TIME:2025-05-09T12:38:32.369111Z
#EXTINF:2.002, no desc
hls/StreamingBasic-video=979200-872524632.m4s
#EXTINF:2.002, no desc
hls/StreamingBasic-video=979200-872524633.m4s
#EXTINF:2.002, no desc
hls/StreamingBasic-video=979200-872524634.m4s
#EXTINF:2.002, no desc
hls/StreamingBasic-video=979200-872524635.m4s
#EXTINF:2.002, no desc
hls/StreamingBasic-video=979200-872524636.m4s
## Media sequence discontinuity
#EXT-X-GAP
#EXTINF:2.002, no desc
hls/StreamingBasic-video=979200-872524637.m4s
## Media sequence discontinuity
#EXT-X-GAP
#EXTINF:2.002, no desc
hls/StreamingBasic-video=979200-872524638.m4s
#EXT-X-PROGRAM-DATE-TIME:2025-05-09T12:38:46.383111Z
#EXTINF:2.002, no desc
hls/StreamingBasic-video=979200-872524639.m4s
#EXTINF:2.002, no desc
hls/StreamingBasic-video=979200-872524640.m4s
audio:
EXTM3U
#EXT-X-VERSION:7
#EXT-X-MEDIA-SEQUENCE:872524632
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-TARGETDURATION:2
#USP-X-TIMESTAMP-MAP:MPEGTS=7096045867,LOCAL=2025-05-09T12:38:32.378400Z
#EXT-X-MAP:URI="hls/StreamingBasic-audio_99500_eng=98800.m4s"
#EXT-X-PROGRAM-DATE-TIME:2025-05-09T12:38:32.378444Z
#EXTINF:2.0053, no desc
hls/StreamingBasic-audio_99500_eng=98800-872524632.m4s
#EXTINF:2.0053, no desc
hls/StreamingBasic-audio_99500_eng=98800-872524633.m4s
#EXTINF:2.0053, no desc
hls/StreamingBasic-audio_99500_eng=98800-872524634.m4s
#EXTINF:1.984, no desc
hls/StreamingBasic-audio_99500_eng=98800-872524635.m4s
#EXTINF:2.0053, no desc
hls/StreamingBasic-audio_99500_eng=98800-872524636.m4s
## Media sequence discontinuity
#EXT-X-GAP
#EXTINF:2.002, no desc
hls/StreamingBasic-audio_99500_eng=98800-872524637.m4s
## Media sequence discontinuity
#EXT-X-GAP
#EXTINF:2.002, no desc
hls/StreamingBasic-audio_99500_eng=98800-872524638.m4s
#EXT-X-DISCONTINUITY
#EXT-X-PROGRAM-DATE-TIME:2025-05-09T12:38:46.778444Z
#EXTINF:1.6213, no desc
hls/StreamingBasic-audio_99500_eng=98800-872524639.m4s
#EXTINF:2.0053, no desc
hls/StreamingBasic-audio_99500_eng=98800-872524640.m4s
In this case playback is broken with avplayer.
Is it conformed to Http Live Streaming?
Is it an avplayer bug?
What are the guidelines to handle such gaps?
I bought two "Apple USB-C to Headphone Jack Adapters". Upon closer inspection, they seems to be of different generations:
The one with product ID 0x110a on top is working fine. The one with product ID 0x110b has two issues:
There is a short but loud click noise on the headphone when I connect it to the iPad.
When I play audio using AVAudioPlayer the first half of a second or so is cut off.
Here's how I'm playing the audio:
audioPlayer = try AVAudioPlayer(contentsOf: url)
audioPlayer?.delegate = self
audioPlayer?.prepareToPlay()
audioPlayer?.play()
Is this a known issue? Am I doing something wrong?
Hello, my company is developing a product that will send data to/from the phone over cable and Wi-Fi. I have three questions:
Do we need an MFi authentication chip in our product if we plan to send video and commands to the iPhone/iPad over USB or Lightning cable?
Likewise, do we need an MFI authentication chip for communication over Wi-Fi? (Informal research suggests that the answer is no to this one.)
And, do we even still need MFI certification at all for Wi-Fi comms? (We are not using HomeKit.)
Thank you!
Topic:
Media Technologies
SubTopic:
Photos & Camera
I'm developing a Final Cut Pro X workflow extension that transcribes audio and creates a text output. I need to allow users to drag this text directly from my extension into FCPX's timeline as titles.
Current Implementation:
Using NSFilePromiseProvider as per Apple's guidelines for drag and drop
Generating valid FCPXML (v1.10) with proper structure:
Complete resources section with format and asset references
Event and project hierarchy
Asset clip with connected title elements
Proper timing and duration calculations
Supporting multiple pasteboard types:
com.apple.finalcutpro.xml.v1-10
com.apple.finalcutpro.xml.v1-9
com.apple.finalcutpro.xml
What's Working:
Drag operation initiates correctly
File promise provider is set up properly
FCPXML generation is successful (verified content)
All required pasteboard types are registered
Proper logging confirms data is being requested and provided
Current Pasteboard Types Offered:
com.apple.NSFilePromiseItemMetaData
com.apple.pasteboard.promised-file-name
com.apple.pasteboard.promised-suggested-file-name
com.apple.pasteboard.promised-file-content-type
Apple files promise pasteboard type
com.apple.pasteboard.NSFilePromiseID
com.apple.pasteboard.promised-file-url
com.apple.finalcutpro.xml.v1-10
com.apple.finalcutpro.xml.v1-9
com.apple.finalcutpro.xml
What additional requirements or considerations are needed to make FCPX accept the dragged FCPXML content? Are there specific requirements for workflow extensions regarding drag and drop operations with titles that aren't documented?
Any insights, especially from those who have implemented similar functionality in FCPX workflow extensions, would be greatly appreciated.
Technical Details:
macOS Version: 15.5 (24F74)
FCPX Version: 11.1.1
Extension built with SwiftUI and AppKit integration
Using NSFilePromiseProvider and NSPasteboardItemDataProvider
Full pasteboard type support for FCPXML versions
We are planning to develop an application using the Apple Music API.
We would like to design our system based on the details of the rate limits mentioned below and have a few questions:
https://developer.apple.com/documentation/applemusicapi/generating-developer-tokens#Request-Rate-Limiting
Regarding the Catalog API (/v1/catalog/*), we understand that server-side caching is enabled, making it less likely to reach the rate limit. Is this understanding correct? (Excluding the search API)
For APIs like the Library API (/v1/me/library/*), where responses vary by user, we assume they are more likely to reach the rate limit. Is this correct?
We plan to implement optimizations to minimize unnecessary API calls. Given this, would the current Music API be able to handle a significant increase in users? (Assuming a DAU of around 100,000 to 1,000,000)
If the API cannot support this scale, would it be allowed under Apple’s policy to cache responses from the Catalog API (/v1/catalog/*) via our proxy server to avoid hitting the rate limit?
The third question is the one we most want to confirm.
AVPictureInPictureControllerContentSource *contentSource = [[AVPictureInPictureControllerContentSource alloc] initWithSampleBufferDisplayLayer:self.renderView.sampleBufferDisplayLayer playbackDelegate:self];
AVPictureInPictureController *pictureInPictureController = [[AVPictureInPictureController alloc] initWithContentSource:contentSource];
pictureInPictureController.delegate = self;
(void)pictureInPictureController:(AVPictureInPictureController *)pictureInPictureController failedToStartPictureInPictureWithError:(NSError *)error
{
//error NSError * domain: @"PGPegasusErrorDomain" - code: -1003 0x00000002819fe3a0
}
when first start the PiP play, I got the error "//error NSError * domain: @"PGPegasusErrorDomain" - code: -1003 0x00000002819fe3a0", why?
and second start is Ok.
I want to create a Live Photo. The project includes a .jpg image and a .mov video (2 seconds).
Two permissions in xcode have been added:
Privacy - Photo Library Usage Description
Privacy - Photo Library Additions Usage Description
Simulate: iphone 16, ios 18.3
The codes in ContentView.swift :
private func saveLivePhoto(imageURL: URL, videoURL: URL, completion: @escaping (Bool, Error?) -> Void) {
PHPhotoLibrary.shared().performChanges {
let creationRequest = PHAssetCreationRequest.forAsset()
let options = PHAssetResourceCreationOptions()
options.shouldMoveFile = false
creationRequest.addResource(with: .photo, fileURL: imageURL, options: options)
creationRequest.addResource(with: .pairedVideo, fileURL: videoURL, options: options)
} completionHandler: { success, error in
DispatchQueue.main.async {
print(error)
completion(success, error)
}
}
}
guard let imageURL = Bundle.main.url(forResource: "livephoto", withExtension: "jpeg"),
let videoURL = Bundle.main.url(forResource: "livephoto", withExtension: "mov") else {
showAlertMessage(title: "error", message: "cant find Live Photo ")
return
}
print("imageURL: \(imageURL)")
print("videoURL: \(videoURL)")
saveLivePhoto(imageURL: imageURL, videoURL: videoURL) { success, error in
if success {
xxxxx
} else {
xxxxx
}
}
Really need help, thanks
My app Balletrax is a music player for people to use while they teach ballet. Used to be you could silence notifications during use, but now the customer seems to have to know how to use Focus mode, remember to turn it on and off, and have to check the notifications one does and doesn't want to use. Is there no way to silence all notifications when the app is in use?
I am developing an iOS app that needs to play spoken audio on demand from a server, while ducking the audio of background music from another app (e.g., SoundtrackYourBrand or Apple Music). This must work even when the app is in the background, and the server dictates when and what audio is played. Ideally, the message should be played within a minute of the server requesting it.
Current Attempt & Observations
I initially tried using Firebase Cloud Messaging (FCM) silent notifications to send a URL to an audio file, which the app would then play using AVPlayer.
This works consistently when the app is active, but in the background, it only works about 60% of the time.
In cases where it fails, iOS ducks the background music (e.g., from SoundtrackYourBrand) but never plays the spoken audio.
Interestingly, when I play the audio without enabling audio ducking, it seems to work 100% of the time from my limited testing, even in the background.
The app has background modes enabled for Audio, Background Fetch, and Remote Notifications.
Best Approach to Achieve This?
I’d like guidance on the best Apple-compliant approach to reliably play audio on command from the server, even when the app is in the background. Some possible paths:
Ensuring the app remains active in the background – Are there recommended ways to prevent the app from getting suspended, such as background tasks, a special background mode, or a persistent connection to the server?
Alternative triggering mechanisms – Would something like VoIP, Push-to-Talk, or another background service be better suited for this use case?
Built-in iOS speech synthesis (AVSpeechSynthesizer) – If playing external audio is unreliable, would generating speech dynamically from text be a more robust approach?
Streaming audio instead of sending a URL – Could continuous streaming from the server keep the app active and allow playback at the right moment?
I want to ensure the solution is reliable and works 100% of the time when needed. Any recommendations on the best approach for this would be greatly appreciated.
Thank you for your time and guidance.
Hello,
I'm developing an app that displays a photo library using UICollectionView and PHCachingImageManager. I'd like to achieve a user experience similar to the native iOS Photos app, where low-quality images are shown quickly while scrolling, and higher-quality images are loaded for visible cells once scrolling stops.
I'm currently using the following approach:
While Scrolling: I'm using the UICollectionViewDataSourcePrefetching protocol. In the prefetchItemsAt method, I call startCachingImages with low-quality options to cache images in advance.
After Scrolling Stops: In the scrollViewDidEndDecelerating method, I intend to load high-quality images for the currently visible cells.
I have a few questions regarding this approach:
What is the best practice for managing both low-quality and high-quality images efficiently with PHCachingImageManager? Is it correct to call startCachingImages with fastFormat options and then call it again with highQualityFormat in scrollViewDidEndDecelerating?
How can I minimize the delay when a low-quality image is replaced by a high-quality one? Are there any additional strategies to help pre-load high-quality images more effectively?
Topic:
Media Technologies
SubTopic:
Photos & Camera
On Apple TV 4K 3rd generation, with tvOS 26 beta 2, when two HomePod 2 are paired to the device, music and movie sources with Dolby Atmos can only be listened to in stereo. dolby atmos not supported
Topic:
Media Technologies
SubTopic:
Audio
Hello everyone,
I’m new to Swift development and have been working on an audio module that plays a specific sound at regular intervals - similar to a workout timer that signals switching exercises every few minutes.
Following AVFoundation documentation, I’m configuring my audio session like this:
let session = AVAudioSession.sharedInstance()
try session.setCategory(
.playback,
mode: .default,
options: [.interruptSpokenAudioAndMixWithOthers, .duckOthers]
)
self.engine.attach(self.player)
self.engine.connect(self.player, to: self.engine.outputNode, format: self.audioFormat)
try? session.setActive(true)
When it’s time to play cues, I schedule playback on a DispatchQueue:
// scheduleAudio uses DispatchQueue
self.scheduleAudio(at: interval.start) {
do {
try audio.engine.start()
audio.node.play()
for sample in interval.samples {
audio.node.scheduleBuffer(sample.buffer, at: AVAudioTime(hostTime: sample.hostTime))
}
} catch {
print("Audio activation failed: \(error)")
}
}
This works perfectly in the foreground. But once the app goes into the background, the scheduled callback runs, yet the audio engine fails to start, resulting in an error with code 561015905.
Interestingly, if the app is already playing audio before going to the background, the scheduled sounds continue to play as expected.
I have added the required background audio mode to my Info plist file by including the key UIBackgroundModes with the value audio.
Is there anything else I should configure? What is the best practice to play periodic audio when the app runs in the background? How do apps like turn-by-turn navigation handle continuous audio playback in the background?
Any advice or pointers would be greatly appreciated!
(Note: this is part 2 of a 3 part posting. See Part 1 or Part 3)
At WWDC25 we launched a new type of Lab event for the developer community - Group Labs. A Group Lab is a panel Q&A designed for a large audience of developers. Group Labs are a unique opportunity for the community to submit questions directly to a panel of Apple engineers and designers. Here are the highlights from the WWDC25 Group Lab for Camera & Photos.
WWDC25 Camera & Photos group lab ran for one hour at 6 PM PST on Tuesday June 10th, 2025
Question 10
Can we directly integrate auto-capture triggers (e.g., when image is steady or text is detected) using Vision and AVFoundation?
Yes apps can use AVCaptureSession's VDO + AVCapturePhotoOutput, run vision on VDO buffers and capture photo when certain scene or text is detected.
Just to be careful to run Vision on VDO buffers async so it doesn't cause frame drops.
Question 11
What Camera or Photos framework features support working with images from external media, like connected cameras or SD cards? Any best practices?
The ImageCaptureCore framework supports camera devices, memory cards, scanners
read and write, where supported
check out the docs to see how to browse connected devices, folders, files, etc.
Question 12
Hi Brad, to follow up on your SwiftUI cautionary note: using AVCaptureVideoPreview inside a UIViewRepresentable, is okay, right? Thanks all for the great info!
Yes, this is totally fine.
AppKit or UIKit views inside appropriate SwiftUI representables should be equivalent performance
Question 13
What’s the “right” way to transition media in my photos app between HDR modes? When I’m in a one-up view, we use HDR, but in other contexts (like thumbnail) we don’t want HDR. Is there a nice way to tone map?
There’s a suite of new System Tone Mapper APIs in this years’ OSes
CoreImage ImageKit CoreAnimation, CoreGraphics
For example:
CoreImage: new CISystemToneMap filter.
CoreAnimation: layer.preferredDynamicRange = CADynamicRangeConstrainedHigh
Using image views (NSImageView/UIImageView/SwiftUI Image/CALayer) support animations on preferredDynamicRange
Can go from high to constrained to standard
Tone mapping is provided by the system (CISystemToneMap for controllable example)
Question 14
What is your recommendation to preprocess and upscale your depth map in order to render a realistic portrait mode image?
One way to do this: the CIEdgePreserveUpsample CIFilter can be use to upsample a lower resolution depth map by using a higher resolution RGB image as a guide.
Question 15
For buffering frames for later processing from real-time camera output should we prefer a AVSampleBufferDisplayLayer centered approach or AVCaptureVideoDataOutputSampleBufferDelegate centered approach? When would we use each?
AVSampleBufferDisplayLayer and AVCaptureVideoDataOutputSampleBufferDelegate are used hand in hand for custom camera preview.
For buffering for later processing, ensure you make copies of VDO buffers to not drop frames from the output
Question 16
Hello, my question is on Deferred Photo Processing? Say I have a photo capture app that adds a CIFilter to the capture. How can I take advantage of Deferred Photo Processing? Since I don’t know how to detect when the deferred captured photo is ready
CIFilter can be called on the final at that point
Photo will have to be re-inserted into the Photo library as adjustment
Question 17
Is digital zoom (e.g., 1.5x) before taking a photo the same as cropping the photo afterward?
digital zoom upscales the image to output dimensions and cropping will yield a smaller output image
while digital zoom will crop, it also upscales
Question 18
How do you design camera interfaces that work for both casual users and photography enthusiasts?
Progressive disclosure: Put the most common controls up front, and make it easy for pros to drill down.
Sensible Defaults: Choose defaults that work well for casual users, but allow those defaults to be modified for photography enthusiasts
A good philosophy is: Keep the simple things easy, make the hard things possible
Question 19
Recent iPhone models introduced macro mode which automatically switch between lenses to take into account of the focal distance difference. Is there official API to implement this, or should I implement them myself using LiDAR values.
Using builtInTripleCamera and builtInDualWideCamera will automatically switch to macro when available
Question 20
a couple of years ago at WWDC, the option of replacing a camera with a virtual camera was mentioned. How does one do that - make the “physical” camera effectively disappear, so only the virtual camera is accessible to the user?
You can't prevent the built-in camera from being available to the user
Question 21
Can developers now integrate custom Core ML models with Vision for on-device photo analysis more seamlessly?
Yes they can, use CoreMLRequest , provide their model container
Been supported for a while (iOS 18/macOS 15)
For more details go to Machine Learning & AI group lab Thursday
use smaller images for better performance
Question 22
What would you recommend for capture of the new immersive and spatial formats?
To capture Spatial Video use AVCaptureMovieFileOutput’s spatialVideoCaptureEnabled property
Not all device formats support spatial capture, check AVCaptureDevice.activeFormat.spatialVideoCaptureSupported
See WWDC 2024 talk “Build compelling spatial photo and video experiences” for more details
Question 23
You mentioned JPEG-XL. What is the current status of support on iOS and macOS for encoding and decoding?
For decoding, we support JPEG-XL files in all our OSes, regular SDR files, as well as ISO HDR files.
For encoding, we only support JPEG-XL for ProRAW DNG capture in the Camera app or via third-party AVFoundation APIs.
If you have any requests for improvement or new features related to JPEG-XL, please file a Feedback request using the Feedback Assistant.
(Note: this is part 2 of a 3 part posting. See Part 1 or Part 3)
Topic:
Media Technologies
SubTopic:
Photos & Camera
Tags:
Image I/O
Photos and Imaging
PhotoKit
Core Image
This is my native module code implementation
I'm getting base64 encoded string from server and passing this to my native module of pcm player to play audio
App.tsx
PcmPlayer.writeChunk(e.data);
PcmPlayer.swift
import AVFoundation
@objc(PcmPlayer)
class PcmPlayer: RCTEventEmitter {
private var engine: AVAudioEngine?
private var playerNode: AVAudioPlayerNode?
private var format: AVAudioFormat?
private var bufferQueue = [Data]()
private var isPlaying = false
private var hasEnded = false
private var scheduledBufferCount = 0
private let minBufferBytes = 50000
private let pcmQueue = DispatchQueue(label: "pcm.queue")
override init() {
super.init()
}
override func supportedEvents() -> [String]! {
return ["onStatus", "onMessage"]
}
@objc(initPlayer:channels:bitsPerSample:)
func initPlayer(_ sampleRate: NSNumber,
channels: NSNumber,
bitsPerSample: NSNumber) {
pcmQueue.async {
self.stopInternal()
let session = AVAudioSession.sharedInstance()
do {
try session.setCategory(.playback, mode: .default, options: [])
try session.setActive(true, options: .notifyOthersOnDeactivation)
try session.setMode(.default)
print("🔈 Audio session active. Output route:", session.currentRoute.outputs)
} catch {
print("❌ Audio session setup failed:", error)
return
}
self.engine = AVAudioEngine()
self.playerNode = AVAudioPlayerNode()
guard let engine = self.engine, let playerNode = self.playerNode else {
print("❌ Engine or playerNode is nil")
return
}
engine.attach(playerNode)
self.format = AVAudioFormat(commonFormat: .pcmFormatFloat32,
sampleRate: sampleRate.doubleValue,
channels: AVAudioChannelCount(channels.uintValue),
interleaved: false)
guard let format = self.format else {
print("❌ Failed to create AVAudioFormat")
return
}
engine.connect(playerNode, to: engine.mainMixerNode, format: format)
do {
try engine.start()
playerNode.play()
engine.mainMixerNode.outputVolume = 1.0
print("✅ AVAudioEngine started with format:", format)
} catch {
print("❌ Engine start failed:", error)
}
self.hasEnded = false
}
}
@objc(writeChunk:)
func writeChunk(_ base64Pcm: String) {
pcmQueue.async {
guard base64Pcm.count >= 10 else {
print("⚠️ Skipping short base64 string")
return
}
var padded = base64Pcm
let mod4 = base64Pcm.count % 4
if mod4 > 0 {
padded += String(repeating: "=", count: 4 - mod4)
}
guard let data = Data(base64Encoded: padded, options: .ignoreUnknownCharacters) else {
print("❌ Failed to decode base64")
return
}
self.bufferQueue.append(data)
print("📥 Received PCM chunk (\(data.count) bytes)")
print("📥 writeChunk called. isPlaying=\(self.isPlaying), bufferQueue.count=\(self.bufferQueue.count)")
if !self.isPlaying {
self.isPlaying = true
self.waitForBufferAndStartPlayback()
} else if self.scheduledBufferCount == 0 {
self.isPlaying = true
self.waitForBufferAndStartPlayback()
}
}
}
private func waitForBufferAndStartPlayback() {
DispatchQueue.global().async {
while self.queueSize() < self.minBufferBytes && !self.hasEnded {
Thread.sleep(forTimeInterval: 0.01)
}
self.writeLoop()
}
}
private func writeLoop() {
DispatchQueue.global().async {
writeLoop: while self.isPlaying {
if self.bufferQueue.isEmpty {
for _ in 0..<100 {
Thread.sleep(forTimeInterval: 0.01)
if !self.bufferQueue.isEmpty { break }
}
if self.bufferQueue.isEmpty {
print("🔇 No more data to play after waiting")
self.isPlaying = false
break writeLoop
}
}
var data: Data?
self.pcmQueue.sync {
if !self.bufferQueue.isEmpty {
data = self.bufferQueue.removeFirst()
}
}
guard let chunk = data else {
print("⚠️ No data to process")
continue
}
if let buffer = self.pcmBufferFromData(chunk) {
self.scheduledBufferCount += 1
self.playerNode?.scheduleBuffer(buffer, completionHandler: {
self.pcmQueue.async {
self.scheduledBufferCount -= 1
if self.bufferQueue.isEmpty && self.scheduledBufferCount == 0 {
print("ℹ️ Playback idle - waiting for more data")
self.isPlaying = false
}
}
})
}
}
}
}
private func pcmBufferFromData(_ data: Data) -> AVAudioPCMBuffer? {
guard let format = self.format else { return nil }
let frameCount = UInt32(data.count / 2)
guard let buffer = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: frameCount) else {
print("❌ Failed to create AVAudioPCMBuffer")
return nil
}
buffer.frameLength = frameCount
guard let floatChannelData = buffer.floatChannelData?[0] else {
print("❌ floatChannelData is nil")
return nil
}
data.withUnsafeBytes { (rawBuffer: UnsafeRawBufferPointer) in
let int16Buffer = rawBuffer.bindMemory(to: Int16.self)
let count = min(int16Buffer.count, Int(frameCount))
for i in 0..<count {
floatChannelData[i] = Float32(int16Buffer[i]) / Float32(Int16.max)
}
}
return buffer
}
@objc(stopPlayer)
func stopPlayer() {
pcmQueue.async {
self.stopInternal()
}
}
private func stopInternal() {
print("🛑 stopInternal called")
self.playerNode?.stop()
self.engine?.stop()
self.engine?.reset()
self.playerNode = nil
self.engine = nil
self.format = nil
self.bufferQueue.removeAll()
self.isPlaying = false
self.hasEnded = true
self.scheduledBufferCount = 0
}
@objc(canWrite:rejecter:)
func canWrite(_ resolve: @escaping RCTPromiseResolveBlock,
rejecter reject: RCTPromiseRejectBlock) {
pcmQueue.async {
resolve(self.bufferQueue.count < 20)
}
}
@objc(flushPlayer:rejecter:)
func flushPlayer(_ resolve: @escaping RCTPromiseResolveBlock,
rejecter reject: RCTPromiseRejectBlock) {
pcmQueue.async {
self.bufferQueue.removeAll()
resolve(nil)
}
}
@objc
static override func requiresMainQueueSetup() -> Bool {
return false
}
private func queueSize() -> Int {
return pcmQueue.sync {
return self.bufferQueue.reduce(0) { $0 + $1.count }
}
}
}
I couldn't able to hear any audio via my real iOS device also it is working fine on emulator.
Topic:
Media Technologies
SubTopic:
Streaming
I'm creating an app that uses AVCaptureSession to pass camera input to AVCaptureMetadataOutput type set [metaout setMetadataObjectTypes:@[AVMetadataObjectTypeFace]] and scan Face.
After updating to OS 26 Beta2 and iOS 26 Beta2, an issue has occurred where the delegate method of AVCaptureMetadataOutputObjectsDelegate is not called on some devices. The following devices are experiencing this issue.
iPad (9th Gen)
iPad air (4th Gen)
iPhone 15
This issue has not occur on any other devices I have.
I tried running the AVFoundation sample code on the Apple Developer site on the above device. The same problem still occurs. https://developer.apple.com/documentation/avfoundation/capture_setup/avcambarcode_detecting_barcodes_and_faces
Are any additional settings required after OS 26 beta and iOS 26 beta? Or is there some problem on the OS side?
I'm capturing video stream from GoPro camera (via UDP MPEG-TS packets) but unable to play in iOS app. can someone provide a source code to decode MPEG-TS data to CMSampleBuffer.
Hi,
Currently I am developing a 3D reconstruction project.
Which requires images to be distortion-free (rectilinear) and with known intrinsics.
The session I am developing on is a builtInDualWideCamera, with isGeometricDistortionCorrectionEnabled set to false to be able to get the intrinsic matrix of the images, isVirtualDeviceConstituentPhotoDeliveryEnabled set to true and isAutoVirtualDeviceFusionEnabled set to false to get both images and isCameraCalibrationDataDeliveryEnabled set to true to actually get the calibration data.
The distortion correction parameters such as lensDistortionLookupTable are used.
The 42 coefficients mapping array is used as described in the AVCameraCalibrationData header file. A simple piecewise linear interpolation.
There are two questions I would like to get support on:
A way to set the calibration parameters in each image.
I have an approach that sets the parameters in the kCGImagePropertyExifDictionary -> "UserComment". Is there a better approach to write calibration parameter data into the images? I feel like this is a bit dirty and there might be a better and neat approach.
For the ultra-wide angle camera's images, the lensDistortionLookupTable contains several zeros at the end of the array.
For example (last 10 elements are zero):
"LensDistortionLookupTable":"0.000000000000000,0.000349554029526,0.001385628827848,0.003071037586778,... ,0.000000000000000,0.000000000000000,0.000000000000000,0.000000000000000,0.000000000000000,0.000000000000000,0.000000000000000,0.000000000000000,0.000000000000000,0.000000000000000"
The problem comes when the complete array is used to correct the image (including zeros), the end result is a wrapped-like-circle image close to the edges of it which is completely wrong.
In contrast, if the LensDistortionLookupTable is used without the last zeros and the new size accommodated the image looks better (although not as rectilinear as if you take the image from the iPhone's camera app), but definitely less distorted.
Including zeros (full array):
Excluding zeros (array size changed):
Am I missing an important point in the usage of the lensDistortionLookupTable where this case is addressed (zeros at the end)?
What is the criteria to shrink/exclude elements of the array?
Any advice is very much welcome.
Before you post —Camera doesn't work on the Simulator— that's no longer true. I've made a solution that makes the Simulator believe there's an actual hardware device connected, allowing users to stream the macOS camera to the iOS Simulator (see for more info RocketSim's documentation: https://docs.rocketsim.app/features/hzQMSrSga7BGWvxdNVdwYs/simulator-camera-support/58tQ5jvevLNSnyUEA7VgAv)
Now, it works for VNDocumentCameraViewController, but when I try opening DataScannerViewController, I directly run into:
Failed to start scanning: The operation couldn’t be completed. (VisionKit.DataScannerViewController.ScanningUnavailable error 0.)
My question:
How does this view controller determine whether scanning is available?
Is there a certain capability the available AVCaptureDevice's need to support maybe?
Any direction would be helpful for me to make this work for developers, making them build apps faster!