Explore the integration of media technologies within your app. Discuss working with audio, video, camera, and other media functionalities.

All subtopics

Post

Replies

Boosts

Views

Activity

Is it possible to get only audio from ScreenCaptureKit?
I'm creating app that listening other app's sound. in this use case, screen data is not needed. but if I don't call SCStream#addStreamOutput(_, type: .screen, ...), console shows this error: [ERROR] _SCStream_RemoteVideoQueueOperationHandlerWithError:701 stream output NOT found. Dropping frame currently I'm setting SCStreamConfiguration#minimumFrameInterval to large value (e.g. 0.1fps) as workaround, but it would be good if i can completely disable screen capture for best performance. there is any way to disable screen capture and only captures apps audio?
5
1
2.7k
Oct ’22
CoreMediaErrorDomain Code=-12884
I have an AVPlayerViewController in my app playing video and my largest source of errors is NSURLErrorDomain Code=-1008. The underlying error they provide is Error Domain=CoreMediaErrorDomain Code=-12884 "(null)". I couldn't find what the error implies or is caused by - osstatus.com also does not have a reference to it.
2
0
2.3k
Nov ’22
iOS 16 Music Kit MusicLibrary methods have stopped working
Hi there, I have a related forum thread here and a Feedback Assistant ticket open, but this issue seems different. Sometime within the last 2-3 weeks, code related to MusicLibrary has stopped working. None of my code has changed. For example, the below two snippets used to work fine:  for track in newTracks {    try await MusicLibrary.shared.add(track, to: targetPlaylist)  } try await MusicLibrary.shared.edit(targetPlaylist, items: items) newTracks and items are both fetched using: try await targetPlaylist.with(.tracks, preferredSource: .catalog).tracks Using preferredSource: .catalog was a workaround used to address the issue in the aforementioned post above. All iOS 16 capable functions are decorated with: @available(iOS 16, *) or in an if block: if #available(iOS 16, *) {... What's happening is that the following is showing up in the console: 2022-11-28 23:31:11.279648+0700 MyApp[38653:6736450] [core] Attempted to register account monitor for types client is not authorized to access: {(     "com.apple.account.iTunesStore" )} 2022-11-28 23:31:11.279718+0700 MyApp[38653:6736450] [Default] <ICUserIdentityStoreACAccountBackend: 0x282adb520> Failed to register for account monitoring. err=Error Domain=com.apple.accounts Code=7 "(null)" 2022-11-28 23:31:11.279758+0700 MyApp[38653:6736450] [Default] ICUserIdentity - Unable to retrieve DSID for userIdentity=<ICUserIdentity 0x2806eb120: [Active Account: <unresolved>]> - error=Error Domain=com.apple.accounts Code=7 "(null)" These errors are not caught by a do/catch block, but I assume they are related to the issue, and I believe they have to do with MyApp trying to access things that Music Kit thinks it's not supposed to. For example, if MyApp attempts to work with a playlist that it did not create, errors would be expected, thrown errors. The thing is that I know I'm working with resources that are created by MyApp. In fact, in trying to test this, I just tried to create a playlist with the below, and the same behavior is occurring: @available(iOS 16, *) func createPlaylist2(name: String, description: String) async -> MusicKit.Playlist? {     do {         Logger.log(.info, "Creating Playlist: \(name)")         Logger.log(.shrug, "Does this work?")         let newPlaylist = try await MusicLibrary.shared.createPlaylist(name: name, description: description) // <= Things stop here!         Logger.log(.success, "New playlist created: \(newPlaylist)") // <= this isn't logged.         return newPlaylist // <= nothing is returned     } catch {         Logger.log(.error, "Could not create new playlist: \(error)") // <= no error logged.     }     return nil } The result is: 2022-11-29 00:15:01.875064+0700 MyApp[38794:6760471] [core] Attempted to register account monitor for types client is not authorized to access: {(     "com.apple.account.iTunesStore" )} 2022-11-29 00:15:01.875372+0700 MyApp[38794:6760471] [Default] <ICUserIdentityStoreACAccountBackend: 0x283005720> Failed to register for account monitoring. err=Error Domain=com.apple.accounts Code=7 "(null)" 2022-11-29 00:15:01.876677+0700 MyApp[38794:6760323] [EntityQuery] Finished executing query in 0.000999928s 2022-11-29 00:15:01.889055+0700 MyApp[38794:6760323] [EntityQuery] Finished fetching results in 0.0120001s 2022-11-29 00:15:01.891235+0700 MyApp[38794:6760329] [core] Attempted to register account monitor for types client is not authorized to access: {(     "com.apple.account.iTunesStore" )} 2022-11-29 00:15:01.891684+0700 MyApp[38794:6760329] [Default] <ICUserIdentityStoreACAccountBackend: 0x283005720> Failed to register for account monitoring. err=Error Domain=com.apple.accounts Code=7 "(null)" 📘 Creating Playlist: TEST PLAYLIST 🤷🏻‍♀️ Does this work? 2022-11-29 00:15:06.697374+0700 MyApp[38794:6760329] [] nw_path_necp_check_for_updates Failed to copy updated result (22) What's really nasty is that errors are not thrown, so they can't be caught and handled in a catch block. I know that iOS 16.1 got released around the end of October, but I really don't know what's going on here. The behavior is showing up in both prod and when testing locally. Any help would be most appreciated. @JoeKhun: Did I miss the memo?
10
1
5.1k
Nov ’22
Ventura Hack for FireWire Core Audio Support on Supported MacBook Pro and others...
Hi all,  Apple dropping on-going development for FireWire devices that were supported with the Core Audio driver standard is a catastrophe for a lot of struggling musicians who need to both keep up to date on security updates that come with new OS releases, and continue to utilise their hard earned investments in very expensive and still pristine audio devices that have been reduced to e-waste by Apple's seemingly tone-deaf ignorance in the cries for on-going support.  I have one of said audio devices, and I'd like to keep using it while keeping my 2019 Intel Mac Book Pro up to date with the latest security updates and OS features.  Probably not the first time you gurus have had someone make the logical leap leading to a request for something like this, but I was wondering if it might be somehow possible of shoe-horning the code used in previous versions of Mac OS that allowed the Mac to speak with the audio features of such devices to run inside the Ventura version of the OS.  Would it possible? Would it involve a lot of work? I don't think I'd be the only person willing to pay for a third party application or utility that restored this functionality. There has to be 100's of thousands of people who would be happy to spare some cash to stop their multi-thousand dollar investment in gear to be so thoughtlessly resigned to the scrap heap.  Any comments or layman-friendly explanations as to why this couldn’t happen would be gratefully received!  Thanks,  em
54
9
26k
Dec ’22
MusicKit MusicCatalogResourceRequest for artists with top-songs returns 504 in non-json format
I'm making a request to get 10 artists with their top songs at once, but for some artists it will always fail with a 504. The response is also in HTML which leads to a decoding error. This is my code var request = MusicCatalogResourceRequest<Artist>(matching: \.id, memberOf: ids) request.properties = properties let response = try await request.response() where ids is MusicItemId. Below I have an input which will always fail 100% of the time, even when retried. 10 elements  - 0 : "51639"  - 1 : "331584"  - 2 : "120199"  - 3 : "45058"  - 4 : "284786497"  - 5 : "44984"  - 6 : "37299"  - 7 : "518462"  - 8 : "39525"  - 9 : "73568" Example response: [DataRequesting] Failed to parse body of response with status code Unknown (504):  <!DOCTYPE html> <html lang="en"> <head>   <style>     body {       font-family: "Helvetica Neue", "HelveticaNeue", Helvetica, Arial, sans-serif;       font-size: 15px;       font-weight: 200;       line-height: 20px;       color: #4c4c4c;       text-align: center;     }     .section {       margin-top: 50px;     }   </style> </head> <body> <div class="section">   <h1>&#63743;</h1>   <h3>Gateway Timeout</h3>   <p>Correlation Key: WFRI6Q5HXAUJYXGNRKQ6YTBYIM</p> </div> </body> </html> I have also tried batching these into 2 requests of 5 artists instead of 1 request of 10 artists which still fails. However, I do have sets of 10 artists that work fine. Anyone know why?
3
1
2.4k
Jan ’23
Saving plugin presets using fullState in Audio Units
Hi, I'm having trouble saving user presets in the plugin for Audio Units. This works well for saving the user presets in the Host, but I get an error when trying to save them in the plugin. I'm not using a parameter tree, but instead using the fullState's getter and setter for saving and retrieving a dictionary with the state. With some simplified parameters it looks something like this: var gain: Double = 0.0 var frequency: Double = 440.0     private var currentState: [String: Any] = [:] override var fullState: [String: Any]? {     get {       // Save the current state       currentState["gain"] = gain       currentState["frequency"] = frequency       // Return the preset state       return ["myPresetKey": currentState]     }     set {       // Extract the preset state       currentState = newValue?["myPresetKey"] as? [String: Any] ?? [:]       // Set the Audio Unit's properties       gain = currentState["gain"] as? Double ?? 0.0       frequency = currentState["frequency"] as? Double ?? 440.0     }  } This works perfectly well for storing user presets when saved in the host. When trying to save them in the plugin to be able to reuse them across hosts, I get the following error in the interface: "Missing key in preset state map". Note that I am testing mostly in AUM. I could not find any documentation for what the missing key is about and how can I get around this. Any ideas?
2
1
1.3k
Mar ’23
EXIF Makernote no read in Ventura
I have a custom app running on a Mac Studio with Ventura that grabs a snapshot image from a network camera. It then adds some extra information into the EXIF "MakerNote" field. However the metadata cannot be read back out of the image when running Ventrua, it can however be read out of the same image file on a Mac that is not running Ventura. It would appear Apple has removed support for reading MakerNote in Ventura but still supports writing MakerNote in Ventura. This code is about 7 years old and written in ObjC and has worked with no issue until Ventura came along. Calls used CGImageDestinationAddImageFromSource(); // used to write the image to disk with the extra metadata - Works on Ventura CGImageSourceCopyPropertiesAtIndex(); // used to read the meta data from an image - does not return "MakeNote" data Is there a new way to read EXIF "MakeNote" data from image files that was introduced with Ventura?
2
1
798
Mar ’23
AVSampleBufferDisplayLayer crash on flush
I am calling AVSampleBufferDisplayLayer.flush from a background queue but this seems to occasionally crash the app. I am calling it from the same thread I pass to - (void)requestMediaDataWhenReadyOnQueue:(dispatch_queue_t)queue usingBlock:(void (^)(void))block;. My question is, is this API threadsafe, or do I need to call flush from the main thread? Or is there another issue that I am not considering? It seems strange to me that this API would trigger an autolayout pass. 0 CoreFoundation 0x00000001bb384e38 __exceptionPreprocess + 164 1 libobjc.A.dylib 0x00000001b451b8d8 objc_exception_throw + 59 2 CoreAutoLayout 0x00000001d7e09e84 _AssertAutoLayoutOnAllowedThreadsOnly + 327 3 CoreAutoLayout 0x00000001d7e00e60 -[NSISEngine withBehaviors:performModifications:] + 35 4 UIKitCore 0x00000001be58fd40 -[UIView _postMovedFromSuperview:] + 671 5 UIKitCore 0x00000001bd56dfec -[UIView(Internal) _addSubview:positioned:relativeTo:] + 1903 6 UIKitCore 0x00000001bda57ccc -[_UITextLayoutCanvasView textViewportLayoutController:configureRenderingSurfaceForTextLayoutFragment:] + 455 7 UIFoundation 0x00000001c588bc9c __48-[NSTextViewportLayoutController layoutViewport]_block_invoke_4 + 151 8 UIFoundation 0x00000001c5836b50 __80-[NSTextLayoutManager enumerateViewportElementsFromLocation:options:usingBlock:]_block_invoke + 43 9 UIFoundation 0x00000001c580e158 __83-[NSTextLayoutManager enumerateTextLayoutFragmentsFromLocation:options:usingBlock:]_block_invoke_2 + 535 10 CoreFoundation 0x00000001bb385350 __NSARRAY_IS_CALLING_OUT_TO_A_BLOCK__ + 23 11 CoreFoundation 0x00000001bb3b24dc -[__NSSingleObjectArrayI enumerateObjectsWithOptions:usingBlock:] + 91 12 UIFoundation 0x00000001c580de28 __83-[NSTextLayoutManager enumerateTextLayoutFragmentsFromLocation:options:usingBlock:]_block_invoke + 775 13 UIFoundation 0x00000001c57f7504 -[NSTextLayoutManager enumerateTextLayoutFragmentsFromLocation:options:usingBlock:] + 659 14 UIFoundation 0x00000001c57f7264 -[NSTextLayoutManager enumerateViewportElementsFromLocation:options:usingBlock:] + 99 15 UIFoundation 0x00000001c57f6d7c -[NSTextViewportLayoutController layoutViewport] + 1299 16 UIKitCore 0x00000001bd580a3c +[UIView(Animation) performWithoutAnimation:] + 75 17 UIKitCore 0x00000001bd5582d0 -[_UITextLayoutCanvasView layoutSubviews] + 139 18 UIKitCore 0x00000001bd5544c8 -[UIView(CALayerDelegate) layoutSublayersOfLayer:] + 1979 19 QuartzCore 0x00000001bca277fc CA::Layer::layout_if_needed(CA::Transaction*) + 499 20 QuartzCore 0x00000001bca3aeb0 CA::Layer::layout_and_display_if_needed(CA::Transaction*) + 147 21 QuartzCore 0x00000001bca4c234 CA::Context::commit_transaction(CA::Transaction*, double, double*) + 443 22 QuartzCore 0x00000001bca81630 CA::Transaction::commit() + 651 23 MediaToolbox 0x00000001ca8d0da0 videoQueueRemote_SetProperty + 367 24 AVFCore 0x00000001cad191b4 __63-[AVSampleBufferVideoRenderer _setContentLayerOnFigVideoQueue:]_block_invoke + 179 25 libdispatch.dylib 0x00000001c299cf88 _dispatch_client_callout + 19 26 libdispatch.dylib 0x00000001c29ac574 _dispatch_lane_barrier_sync_invoke_and_complete + 55 27 AVFCore 0x00000001cad190d0 -[AVSampleBufferVideoRenderer _setContentLayerOnFigVideoQueue:] + 167 28 AVFCore 0x00000001cad14674 -[AVSampleBufferVideoRenderer _createVideoQueue:errorStep:] + 195 29 AVFCore 0x00000001cad14ac8 -[AVSampleBufferVideoRenderer createVideoQueue:] + 55 30 AVFCore 0x00000001cad179cc -[AVSampleBufferVideoRenderer flushWithRemovalOfDisplayedImage:completionHandler:] + 439 31 App 0x0000000102370214 -[AVSampleBufferDisplayLayer flush] + 51
1
0
865
Mar ’23
[PHCollectionList isPrivacySensitiveAlbum]: unrecognized selector crash
I cannot find anything documentation re: isPrivacySensitiveAlbum. I've granted my app access to all photos. Not sure what else to try Code that triggers the crash: let options = PHFetchOptions() options.fetchLimit = 1 let assetColl = PHAssetCollection.fetchAssetCollections(withLocalIdentifiers: [localId], options: options) if assetColl.count > 0 { if let asset = PHAsset.fetchKeyAssets(in: assetColl.firstObject!, options: options) stack trace from here on `2023-04-15 06:34:41.628537-0700 DPF[33615:6484880] -[PHCollectionList isPrivacySensitiveAlbum]: unrecognized selector sent to instance 0x7ff09232aec0 2023-04-15 06:34:41.632378-0700 DPF[33615:6484880] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[PHCollectionList isPrivacySensitiveAlbum]: unrecognized selector sent to instance 0x7ff09232aec0' *** First throw call stack: ( 0 CoreFoundation 0x00007ff80045478b __exceptionPreprocess + 242 1 libobjc.A.dylib 0x00007ff80004db73 objc_exception_throw + 48 2 CoreFoundation 0x00007ff8004638c4 +[NSObject(NSObject) instanceMethodSignatureForSelector:] + 0 3 CoreFoundation 0x00007ff800458c66 ___forwarding___ + 1443 4 CoreFoundation 0x00007ff80045ae08 _CF_forwarding_prep_0 + 120 5 Photos 0x00007ff80b8480e1 +[PHAsset fetchKeyAssetsInAssetCollection:options:] + 86 6 DPF 0x0000000100791029 $s3DPF16AlbumListFetcherV22loadKeyImageForLocalIdySo7UIImageCSgSSYaFTY0_ + 569`
2
0
712
Apr ’23
Audio / Video sync issue on iOS using AVSampleBufferRenderSynchronizer
My current app implements a custom video player, based on a AVSampleBufferRenderSynchronizer synchronising two renderers: an AVSampleBufferDisplayLayer receiving decoded CVPixelBuffer-based video CMSampleBuffers, and an AVSampleBufferAudioRenderer receiving decoded lpcm-based audio CMSampleBuffers. The AVSampleBufferRenderSynchronizer is started when the first image (in presentation order) is decoded and enqueued, using avSynchronizer.setRate(_ rate: Float, time: CMTime), with rate = 1 and time the presentation timestamp of the first decoded image. Presentation timestamps of video and audio sample buffers are consistent, and on most streams, the audio and video are correctly synchronized. However on some network streams, on iOS, the audio and video aren't synchronized, with a time difference that seems to increase with time. On the other hand, with the same player code and network streams on macOS, the synchronization always works fine. This reminds me of something I've read, about cases where an AVSampleBufferRenderSynchronizer could not synchronize audio and video, causing them to run with independent and potentially drifting clocks, but I cannot find it again. So, any help / hints on this sync problem will be greatly appreciated! :)
1
0
1k
May ’23
ApplicationMusicPlayer macOS vs. Catalyst
ApplicationMusicPlayer is available on the Mac! 🎉🎉🎉 Enormous thanks to @JoeKun and the team. I've already gotten my app up and running through Catalyst, and I've successfully played music! I also got some timeouts, but that was happening on my phone a lot that day too, so maybe my local CDN was just having a bad day. I wanted to ask this question in a lab this week, but the timing didn't work out: Do you expect the experience to be the same using ApplicationMusicPlayer on a Catalyst vs a macOS target? I'm hoping to reuse much of my iPad app and go the Catalyst route, but I wanted to double check that the new support wasn't just for macOS.
2
1
803
Jun ’23
AVSpeechSynthesizer Broken on iOS 17
I just upgraded to iOS 17 and it looks like AVSpeechSynthesizer is now broken. I noticed when feeding certain strings to AVSpeechUtterance it just dies flat out stops after only speaking a portion of the string. For example I fed it a string of approx. 1200 words and it speaks up to around 300 words or so and then just stops. The synthesizer delegate method -speechSynthesizer:didFinishSpeechUtterance: is called when this happens, as if this is supposed to be the end even though it is not even close to being finishes. Was working fine on iOS 16. FWIW I create the AVSpeechUtterance with -initWithString:
15
8
5.1k
Sep ’23
AV1 Hardware Decoding
Recently I've been trying to play some AV1-encoded streams on my iPhone 15 Pro Max. First, I check for hardware support: VTIsHardwareDecodeSupported(kCMVideoCodecType_AV1); // YES Then I need to create a CMFormatDescription in order to pass it into a VTDecompressionSession. I've tried the following: { mediaType:'vide' mediaSubType:'av01' mediaSpecific: { codecType: 'av01' dimensions: 394 x 852 } extensions: {{ CVFieldCount = 1; CVImageBufferChromaLocationBottomField = Left; CVImageBufferChromaLocationTopField = Left; CVPixelAspectRatio = { HorizontalSpacing = 1; VerticalSpacing = 1; }; FullRangeVideo = 0; }} } but VTDecompressionSessionCreate gives me error -8971 (codecExtensionNotFoundErr, I assume). So it has something to do with the extensions dictionary? I can't find anywhere which set of extensions is necessary for it to work 😿. VideoToolbox has convenient functions for creating descriptions of AVC and HEVC streams (CMVideoFormatDescriptionCreateFromH264ParameterSets and CMVideoFormatDescriptionCreateFromHEVCParameterSets), but not for AV1. As of today I am using XCode 15.0 with iOS 17.0.0 SDK.
8
3
7.5k
Oct ’23
MusicKit / macOS : Song.Artwork not nil when there is no Artwork
I'm using iCloud Music Library. I’m using macOS 14.1 (23B74) and iOS 17.1. i’m using MusicKit to find songs that do not have artwork. On iOS, Song.artwork will be nil for items I know do not have artwork. On macOS, Song.artwork is not nil. However when the songs are shown in Music.app, they do not have Artwork. Is this expected? Alternately, is there a more correct way to determine that a Song has no Artwork? I have also filed FB13315721. Thank you for any tips!
1
0
682
Oct ’23
Detect when (internal or external) microphone is being used
Hello, I used kAudioDevicePropertyDeviceIsRunningSomewhere to check if an internal or external microphone is being used. My code works well for the internal microphone, and for microphones which are connected using a cable. External microphones which are connected using bluetooth are not reporting their status. The status is always requested successfully, but it is always reported as inactive. Main relevant parts in my code : static inline AudioObjectPropertyAddress makeGlobalPropertyAddress(AudioObjectPropertySelector selector) { AudioObjectPropertyAddress address = { selector, kAudioObjectPropertyScopeGlobal, kAudioObjectPropertyElementMaster, }; return address; } static BOOL getBoolProperty(AudioDeviceID deviceID, AudioObjectPropertySelector selector) { AudioObjectPropertyAddress const address = makeGlobalPropertyAddress(selector); UInt32 prop; UInt32 propSize = sizeof(prop); OSStatus const status = AudioObjectGetPropertyData(deviceID, &address, 0, NULL, &propSize, &prop); if (status != noErr) { return 0; //this line never gets executed in my tests. The call above always succeeds, but it always gives back "false" status. } return static_cast<BOOL>(prop == 1); } ... __block BOOL microphoneActive = NO; iterateThroughAllInputDevices(^(AudioObjectID object, BOOL *stop) { if (getBoolProperty(object, kAudioDevicePropertyDeviceIsRunningSomewhere) != 0) { microphoneActive = YES; *stop = YES; } }); What could cause this and how could it be fixed? Thank you for your help in advance!
3
0
1.1k
Nov ’23
AVPlayer with multiple audio tracks plays audio differently when start
Hi, I'm trying to play multiple video/audio file with AVPlayer using AVMutableComposition. Each video/audio file can process simultaneously so I set each video/audio in individual tracks. I use only local file. let second = CMTime(seconds: 1, preferredTimescale: 1000) let duration = CMTimeRange(start: .zero, duration: second) var currentTime = CMTime.zero for _ in 0...4 { let mutableTrack = composition.addMutableTrack( withMediaType: .audio, preferredTrackID: kCMPersistentTrackID_Invalid ) try mutableTrack?.insertTimeRange( duration, of: audioAssetTrack, at: currentTime ) currentTime = currentTime + second } When I set many audio tracks (maybe more than 5), the first part sounds a little different from original when it starts. It seems like audio's front part is skipped. But when I set only two tracks, AVPlayer plays as same as original file. avPlayer.play() How can I fix it? Why do audio tracks affect that don't have any playing parts when start? Please let me know.
1
2
1.3k
Dec ’23
Synchronized depth and video data not being received with builtInLiDARDepthCamera
Hello, Faced with a really perplexing issue. Primary problem is that sometimes I get depth and video data as expected, but at other times I don't. And sometimes I'll get both data outputs for a 4-5 frames and then it'll just stop. The source code I implemented is a modified version of the sample code provided by Apple, and interestingly enough I can't re-create this issue with the Apple sample app. So wondering what I could be doing wrong? Here's the code for setting up the capture input. preferredDepthResolution is 1280 in my case. I'm running this on an iPad Pro (6th gen). iOS version 17.0.3 (21A360). Encounter this issue on iPhone 13 Pro as well. iOS version is 17.0 (21A329) private func setupLiDARCaptureInput() throws { // Look up the LiDAR camera. guard let device = AVCaptureDevice.default(.builtInLiDARDepthCamera, for: .video, position: .back) else { throw ConfigurationError.lidarDeviceUnavailable } guard let format = (device.formats.last { format in format.formatDescription.dimensions.width == preferredWidthResolution && format.formatDescription.mediaSubType.rawValue == kCVPixelFormatType_420YpCbCr8BiPlanarFullRange && format.videoSupportedFrameRateRanges.first(where: {$0.maxFrameRate >= 60}) != nil && !format.isVideoBinned && !format.supportedDepthDataFormats.isEmpty }) else { throw ConfigurationError.requiredFormatUnavailable } guard let depthFormat = (format.supportedDepthDataFormats.last { depthFormat in depthFormat.formatDescription.mediaSubType.rawValue == kCVPixelFormatType_DepthFloat16 }) else { throw ConfigurationError.requiredFormatUnavailable } // Begin the device configuration. try device.lockForConfiguration() // Configure the device and depth formats. device.activeFormat = format device.activeDepthDataFormat = depthFormat let desc = format.formatDescription dimensions = CMVideoFormatDescriptionGetDimensions(desc) let duration = CMTime(value:1, timescale:CMTimeScale(60)) device.activeVideoMinFrameDuration = duration device.activeVideoMaxFrameDuration = duration // Finish the device configuration. device.unlockForConfiguration() self.device = device print("Selected video format: \(device.activeFormat)") print("Selected depth format: \(String(describing: device.activeDepthDataFormat))") // Add a device input to the capture session. let deviceInput = try AVCaptureDeviceInput(device: device) captureSession.addInput(deviceInput) guard let audioDevice = AVCaptureDevice.default(for: .audio) else { return } // Configure audio input - always configure audio even if isAudioEnabled is false audioDeviceInput = try! AVCaptureDeviceInput(device: audioDevice) captureSession.addInput(audioDeviceInput) deviceSystemPressureStateObservation = device.observe( \.systemPressureState, options: .new ) { _, change in guard let systemPressureState = change.newValue else { return } print("system pressure \(systemPressureState.levelAsString()) due to \(systemPressureState.factors)") } } Here's how I'm setting up the output: private func setupLiDARCaptureOutputs() { // Create an object to output video sample buffers. videoDataOutput = AVCaptureVideoDataOutput() captureSession.addOutput(videoDataOutput) // Create an object to output depth data. depthDataOutput = AVCaptureDepthDataOutput() depthDataOutput.isFilteringEnabled = false captureSession.addOutput(depthDataOutput) audioDeviceOutput = AVCaptureAudioDataOutput() audioDeviceOutput.setSampleBufferDelegate(self, queue: videoQueue) captureSession.addOutput(audioDeviceOutput) // Create an object to synchronize the delivery of depth and video data. outputVideoSync = AVCaptureDataOutputSynchronizer(dataOutputs: [depthDataOutput, videoDataOutput]) outputVideoSync.setDelegate(self, queue: videoQueue) // Enable camera intrinsics matrix delivery. guard let outputConnection = videoDataOutput.connection(with: .video) else { return } if outputConnection.isCameraIntrinsicMatrixDeliverySupported { outputConnection.isCameraIntrinsicMatrixDeliveryEnabled = true } } The top part of my delegate implementation is as follows: func dataOutputSynchronizer( _ synchronizer: AVCaptureDataOutputSynchronizer, didOutput synchronizedDataCollection: AVCaptureSynchronizedDataCollection ) { // Retrieve the synchronized depth and sample buffer container objects. guard let syncedDepthData = synchronizedDataCollection.synchronizedData(for: depthDataOutput) as? AVCaptureSynchronizedDepthData, let syncedVideoData = synchronizedDataCollection.synchronizedData(for: videoDataOutput) as? AVCaptureSynchronizedSampleBufferData else { if synchronizedDataCollection.synchronizedData(for: depthDataOutput) == nil { print("no depth data at time \(mach_absolute_time())") } if synchronizedDataCollection.synchronizedData(for: videoDataOutput) == nil { print("no video data at time \(mach_absolute_time())") } return } print("received depth data \(mach_absolute_time())") } As you can see, I'm console logging whenever depth data is not received. Note because I'm driving the video frames at 60 fps, its expected that I'll only receive depth data for every alternate video frame. Console output is posted as a follow up comment (because of the character limit). I edited some lines out for brevity. You'll see it started streaming correctly but after a while it stopped received both video and depth outputs (in some other runs, it works perfectly and in some other runs I receive no depth data whatsoever). One thing to note, I sometimes run quicktime mirroring to see the device screen to see what the app is displaying (so not sure if that's causing any interference - that said I don't see any system pressure changes either). Any help is most appreciated! Thanks.
3
0
1.2k
Dec ’23
Safari Fairplay WebKitMediaKeyError (code: 6, systemCode: 4294955417)
Hi Apple Team, we are observing following error intermittently when trying to playback FairPlay protected HLS streams. The error happens immediately after loading the certificate. Playback with same certificate on same device(Mac, iPhone) works most of time but intermittently this error is observed with following codes. The code=6 means MEDIA_KEYERR_DOMAIN but I did not find any information on what does systemCode=4294955417 mean? Is there a way to check what does this system code mean and what could be causing this intermittent behaviour? { "code": 6, "systemCode": 4294955417 }
1
0
930
Dec ’23