I'm trying to do some realtime audio processing on audio served from an HLS stream (i.e. an AVPlayer created using an M3U HTTP URL). It doesn't seem like attaching an AVAudioMix configured with with an `audioTapProcessor` has any effect; none of the callbacks except `init` are being invoked. Is this a known limitation? If so, is this documented somewhere?If the above is a limitation, what are my options using some of the other audio APIs? I looked into `AVAudioEngine` as well but it doesn't seem like there's any way I can configure any of the input node types to use an HLS stream. Am I wrong? Are there lower level APIs available to play HLS streams that provide the necessary hooks?Alternatively, is there some generic way to tap into all audio being output by my app regardless of its source?Thanks a lot!

I am currently working with AVAssetWriterInput with interlaced source.I want to know the proper procedure to preserve 'fiel' format description extension through AVAssetWriterInput compression operation.Do you anyone have an answer on this?>>Technical Note TN2162>>Uncompressed Y´CbCr Video in QuickTime Files>>The 'fiel' ImageDescription Extension: Field/Frame Information//What I have tried is following:First decode into sample buffer with Field/Frame information- Decode DV-NTSC source and get current FieldCount/Detail of source data- Create a kCVPixelFormatType_422YpCbCr8 CVImageBuffer from source- CMSetAttachments() kCVImageBufferFieldCountKey/kCVImageBufferFieldDetailKey with FieldCount=2/FieldDetail=14 to imageBuffer, same as DV-NTSC source- CMVideoFormatDescriptionCreateForImageBuffer()- CMSampleBufferCreateForImageBuffer()- Now decompressed samplebuffer has kCMFormatDescriptionExtension_FieldXXX same as sourceSecond encode using ProRes422- Next I create AVAssetWriterInput to transcode from _422YpCbCr8 into AVVideoCodecAppleProRes422- Now AVAssetWriter can produce ProRes422 mov file, but it does not contain the format description extension 'fiel'.- ProRes422 Encoder does preserve 'colr', 'pasp', 'clap' same as source, but no 'fiel' is there.I have also tried to serve format description with kCMFormatDescriptionExtension_FieldXXX with initWithMediaType:outputSettings:sourceFormatHint:but no success.//

Hi,I'm having two problems using the scheduleBuffer function of AVAudioPlayerNode.Background: my app generates audio programatically, which is why I am using this function. I also need low latency. Therefore, I'm using a strategy of scheduling a small number of buffers, and using the completion handler to keep the process moving forward by scheduling one more buffer for each one that completes.I'm seeing two problems with this approach:One, the total memory consumed by my app grows steadily while the audio is playing, which suggests that the audio buffers are never being deallocated or some other runaway process is underway. (The Leaks tool doesn't detect any leaks, however).Two, audio playback sometimes stops, particularly on slower devices. By "stops", what I mean is that at some point I schedule a buffer and the completion block for that buffer is never called. When this happens, I can't even clear the problem by stopping the player.Now, regarding the first issue, I suspected that if my completion block recursively scheduled another buffer with another completion block, I would probably end up blowing out the stack with an infinite recursion. To get around this, instead of directly scheduling the buffer in the completion block, I set it up to enqueue the schedule in a dispatch queue. However, this doesn't seem to solve the problem.Any advice would be appreciated. Thanks.

I'm using AVAssetReaders with AVSampleBufferDisplayLayers to display multiple videos at once. I'm seeing this issue on iOS 13.1.3, 13.2b2, on various hardware like iPad 10.5 and iPad 12.9.It works well for a while, then a random call to copyNextSampleBuffer never returns, blocking that thread indefinitely and eating up resources.I have tried different threading approaches with no avail:If copyNextSampleBuffer() and reader.cancelReading() are done on the same queue, then copyNextSampleBuffer() gets stuck and the cancelReading() never gets processed because the queue is blocked. If I manually (with the debugger) jump in on that blocked queue and execute cancelReading(), immediately an EXC_BREAKPOINT crashes the appIf copyNextSampleBuffer() and reader.cancelReading() are done on different queues, then copyNextSampleBuffer() crashes with EXC_BAD_ACCESSHere's the stacktrace (same queue approach). I don't understand why it's stuck, my expectation is that copyNextSampleBuffer should always return (ie. with nil in error case).VideoPlayerView: UIView with AVSampleBufferDisplayLayerAVAssetFactory: Singleton with the queue that creates & manages all AVAssetReader / AVAsset* objects* thread #22, queue = 'AVAssetFactory' frame #0: 0x00000001852355f4 libsystem_kernel.dylib`mach_msg_trap + 8 frame #1: 0x0000000185234a60 libsystem_kernel.dylib`mach_msg + 72 frame #2: 0x00000001853dc068 CoreFoundation`__CFRunLoopServiceMachPort + 216 frame #3: 0x00000001853d7188 CoreFoundation`__CFRunLoopRun + 1444 frame #4: 0x00000001853d68bc CoreFoundation`CFRunLoopRunSpecific + 464 frame #5: 0x000000018f42b6ac AVFoundation`-[AVRunLoopCondition _waitInMode:untilDate:] + 400 frame #6: 0x000000018f38f1dc AVFoundation`-[AVAssetReaderOutput copyNextSampleBuffer] + 148 frame #7: 0x000000018f3900f0 AVFoundation`-[AVAssetReaderTrackOutput copyNextSampleBuffer] + 72 * frame #8: 0x0000000103309d98 Photobooth`closure #1 in AVAssetFactory.nextSampleBuffer(reader=0x00000002814016f0, retval=(Swift.Optional<CoreMedia.CMSampleBuffer>, Swift.Optional<AVFoundation.AVAssetReader.Status>) @ 0x000000016dbd1cb8) at AVAssetFactory.swift:108:34 frame #9: 0x0000000102f4f480 Photobooth`thunk for @callee_guaranteed () -> () at <compiler-generated>:0 frame #10: 0x0000000102f4f4a4 Photobooth`thunk for @escaping @callee_guaranteed () -> () at <compiler-generated>:0 frame #11: 0x000000010bfe6c04 libdispatch.dylib`_dispatch_client_callout + 16 frame #12: 0x000000010bff5888 libdispatch.dylib`_dispatch_lane_barrier_sync_invoke_and_complete + 124 frame #13: 0x0000000103309a5c Photobooth`AVAssetFactory.nextSampleBuffer(reader=0x00000002814016f0, self=0x0000000281984f60) at AVAssetFactory.swift:101:20 frame #14: 0x00000001032ab690 Photobooth`closure #1 in VideoPlayerView.setRequestMediaLoop(self=0x000000014b8da1d0, handledCompletion=false) at VideoPlayerView.swift:254:70 frame #15: 0x0000000102dce978 Photobooth`thunk for @escaping @callee_guaranteed () -> () at <compiler-generated>:0 frame #16: 0x000000018f416848 AVFoundation`-[AVMediaDataRequester _requestMediaDataIfReady] + 80 frame #17: 0x000000010bfe5828 libdispatch.dylib`_dispatch_call_block_and_release + 24 frame #18: 0x000000010bfe6c04 libdispatch.dylib`_dispatch_client_callout + 16 frame #19: 0x000000010bfedb74 libdispatch.dylib`_dispatch_lane_serial_drain + 744 frame #20: 0x000000010bfee744 libdispatch.dylib`_dispatch_lane_invoke + 500 frame #21: 0x000000010bff9ae4 libdispatch.dylib`_dispatch_workloop_worker_thread + 1324 frame #22: 0x000000018517bfa4 libsystem_pthread.dylib`_pthread_wqthread + 276I've tried all kinds of other things like making sure the AVAssets and all objects are made on one queue, and stopping the AVAssetReaders from ever deallocing to see if that helps. Nothing works. Any ideas?

I'm trying to make an app that plays audio when push is received, even while the ringer is silent.I have the audio background mode enabled and the app works (usually) fine, but sometimes i get a mysterious 561015905 error.The error isAVAudioSession.ErrorCode.cannotStartPlayingAnd docs indicate:This error type can occur if the app’s Information property list doesn’t permit audio use. It can also occur if the app is in the background and using a category that doesn’t allow background audio.https://developer.apple.com/documentation/avfoundation/avaudiosession/errorcode/cannotstartplayingHowever, the audio mode is definitely enabled and the category is always playback that should allow background playing.(this works 95-99% of times)So, with intesnse testing i have found that once every 20-100 incoming pushes i get this error:Error: Error Domain=NSOSStatusErrorDomain Code=561015905 "(null)"Why?

Hi, I'm working on an app where a user can scroll through a feed of short videos (a bit like TikTok). I do some pre-fetching of videos a couple positions ahead of the user's scroll position, so the video can start playing as soon as he or she scrolls to the video. Currently, I just pre-fetch by initializing a few AVPlayers. However, I'd like to add a better caching system. I'm looking for the best way to: get videos to start playing as possible, while making sure to minimize re-downloading of videos if a user scrolls away from a video and back to it. Is there a way that I can cache the contents of an AVPlayer that has loaded an HLS video? Alternatively, I've explored using AVAssetDownloadTask to download the HLS videos. My issue is that I can't download the full video and then play it - I need to start playing the video as soon as the user scrolls to it, even if it's not done downloading. Is there a way to start an HLS download with an AVAssetDownloadTask, and then start playing the video while it continues downloading? Thank you!

We are creating a watch party app that allows you to video chat with your friends and play a YouTube video at the same time. The video is played using Google's youtube-ios-player-helperlibrary which uses a WKWebView with their iframe API, as that's the only way to play it without violating the Terms of Service. We need the ability to change the volume of the YouTube video separately from the video chat, so you can hear your friends over the video for example. Unfortunately it's not possible to directly change the volume because iOS does not support changing the volume via JavaScript - https://developer.apple.com/library/archive/documentation/AudioVideo/Conceptual/Using_HTML5_Audio_Video/Device-SpecificConsiderations/Device-SpecificConsiderations.html#//apple_ref/doc/uid/TP40009523-CH5-SW10, unlike macOS. Setting volume doesn't do anything and getting it always returns 1. Users can change the volume with the hardware buttons but this applies to all audio including the video chat, not just the YouTube video. Someone found a workaround - https://stackoverflow.com/a/37315071/1795356 to get the underlying AVPlayer and change its volume natively. This worked with UIWebView but does not work now that it uses WKWebView. What can be done to change the volume of the YouTube video?

Summary: I am using the Vision framework, in conjunction with AVFoundation, to detect facial landmarks of each face in the camera feed (by way of the VNDetectFaceLandmarksRequest). From here, I am taking the found observations and unprojecting each point to a SceneKit View (SCNView), then using those points as the vertices to draw a custom geometry that is textured with a material over each found face. Effectively, I am working to recreate how an ARFaceTrackingConfiguration functions. In general, this task is functioning as expected, but only when my device is using the front camera in landscape right orientation. When I rotate my device, or switch to the rear camera, the unprojected points do not properly align with the found face as they do in landscape right/front camera. Problem: When testing this code, the mesh appears properly (that is, appears affixed to a user's face), but again, only when using the front camera in landscape right. While the code runs as expected (that is, generating the face mesh for each found face) in all orientations, the mesh is wildly misaligned in all other cases. My belief is this issue either stems from my converting the face's bounding box (using VNImageRectForNormalizedRect, which I am calculating using the width/height of my SCNView, not my pixel buffer, which is typically much larger), though all modifications I have tried result in the same issue. Outside of that, I also believe this could be an issue with my SCNCamera, as I am a bit unsure how the transform/projection matrix works and whether that would be needed here. Sample of Vision Request Setup: // Setup Vision request options var requestHandlerOptions: [VNImageOption: AnyObject] = [:] // Setup Camera Intrinsics let cameraIntrinsicData = CMGetAttachment(sampleBuffer, key: kCMSampleBufferAttachmentKey_CameraIntrinsicMatrix, attachmentModeOut: nil) if cameraIntrinsicData != nil { requestHandlerOptions[VNImageOption.cameraIntrinsics] = cameraIntrinsicData } // Set EXIF orientation let exifOrientation = self.exifOrientationForCurrentDeviceOrientation() // Setup vision request handler let handler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer, orientation: exifOrientation, options: requestHandlerOptions) // Setup the completion handler let completion: VNRequestCompletionHandler = {request, error in let observations = request.results as! [VNFaceObservation] // Draw faces DispatchQueue.main.async { drawFaceGeometry(observations: observations) } } // Setup the image request let request = VNDetectFaceLandmarksRequest(completionHandler: completion) // Handle the request do { try handler.perform([request]) } catch { print(error) } Sample of SCNView Setup: // Setup SCNView let scnView = SCNView() scnView.translatesAutoresizingMaskIntoConstraints = false self.view.addSubview(scnView) scnView.showsStatistics = true NSLayoutConstraint.activate([ scnView.leadingAnchor.constraint(equalTo: self.view.leadingAnchor), scnView.topAnchor.constraint(equalTo: self.view.topAnchor), scnView.bottomAnchor.constraint(equalTo: self.view.bottomAnchor), scnView.trailingAnchor.constraint(equalTo: self.view.trailingAnchor) ]) // Setup scene let scene = SCNScene() scnView.scene = scene // Setup camera let cameraNode = SCNNode() let camera = SCNCamera() cameraNode.camera = camera scnView.scene?.rootNode.addChildNode(cameraNode) cameraNode.position = SCNVector3(x: 0, y: 0, z: 16) // Setup light let ambientLightNode = SCNNode() ambientLightNode.light = SCNLight() ambientLightNode.light?.type = SCNLight.LightType.ambient ambientLightNode.light?.color = UIColor.darkGray scnView.scene?.rootNode.addChildNode(ambientLightNode) Sample of "face processing" func drawFaceGeometry(observations: [VNFaceObservation]) { // An array of face nodes, one SCNNode for each detected face var faceNode = [SCNNode]() // The origin point let projectedOrigin = sceneView.projectPoint(SCNVector3Zero) // Iterate through each found face for observation in observations { // Setup a SCNNode for the face let face = SCNNode() // Setup the found bounds let faceBounds = VNImageRectForNormalizedRect(observation.boundingBox, Int(self.scnView.bounds.width), Int(self.scnView.bounds.height)) // Verify we have landmarks if let landmarks = observation.landmarks { // Landmarks are relative to and normalized within face bounds let affineTransform = CGAffineTransform(translationX: faceBounds.origin.x, y: faceBounds.origin.y) .scaledBy(x: faceBounds.size.width, y: faceBounds.size.height) // Add all points as vertices var vertices = [SCNVector3]() // Verify we have points if let allPoints = landmarks.allPoints { // Iterate through each point for (index, point) in allPoints.normalizedPoints.enumerated() { // Apply the transform to convert each point to the face's bounding box range _ = index let normalizedPoint = point.applying(affineTransform) let projected = SCNVector3(normalizedPoint.x, normalizedPoint.y, CGFloat(projectedOrigin.z)) let unprojected = sceneView.unprojectPoint(projected) vertices.append(unprojected) } } // Setup Indices var indices = [UInt16]() // Add indices // ... Removed for brevity ... // Setup texture coordinates var coordinates = [CGPoint]() // Add texture coordinates // ... Removed for brevity ... // Setup texture image let imageWidth = 2048.0 let normalizedCoordinates = coordinates.map { coord -> CGPoint in let x = coord.x / CGFloat(imageWidth) let y = coord.y / CGFloat(imageWidth) let textureCoord = CGPoint(x: x, y: y) return textureCoord } // Setup sources let sources = SCNGeometrySource(vertices: vertices) let textureCoordinates = SCNGeometrySource(textureCoordinates: normalizedCoordinates) // Setup elements let elements = SCNGeometryElement(indices: indices, primitiveType: .triangles) // Setup Geometry let geometry = SCNGeometry(sources: [sources, textureCoordinates], elements: [elements]) geometry.firstMaterial?.diffuse.contents = textureImage // Setup node let customFace = SCNNode(geometry: geometry) sceneView.scene?.rootNode.addChildNode(customFace) // Append the face to the face nodes array faceNode.append(face) } // Iterate the face nodes and append to the scene for node in faceNode { sceneView.scene?.rootNode.addChildNode(node) } }

Hello, I am using HLS audio stream and I get metadata to show the song info, artist info etc., in the app using AVPlayer. In iOS 14, I see the artwork data (which embeds in WXXX frame of id3 tag) is coming over across along with other metadata. The same stream is working fine and showing artwork data in iOS 13. Does anything changed in iOS 14 which effects this metadata? Please let me know Thank you

I am currently working on a macOS app which will be creating very large video files with up to an hour of content. However, generating the images and adding them to AVAssetWriter leads to VTDecoderXPCService using 16+ GB of memory and the kernel-task using 40+ GB (the max I saw was 105GB). It seems like the generated video is not streamed onto the disk but rather written to memory for it to be written to disk all at once. How can I force it to stream the data to disk while the encoding is happening? Btw. my app itself consistently needs around 300MB of memory, so I don't think I have a memory leak here. Here is the relevant code: func analyse() { 				self.videoWritter = try! AVAssetWriter(outputURL: outputVideo, fileType: AVFileType.mp4) 				let writeSettings: [String: Any] = [ 						AVVideoCodecKey: AVVideoCodecType.h264, 						AVVideoWidthKey: videoSize.width, 						AVVideoHeightKey: videoSize.height, 						AVVideoCompressionPropertiesKey: [ 								AVVideoAverageBitRateKey: 10_000_000, 						] 				] 				self.videoWritter!.movieFragmentInterval = CMTimeMake(value: 60, timescale: 1) 				self.frameInput = AVAssetWriterInput(mediaType: AVMediaType.video, outputSettings: writeSettings) 				self.frameInput?.expectsMediaDataInRealTime = true 				self.videoWritter!.add(self.frameInput!) 				if self.videoWritter!.startWriting() == false { 						print("Could not write file: \(self.videoWritter!.error!)") 						return 				} } func writeFrame(frame: Frame) { 				/* some more code to determine the correct frame presentation time stamp */ 				let newSampleBuffer = self.setTimeStamp(frame.sampleBuffer, newTime: self.nextFrameStartTime!) 				self.frameInput!.append(newSampleBuffer) 				/* some more code */ } func setTimeStamp(_ sample: CMSampleBuffer, newTime: CMTime) -> CMSampleBuffer { 				var timing: CMSampleTimingInfo = CMSampleTimingInfo( 						duration: CMTime.zero, 						presentationTimeStamp: newTime, 						decodeTimeStamp: CMTime.zero 				) 				var newSampleBuffer: CMSampleBuffer? 				CMSampleBufferCreateCopyWithNewTiming( 						allocator: kCFAllocatorDefault, 					 sampleBuffer: sample, 					 sampleTimingEntryCount: 1, 					 sampleTimingArray: &timing, 					 sampleBufferOut: &newSampleBuffer 			 ) 				return	newSampleBuffer! 		} My specs: MacBook Pro 2018 32GB Memory macOS Big Sur 11.1

Hello there, in our team we were requested to add the possibility to manually select the video quality. I know that HLS is an adaptive stream and that depending on the network condition it choose the best quality that fits to the current situation. I also tried some setting with preferredMaximumResolution and preferredPeakBitRate but none of them worked once the user was watching the steam. I also tried something like replacing the currentPlayerItem with the new configuration but anyway this only allowed me to downgrade the quality of the video. When I wanted to set it for example to 4k it did not change to that track event if I set a very high values to both params mentioned above. My question is if there is any method which would allow me to force certain quality from the manifest file. I already have some kind of extraction which can parse the manifest file and provide me all the available information but I couldn't still figure out how to make the player reproduce specific stream with my desired quality from the available playlist.

Due to legal restrictions I need to prevent my app's users from skipping and fast-forwarding the content that is played by AVPlayerViewController. I use playerViewController(:willResumePlaybackAfterUserNavigatedFrom:to:) and playerViewController(:timeToSeekAfterUserNavigatedFrom:to:) delegate methods to control the skipping behaviour. However, those delegate methods are only triggered for skip +/- 10, but not for fast-forwarding/rewinding.  Is there a way to prevent fast-forwarding in addition to skipping in AVPlayerViewController? Here is an example of the code I use: class ViewController: UIViewController {   override func viewDidAppear(_ animated: Bool) {     super.viewDidAppear(animated)     setUpPlayerViewController()   }   private func setUpPlayerViewController() {     let playerViewController = AVPlayerViewController()     playerViewController.delegate = self guard let url = URL(string: "https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_ts/master.m3u8") else {       debugPrint("URL is not found")       return     }     let playerItem = AVPlayerItem(url: url)     let player = AVPlayer(playerItem: playerItem)     playerViewController.player = player     present(playerViewController, animated: true) {       playerViewController.player?.play()     }   } } extension ViewController: AVPlayerViewControllerDelegate {   public func playerViewController(_ playerViewController: AVPlayerViewController, willResumePlaybackAfterUserNavigatedFrom oldTime: CMTime, to targetTime: CMTime) { // Triggered on skip +/- 10, but not on fast-forwarding/rewinding     print("playerViewController(_:willResumePlaybackAfterUserNavigatedFrom:to:)")   }   public func playerViewController(_ playerViewController: AVPlayerViewController, timeToSeekAfterUserNavigatedFrom oldTime: CMTime, to targetTime: CMTime) -> CMTime {     // Triggered on skip +/- 10, but not on fast-forwarding/rewinding     print("playerViewController(_:timeToSeekAfterUserNavigatedFrom:to:)")     return targetTime   } }

I ran into a strange problem. A camera app using AVFoundation, I use the following code; captureDevice = AVCaptureDevice.default(AVCaptureDevice.DeviceType.builtInUltraWideCamera, for: AVMediaType.video, position: .back) then, let isAutoFocusSupported = captureDevice.isFocusModeSupported(.autoFocus) "isAutoFocusSupported" should be "true". For iPhone 13 pro, it is "true". But for 13 / 13 mini, it is "false". Why?

I have code that has worked for many years for writing ProRes files, and it is now failing on the new M1 Max MacBook. Specifically, if I construct buffers with the pixel type "kCVPixelFormatType_64ARGB", after a few frames of writing, the pixel buffer pool becomes nil. This code works just fine on non Max processors (Intel and base M1 natively). Here's a sample main that demonstrates the problem. Am I doing something wrong here? //  main.m //  TestProresWriting // #import <Foundation/Foundation.h> #import <AVFoundation/AVFoundation.h> int main(int argc, const char * argv[]) {     @autoreleasepool {         int timescale = 24;         int width = 1920;         int height = 1080;         NSURL *url = [NSURL URLWithString:@"file:///Users/diftil/TempData/testfile.mov"];         NSLog(@"Output file = %@", [url absoluteURL]);         NSFileManager *fileManager = [NSFileManager defaultManager];         NSError *error = nil;         [fileManager removeItemAtURL:url error:&error];         // Set up the writer         AVAssetWriter *trackWriter = [[AVAssetWriter alloc] initWithURL:url                                                    fileType:AVFileTypeQuickTimeMovie                                                         error:&error];         // Set up the track         NSDictionary *videoSettings = [NSDictionary dictionaryWithObjectsAndKeys:                                        AVVideoCodecTypeAppleProRes4444, AVVideoCodecKey,                                        [NSNumber numberWithInt:width], AVVideoWidthKey,                                        [NSNumber numberWithInt:height], AVVideoHeightKey,                                        nil];                  AVAssetWriterInput *track = [AVAssetWriterInput assetWriterInputWithMediaType:AVMediaTypeVideo                                                         outputSettings:videoSettings];         // Set up the adapter         NSDictionary *attributes = [NSDictionary                                     dictionaryWithObjects:                                     [NSArray arrayWithObjects:[NSNumber numberWithUnsignedInt:kCVPixelFormatType_64ARGB], // This pixel type causes problems on M1 Max, but works on everything else                                      [NSNumber numberWithUnsignedInt:width],[NSNumber numberWithUnsignedInt:height],                                      nil]                                     forKeys:                                     [NSArray arrayWithObjects:(NSString *)kCVPixelBufferPixelFormatTypeKey,                                      (NSString*)kCVPixelBufferWidthKey, (NSString*)kCVPixelBufferHeightKey,                                      nil]];         /*         NSDictionary *attributes = [NSDictionary                                     dictionaryWithObjects:                                     [NSArray arrayWithObjects:[NSNumber numberWithUnsignedInt:kCVPixelFormatType_32ARGB], // This pixel type works on M1 Max                                      [NSNumber numberWithUnsignedInt:width],[NSNumber numberWithUnsignedInt:height],                                      nil]                                     forKeys:                                     [NSArray arrayWithObjects:(NSString *)kCVPixelBufferPixelFormatTypeKey,                                      (NSString*)kCVPixelBufferWidthKey, (NSString*)kCVPixelBufferHeightKey,                                      nil]];         */         AVAssetWriterInputPixelBufferAdaptor *pixelBufferAdaptor = [AVAssetWriterInputPixelBufferAdaptor                             assetWriterInputPixelBufferAdaptorWithAssetWriterInput:track                             sourcePixelBufferAttributes:attributes];         // Add the track and start writing         [trackWriter addInput:track];         [trackWriter startWriting];         CMTime startTime = CMTimeMake(0, timescale);         [trackWriter startSessionAtSourceTime:startTime];         while(!track.readyForMoreMediaData);         int frameTime = 0;         CVPixelBufferRef frameBuffer = NULL;         for (int i = 0; i < 100; i++)         {             NSLog(@"Frame %@", [NSString stringWithFormat:@"%d", i]);             CVPixelBufferPoolRef PixelBufferPool = pixelBufferAdaptor.pixelBufferPool;             if (PixelBufferPool == nil)             {                 NSLog(@"PixelBufferPool is invalid.");                 exit(1);             }             CVReturn ret = CVPixelBufferPoolCreatePixelBuffer(nil, PixelBufferPool, &frameBuffer);             if (ret != kCVReturnSuccess)             {                 NSLog(@"Error creating framebuffer from pool");                 exit(1);             }             CVPixelBufferLockBaseAddress(frameBuffer, 0);             // This is where we would put image data into the buffer.  Nothing right now.             CVPixelBufferUnlockBaseAddress(frameBuffer, 0);             while(!track.readyForMoreMediaData);             CMTime presentationTime = CMTimeMake(frameTime+(i*timescale), timescale);             BOOL result = [pixelBufferAdaptor appendPixelBuffer:frameBuffer                                            withPresentationTime:presentationTime];             if (result == NO)             {                 NSLog(@"Error appending to track.");                 exit(1);             }             CVPixelBufferRelease(frameBuffer);         }         // Close everything         if ( trackWriter.status == AVAssetWriterStatusWriting)             [track markAsFinished];         NSLog(@"Completed.");     }     return 0; }

I receive a buffer from[AVSpeechSynthesizer convertToBuffer:fromBuffer:] and want to schedule it on an AVPlayerNode. The player node's output format need to be something that the next node could handle and as far as I understand most nodes can handle a canonical format. The format provided by AVSpeechSynthesizer is not something thatAVAudioMixerNode supports. So the following:   AVAudioEngine *engine = [[AVAudioEngine alloc] init];   playerNode = [[AVAudioPlayerNode alloc] init];   AVAudioFormat *format = [[AVAudioFormat alloc] initWithSettings:utterance.voice.audioFileSettings];   [engine attachNode:self.playerNode];   [engine connect:self.playerNode to:engine.mainMixerNode format:format]; Throws an exception: Thread 1: "[[busArray objectAtIndexedSubscript:(NSUInteger)element] setFormat:format error:&nsErr]: returned false, error Error Domain=NSOSStatusErrorDomain Code=-10868 \"(null)\"" I am looking for a way to obtain the canonical format for the platform so that I can use AVAudioConverter to convert the buffer. Since different platforms have different canonical formats, I imagine there should be some library way of doing this. Otherwise each developer will have to redefine it for each platform the code will run on (OSX, iOS etc) and keep it updated when it changes. I could not find any constant or function which can make such format, ASDB or settings. The smartest way I could think of, which does not work:   AudioStreamBasicDescription toDesc;   FillOutASBDForLPCM(toDesc, [AVAudioSession sharedInstance].sampleRate,                      2, 16, 16, kAudioFormatFlagIsFloat, kAudioFormatFlagsNativeEndian);   AVAudioFormat *toFormat = [[AVAudioFormat alloc] initWithStreamDescription:&toDesc]; Even the provided example for iPhone, in the documentation linked above, uses kAudioFormatFlagsAudioUnitCanonical and AudioUnitSampleType which are deprecated. So what is the correct way to do this?

I am using AVFoundation for live camera view. I can get my device from the current video input (of type AVCaptureDeviceInput) like: let device = videoInput.device The device's active format has a isPortraitEffectSupported. How can I set the Portrait Effect on and off in live camera view? I setup the camera like this: private var videoInput: AVCaptureDeviceInput! private let session = AVCaptureSession() private(set) var isSessionRunning = false private var renderingEnabled = true private let videoDataOutput = AVCaptureVideoDataOutput() private let photoOutput = AVCapturePhotoOutput() private(set) var cameraPosition: AVCaptureDevice.Position = .front func configureSession() { sessionQueue.async { [weak self] in guard let strongSelf = self else { return } if strongSelf.setupResult != .success { return } let defaultVideoDevice: AVCaptureDevice? = strongSelf.videoDeviceDiscoverySession.devices.first(where: {$0.position == strongSelf.cameraPosition}) guard let videoDevice = defaultVideoDevice else { print("Could not find any video device") strongSelf.setupResult = .configurationFailed return } do { strongSelf.videoInput = try AVCaptureDeviceInput(device: videoDevice) } catch { print("Could not create video device input: \(error)") strongSelf.setupResult = .configurationFailed return } strongSelf.session.beginConfiguration() strongSelf.session.sessionPreset = AVCaptureSession.Preset.photo // Add a video input. guard strongSelf.session.canAddInput(strongSelf.videoInput) else { print("Could not add video device input to the session") strongSelf.setupResult = .configurationFailed strongSelf.session.commitConfiguration() return } strongSelf.session.addInput(strongSelf.videoInput) // Add a video data output if strongSelf.session.canAddOutput(strongSelf.videoDataOutput) { strongSelf.session.addOutput(strongSelf.videoDataOutput) strongSelf.videoDataOutput.videoSettings = [kCVPixelBufferPixelFormatTypeKey as String: Int(kCVPixelFormatType_32BGRA)] strongSelf.videoDataOutput.setSampleBufferDelegate(self, queue: strongSelf.dataOutputQueue) } else { print("Could not add video data output to the session") strongSelf.setupResult = .configurationFailed strongSelf.session.commitConfiguration() return } // Add photo output if strongSelf.session.canAddOutput(strongSelf.photoOutput) { strongSelf.session.addOutput(strongSelf.photoOutput) strongSelf.photoOutput.isHighResolutionCaptureEnabled = true } else { print("Could not add photo output to the session") strongSelf.setupResult = .configurationFailed strongSelf.session.commitConfiguration() return } strongSelf.session.commitConfiguration() } } func prepareSession(completion: @escaping (SessionSetupResult) -&gt; Void) { sessionQueue.async { [weak self] in guard let strongSelf = self else { return } switch strongSelf.setupResult { case .success: strongSelf.addObservers() if strongSelf.photoOutput.isDepthDataDeliverySupported { strongSelf.photoOutput.isDepthDataDeliveryEnabled = true } if let photoOrientation = AVCaptureVideoOrientation(interfaceOrientation: interfaceOrientation) { if let unwrappedPhotoOutputConnection = strongSelf.photoOutput.connection(with: .video) { unwrappedPhotoOutputConnection.videoOrientation = photoOrientation } } strongSelf.dataOutputQueue.async { strongSelf.renderingEnabled = true } strongSelf.session.startRunning() strongSelf.isSessionRunning = strongSelf.session.isRunning strongSelf.mainQueue.async { strongSelf.previewView.videoPreviewLayer.session = strongSelf.session } completion(strongSelf.setupResult) default: completion(strongSelf.setupResult) } } } Then to I set isPortraitEffectsMatteDeliveryEnabled like this: func setPortraitAffectActive(_ state: Bool) { sessionQueue.async { [weak self] in guard let strongSelf = self else { return } if strongSelf.photoOutput.isPortraitEffectsMatteDeliverySupported { strongSelf.photoOutput.isPortraitEffectsMatteDeliveryEnabled = state } } } However, I don't see any Portrait Effect in the live camera view! Any ideas why?

Setting a voice for AVSpeechSynthesizer leads to an heap buffer overflow. Turn on address sanitizer in Xcode 14 beta and run the following code. Anybody else experiencing this problem, is there any workaround? let synthesizer = AVSpeechSynthesizer() var synthVoice : AVSpeechSynthesisVoice? func speak() { let voices = AVSpeechSynthesisVoice.speechVoices()           for voice in voices {       if voice.name == "Daniel" {    // select e.g. Daniel voice         synthVoice = voice       }     }           let utterance = AVSpeechUtterance(string: "Test 1 2 3")           if let synthVoice = synthVoice { utterance.voice = synthVoice     }           synthesizer.speak(utterance) // AddressSanitizer: heap-buffer-overflow }

Hello, We are developing to shoot 48MP Apple Pro Raw feature in my app, but we couldn't find any documentation on shooting 48MP Apple Pro Raw. Is it possible to shoot 48MP Apple ProRaw with a third-party app? Any help is much appreciated. Thanks.

Hello. In iOS16, I faced the problem that main thread become deadlock when call copyNextSampleBuffer method in AVAssetReaderOutput in requestMediaDataWhenReady method of AVAssetWriterInput. Main thread become deadlock sometimes and main thread's call stack like below. Thread 1 Queue : com.apple.main-thread (serial) #0 0x00000001ebd73b48 in mach_msg2_trap () #1 0x00000001ebd86008 in mach_msg2_internal () #2 0x00000001ebd86248 in mach_msg_overwrite () #3 0x00000001ebd7408c in mach_msg () #4 0x00000001b0059564 in CA::Render::Message::send_message() () #5 0x00000001b0235084 in CA::Render::Encoder::send_message(unsigned int, unsigned int, unsigned int*, unsigned long) () #6 0x00000001b0002274 in CA::Context::commit_transaction(CA::Transaction*, double, double*) () #7 0x00000001b0035148 in CA::Transaction::commit() () #8 0x00000001b001e4c8 in CA::Transaction::flush_as_runloop_observer(bool) () #9 0x00000001b1004870 in _UIApplicationFlushCATransaction () #10 0x00000001b1151c78 in _UIUpdateSequenceRun () #11 0x00000001b17958f8 in schedulerStepScheduledMainSection () #12 0x00000001b1794ac4 in runloopSourceCallback () #13 0x00000001aea32394 in __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ () #14 0x00000001aea3e76c in __CFRunLoopDoSource0 () #15 0x00000001ae9c2650 in __CFRunLoopDoSources0 () #16 0x00000001ae9d7fe8 in __CFRunLoopRun () #17 0x00000001ae9dd314 in CFRunLoopRunSpecific () #18 0x00000001e851f368 in GSEventRunModal () #19 0x00000001b0ea23e8 in -[UIApplication _run] () #20 0x00000001b0ea204c in UIApplicationMain () #21 0x000000010506f918 in main at /Users/user/Desktop/LINE/QA/line-ios/Talk/AppDelegate.swift:72 #22 0x00000001cd8d9960 in start () I tried to delay some logic to call markAsFinished and finishWriting methods of AVAssetWriter but that couldn't solve this problem. Is there anyone to experience same issue as me? If so, let me know what is workaround code if you solve this problem?

Hi! Just created a new empty project with just a AVSpeechSynthesizer var and when I try to play a tts string in app console I see: Query for com.apple.MobileAsset.VoiceServices.VoiceResources failed: 2 Unable to list voice folder [AXTTSCommon] MauiVocalizer: 11006 (Can't compile rule): regularExpression=\Oviedo(?=, (\x1b\pause=\d+\)?Florida)\b, message=unrecognized character follows , characterPosition=1 [AXTTSCommon] MauiVocalizer: 16038 (Resource load failed): component=ttt/re, uri=, contentType=application/x-vocalizer-rettt+text, lhError=88602000 Tested with Xcode Version 14.0.1 (14A400) on iPhone 14 Pro device ( iOS 16.0.3 ). More, in case Address Sanitizer is active app crashes: ERROR: AddressSanitizer: heap-buffer-overflow on address 0x000103e5ae75 at pc 0x000100ada758 bp 0x00016ff6e030 sp 0x00016ff6d7e8 READ of size 3062 at 0x000103e5ae75 thread T11 #0 0x100ada754 in wrap_strlen+0x164 (/private/var/containers/Bundle/Application/F72885E1-54FA-4FB9-B32E-320C8A8770A2/TestTTS.app/Frameworks/libclang_rt.asan_ios_dynamic.dylib:arm64e+0x16754)  #1 0x235339d78 in IsThisUrlOrRealPath+0x30 (/System/Library/PrivateFrameworks/TextToSpeechMauiSupport.framework/TextToSpeechMauiSupport:arm64e+0x2fd78) #2 0x235775994 in ve_ttsResourceLoad+0x16c (/System/Library/PrivateFrameworks/TextToSpeechMauiSupport.framework/TextToSpeechMauiSupport:arm64e+0x46b994)