I was wondering of anyone had guidance on how to “livestream“ MV-HEVC content. More specifically, I have a left and right eye view for stereoscopic content (perhaps, for example, the views were taken from a stereoscopic video being passed through an AVPlayer). I know, based on sample code, that I can convert the stereoscopic video into a MV-HEVC file using AVAssetWriter. However, how would I take the stereoscopic video and encode it, in realtime, to a stream that could then leverage HLS Tools to deliver to clients? Is AVFoundation capable of this directly? Or is there an API within VideoToolbox that can help with this?

I'm getting an issue even unencrypted video playback also failing with status failed. Error Domain=CoreMediaErrorDomain Code=-12927 "(null)" I unable to find any info on above error code. Is there some way to look this up? Sample master M3U8 is shared below. Note: If I use any variant M3U8 then it is working playing fine.

After the session video, "Build a great Lock Screen camera capture experience", was unclear about the UI. So do developers need to provide a whole new UI in the extension? The main UI cannot be repurposed?

Is AVQT capable of being used to measure encoding quality of PQ or HLG based content beyond SDR? If so, how am I able to leverage it. If not, is there a roadmap for timing to enable this type of tool?

Hello, I have converted UIImage to CVPixelBuffer. I am creating a video writing app. In some cases, the same CVPixelBuffer should last in the video for 2 seconds or more. However, I need to add 30 CVPixelBuffers per second because the video, to work on social media, must be 30 frames per second. The problem is that whenever I try to add frames to long videos, like 50-minute videos, it gives an error. The error is something like "Operation cannot be completed". Give me an example of a loop to add 30 CVPixelBuffers per second to a currently written video. Example: while true { if videoInput.isReadyForMoreMediaData { break } if videoInput.isReadyForMoreMediaData, let buffer = videoProvider.getNextFrame() { adaptor.append(buffer, withPresentationTime: CMTime(value: 1, timescale: 30)) } } I await your response.

Hey! I'm working on a camera app and I've noticed that the .builtInTripleCamera doesn't behave anything like the native app. Tested on iPhone 15 Pro Max and iPhone 12 Pro Max. The documentation states the following, but that seems quite different from what is happening in the app: Automatic switching from one camera to another occurs when the zoom factor, light level, and focus position allow. So, does it automatically switch like the native camera, or do I need to do something? Custom Camera vs Native Camera Custom Camera Native Camera The code was adapted from the Apple's project AVCamFilter. Just download the AVCamFilter and update videoDeviceDiscoverySession: private let videoDeviceDiscoverySession = AVCaptureDevice.DiscoverySession( deviceTypes: [.builtInTripleCamera], mediaType: .video, position: .unspecified )

I am trying to use the AVCamFilter Apple sample project discussed in this WWDC session to get depth data using the dual camera. The project has built-in features to get depth data from the dual camera. When the sample project was written builtInDualWideCamera didn't exist yet, and the project only tries to get builtInDualCamera and builtInWideAngleCamera. When I run the project on my iPad Pro it doesn't show any of the depth-related UI because the device doesn't have a builtInDualCamera device. So I added builtInDualWideCamera in to the videoDeviceDiscoverySession, and it seems to get that device properly, but isDepthDataDeliverySupported is returning false still. Is there some reason why isDepthDataDeliverySupported is false even though I seem to be using a dual camera device? I know the device has a builtInLiDARDepthCamera but I wanted to try out the dual camera depth data to see how it performs for shorter distances. I wouldn't have expected the dual camera depth data delivery to be made unavailable on the device just because the LiDAR sensor is already available. Using iPadOS 17.5.1, iPad Pro 11-inch 4th generation. The depth feature of this sample app works fine on an iPhone 15 I tested. Also tried on an iPhone 15 Pro and it worked even though that device also has a LiDAR sensor, so the issue is presumably not related to the fact that the iPad Pro has a LiDAR sensor.

I have built a camera application which uses a AVCaptureSession with the AVCaptureDevice set to .builtInDualWideCamera and isVirtualDeviceConstituentPhotoDeliveryEnabled=true to enable delivery of "simultaneous" photos (AVCapturePhoto) for a single capture request. I am using the hd1920x1080 preset, but both the wide and ultra-wide photos are being delivered in the highest possible resolution (4224x2376). I've tried to disable any setting that suggests that it should be using that 4k resolution rather than 1080p on the AVCapturePhotoOutput, AVCapturePhotoSettings and AVCaptureDevice, but nothing has worked. Some debugging that I've done: When I turn off constituent photo delivery by commenting out the line of code below, I end up getting a single photo delivered with the 1080p resolution, as you'd expect. // photoSettings.virtualDeviceConstituentPhotoDeliveryEnabledDevices = captureDevice.constituentDevices I tried the constituent photo delivery with the .builtInDualCamera and got only 4k results (same as described above) I tried using a AVCaptureMultiCamSession with .builtInDualWideCamera and also only got 4k imagery I inspected the resolved settings on photo.resolvedSettings.photoDimensions, and the dimensions suggest the imagery should be 1080p, but then when I inspect the UIImage, it is always 4k. guard let imageData = photo.fileDataRepresentation() else { return } guard let capturedImage = UIImage(data: imageData ) else { return } print("photo.resolvedSettings.photoDimensions", photo.resolvedSettings.photoDimensions) // 1920x1080 print("capturedImage.size", capturedImage.size) // 4224x2376 -- Any help here would be greatly appreciated, because I've run out of things to try and documentation to follow 🙏

I'm experiencing an issue with microphone recording in my app when launched from a Shortcut. The app works correctly when launched directly, but launching it through the Shortcut results in the "Session activation failed" error (code 561015905). Here's what I've done so far: My app has microphone permission granted. The startRecording function sets the audio session category to .playAndRecord. I've implemented error handling within startRecording to catch the error code. The Shortcut workflow includes an action to launch the app (no explicit microphone permission request within the Shortcut). xcode version - 15.2 iphone ios version - 17.4.1

when audio file's magic number is 49443302 not 49443303, AVAudioPlayer's duration property return wrong value, actually it cause by engiTunSMPB but I want to know why it happen only in ID3 version 2 (49443302) example: only difference in two mp3 file is the magic number and check the duration returns this result source code under here ContentView.swift

I have an AVPlayerViewController in my app playing video and my largest source of errors is NSURLErrorDomain Code=-1008. The underlying error they provide is Error Domain=CoreMediaErrorDomain Code=-12884 "(null)". I couldn't find what the error implies or is caused by - osstatus.com also does not have a reference to it.

When I send a build in Xcode the process occurs normally, but a few minutes later I receive an e-mail saying: "ITMS-90683: Missing purpose string in Info.plist - Your app’s code references one or more APIs that access sensitive user data, or the app has one or more entitlements that permit such access. The Info.plist file for the “***.app” bundle should contain a NSMicrophoneUsageDescription key with a user-facing purpose string explaining clearly and completely why your app needs the data. If you’re using external libraries or SDKs, they may reference APIs that require a purpose string. While your app might not use these APIs, a purpose string is still required." So the problem is the description of the use of the microphone, right? As the attached image shows that I have already done this process, and I continue to receive this error. Even when I remove the part of the avfoundation code that uses the microphone to try to submit the build, the error continues to be returned to me

Is there possible to record audio during application in kill/terminate state? I am working with application where I have to record microphone voice for some process so I can it during application in foreground and background state, I have no idea about application kill state.

In the code example provided there is a bool in the Video object to set a video as 3D: /// A Boolean value that indicates whether the video contains 3D content. let is3D: Bool I have a hosted spatial video that I know works correctly on the AVP player. When I point the Videos.json file to the this URL and set is3D=true my 3D video doesn't show up and I get the follow error: iPVC/1-0 Received playback error: [Error Domain=AVFoundationErrorDomain Code=-11850 "Operation Stopped" UserInfo={NSLocalizedFailureReason=The server is not correctly configured., NSLocalizedDescription=Operation Stopped, NSUnderlyingError=0x30227c510 {Error Domain=CoreMediaErrorDomain Code=-12939 "byte range length mismatch - should be length 2 is length 2434" UserInfo={NSDescription=byte range length mismatch - should be length 2 is length 2434, NSURL=https: <omitted for post> }}}] Can anyone tell me what might be going on? The error is telling me my server is not configured correctly. For context, I'm using a google drive to deliver dynamic images/videos using: https://drive.google.com/uc?export=download&id= <file ID> And the above works great for my images and 2d videos. Is there something I need to do specifically when delivering MV-HEVC videos?

Hey all, I have a pretty complicated camera setup so bear with me. You know how Instagram's Camera supports recording a video and flipping Camera devices while recording? I built the same thing using AVCaptureVideoDataOutput and it works fine, but it does not support rotation. (neither does Instagram, but I still need it, lol) So there's two ways to implement rotation (and mirroring) in AVCaptureVideoDataOutput: 1. Set it on the AVCaptureConnection Rotation and vertical mirror mode can be set directly on the AVCaptureVideoDataOutput's connection to the Camera: let output = AVCaptureVideoDataOutput(...) cameraSession.addOutput(output) for connection in output.connections { connection.videoRotation = 90 connection.isVideoMirrored = true } But according to the documentation this is expensive and comes with a performance overhead. I haven't really benchmarked it yet, but I assume rotating and mirroring 4k buffers isn't cheap. I'm building a camera library that is used by a lot of people, so all performance decisions have a big impact. 2. Set it on AVAssetWriter Instead of actually physically rotating large pixel buffers, we can also just set the AVAssetWriter's transform property to some affine transformation - which is comparable to how EXIF tags work. We can set both rotation and mirror modes using CGAffineTransforms. Obviously this is much more efficient and does not come with a performance overhead on the camera pipeline at all, so I'd prefer to go this route. Problem The problem is that when I start recording with the front Camera (AVAssetWriter.transform has a mirror on the CGAffineTransform), and then flip to the back Camera, the back Camera is also mirrored. Now I thought I could just avoid rotation on my buffers and only use isVideoMirrored on the AVCaptureConnection when we are using the front camera, which is a fair performance compromise - but this won't work because isVideoMirrored applies mirroring alongside the vertical axis - and since the video stream is naturally in landscape orientation, this will flip the image upside down instead of mirroring it alongside the vertical axis... whoops! 😅 This is pretty obvious as the transform applies to the entire video stream, but now I am not sure if the AVAssetWriter approach will work for my use-case. I think I will need to eagerly physically rotate the pixel buffers by setting the AVCaptureConnection's videoRotation & isVideoMirrored properties, but I wanted to ask here in case someone knows any alternatives to doing that in order to avoid the performance and memory overhead of rotating buffers? Thanks!

I've been studying the AVCam example and notice that everything pertaining to state transitions for the capture session is performed on a dedicated DispatchQueue. My question is this: Can I use an actor instead?

I'm getting the following output from my iOS app's debug console, note the error on the last line: Capture format keys: ["600x600@25", "1200x1200@5", "1200x1200@30", "1600x1200@2", "1600x1200@30", "3200x2400@15", "3200x2400@2", "600x600@30"] Start capture session for 1600x1200@30: <AVCaptureSession: 0x303c70190 [AVCaptureSessionPresetPhoto]> Stop capture session: <AVCaptureSession: 0x303c70190 [AVCaptureSessionPresetInputPriority]> <AVCaptureDeviceInput: 0x303ebb720 [Medwand S3 Camera]>[vide] -> <AVCaptureVideoDataOutput: 0x303edf1e0> <AVCaptureDeviceInput: 0x303ebb720 [Medwand S3 Camera]>[vide] -> <AVCapturePhotoOutput: 0x303ee3e20> <AVCaptureDeviceInput: 0x303ebb720 [Medwand S3 Camera]>[vide] -> <AVCaptureVideoPreviewLayer: 0x3030b33c0> Start capture session for 600x600@30: <AVCaptureSession: 0x303c70190 [AVCaptureSessionPresetInputPriority]> <AVCaptureDeviceInput: 0x303ebb720 [Medwand S3 Camera]>[vide] -> <AVCaptureVideoDataOutput: 0x303edf1e0> <AVCaptureDeviceInput: 0x303ebb720 [Medwand S3 Camera]>[vide] -> <AVCapturePhotoOutput: 0x303ee3e20> <AVCaptureDeviceInput: 0x303ebb720 [Medwand S3 Camera]>[vide] -> <AVCaptureVideoPreviewLayer: 0x3030b33c0> <<<< FigSharedMemPool >>>> Fig assert: "blkHdr->useCount > 0" at (FigSharedMemPool.c:591) - (err=0) This is in response to trying to switch capture formats between the two key modes that must regularly be used by my application. Below you will find the functions that I use to start and stop capturing frames to my preview layer. I have a UI with three buttons. Off Mode 1 Mode 2 If I tap the Off button in between tapping Mode 1 or Mode 2 all is well; I can do this all day. However, attempt to jump between Mode 1andMode 2` directly I run into issues. I added a layer of software between the UI and the underlying functions so that I could make sure to turn off the Camera before turning it back on in the opposite mode and was surprised to get this output. Can someone at Apple please tell me what is going on here? For the rest of you, if anyone knows the magic incantation to safely switch camera formats, please paste that code here. Thanks. I've included my code below. func start(for deviceFormat: String) { sessionQueue.async { [unowned self] in logger.debug("Start capture session for \(deviceFormat): \(self.captureSession)") do { guard let format = formatDict[deviceFormat] else { throw Error.captureFormatNotFound } captureSession.stopRunning() captureSession.beginConfiguration() // May not be necessary. try captureDevice.lockForConfiguration() // Without this we get an error. captureDevice.activeFormat = format captureDevice.unlockForConfiguration() // Matching function: Necessary. captureSession.commitConfiguration() // Matching function: May not be necessary. captureSession.startRunning() } catch { logger.fault("Failed to start camera: \(error.localizedDescription)") errorPublisher.send(error) } } } func stop() { sessionQueue.async { [unowned self] in logger.debug("Stop capture session: \(self.captureSession)") captureSession.stopRunning() } }

We are facing a weird behaviour when implementing the AirPlay functionality of our iOS app. When we test our app on Apple TV devices everything works fine. On some smart TVs with a specific AirPlay receiver version, (more details below) the stream gets stuck on buffering state immediately after switching to AirPlay mode. On other smart TVs, with different AirPlay receiver version, everything works as expected. The interesting part is that other free or DRM protected streams, work fine on all devices. Smart TVs that AirPlay works fine AirPlay Version -> 25.06 (19.9.9) Smart TVs that AirPlay stuck at buffering state: AirPlayReceiverSDKVersion -> 3.3.0.54 AirPlayReceiverAppVersion -> 53.122.0 You can reproduce this issue using the following stream url: https://tr.vod.cdn.cosmotetvott.gr/v1/310/668/1674288197219/1674288197219.ism/.m3u8?qual=a&ios=1&hdnts=st=1713194669\~exp=1713237899\~acl=\*/310/668/1674288197219/1674288197219.ism/\*\~id=cab757e3-9922-48a5-988b-3a4f5da368b6\~data=de9bbd0100a8926c0311b7dbe5389f7d91e94a199d73b6dc75ea46a4579769d7~hmac=77b648539b8f3a823a7d398d69e5dc7060632c29 If this link expires, notify me to send a new one for testing. Could you please provide to us any specific suggestion as to what causes this issue on those specific streams?

Hi everyone, I am having a problem on AVPlayer when I try to play some videos. The video starts for a few seconds, but immediately after I see a black screen and in the console there is the following error: <__NSArrayM 0x14dbf9f30>( { StreamPlaylistError = "-12314"; comment = "have audio audio-aacl-54 in STREAMINF without EXT-X-MEDIA audio group"; date = "2024-05-13 20:46:19 +0000"; domain = CoreMediaErrorDomain; status = "-12642"; uri = "http://127.0.0.1:8080/master.m3u8"; }, { "c-conn-type" = 1; "c-severity" = 2; comment = "Playlist parse error"; "cs-guid" = "871C1871-D566-4A3A-8465-2C58FDC18A19"; date = "2024-05-13 20:46:19 +0000"; domain = CoreMediaErrorDomain; status = "-12642"; uri = "http://127.0.0.1:8080/master.m3u8"; } )

When using VideoPlayer in SwiftUI it adds a black background to the video player and I would like to make it clear. Trying to add .background() of clear does nothing. Seems like a really silly "default" style to add this black background to a video. Any suggestions on how to achieve this?