Hello, We are currently developing a mobile game using Unreal Engine 5, and we have encountered an issue where a specific video (mp4 format) stops displaying at a particular frame during playback within the game. The code within Unreal fails at the following point, causing the issue: CMTime OutputItemTime = [Output itemTimeForHostTime:CACurrentMediaTime()]; if (![Output hasNewPixelBufferForItemTime:OutputItemTime]) { return; } We have referred to the following Apple documentation: AVPlayerTimeControlStatus reasonForWaitingToPlay Upon logging, we observed the following: [2024.08.13-05.18.35:266][429]LogTemp: AMP PlayerItem.status AVPlayerItemStatusReadyToPlay [2024.08.13-05.18.35:266][429]LogTemp: AMP MediaPlayer.timeControlStatus AVPlayerTimeControlStatusWaitingToPlayAtSpecifiedRate [2024.08.13-05.18.35:266][429]LogTemp: AMP reasonForWaitingToPlay: AVPlayerWaitingToMinimizeStallsReason [2024.08.13-05.18.35:266][429]LogTemp: AMP MediaPlayer.rate 1.000000 [2024.08.13-05.18.35:268][430]LogTemp: avf CurrentMediaTime : 455097.836833 [2024.08.13-05.18.35:268][430]LogTemp: avf OutputItemTime: 3.868346 [2024.08.13-05.18.35:268][430]LogTemp: avf Sampler::Tick() fail hasNewPixelBufferForItemTime OutputItemTime: 3.868346 This issue consistently occurs with videos that have the following specifications: Codec: H.264 Resolution: 1080x608 Bitrate: 7,922,135 bits/sec Duration: 90.17 seconds Frame Rate: 30.0 fps Pixel Format: yuv420p Profile: Main We would like to inquire about the possible reasons for the playback failure and the recommended MP4 specifications for seamless playback on Apple devices. Specifically, we need guidance on recommended resolution, FPS, profile, level, and bitrate limits. Your assistance would be greatly appreciated.

I am working on a project for a university which wants to alter the passthrough camera feed more so than the standard filters (saturation/contract/etc) that some of the headsets provide. I don't have access to the headset or enterprise SDK yet, as I'd like to nail down whether or not this is feasible before we purchase the hardware. In the API I see I can use CameraFrameProvider to access a CameraFrame and then grab a sample. The sample has a CVPixelBuffer. I have 2 questions regarding the pixelBuffer: I see that the buffer itself is read only but can I alter the bytes within this pixel buffer? Lets say change all green pixels to red (not my actual use case but just an example) Will the updated pixel buffer then be used in the passthrough screen? If not, then is there any way to have control over the video feed that is being displayed as passthrough? Our ideal setup would be to have access to a frame, alter it however we want, and then have the frame displayed in passthrough. I realize I could take the feed and copy it into a floating window and alter that, but that breaks the immersion we are shooting to create here. Thanks in advance!

I'm making an app that reads a ProRes file, processes each frame through metal to resize and scale it, then outputs a new ProRes file. In the future the app will support other codecs but for now just ProRes. I'm reading the ProRes 422 buffers in the kCVPixelFormatType_422YpCbCr16 pixel format. This is what's recommended by Apple in this video https://developer.apple.com/wwdc20/10090?time=599. When the MTLTexture is run through a metal performance shader, the colorspace seems to force RGB or is just not allowing yCbCr textures as the output is all green/purple. If you look at the render code, you will see there's a commented out block of code to just blit copy the outputTexture, if you perform the copy instead of the scaling through MPS, the output colorspace is fine. So it appears the issue is from Metal Performance Shaders. Side note - I noticed that when using this format, it brings in the YpCbCr texture as a single plane. I thought it's preferred to handle this as two separate planes? That said, if I have two separate planes, that makes my app more complicated as I would need to scale both planes or merge it to RGB. But I'm going for the most performance possible. A sample project can be found here: https://www.dropbox.com/scl/fo/jsfwh9euc2ns2o3bbmyhn/AIomDYRhxCPVaWw9XH-qaN0?rlkey=sp8g0sb86af1u44p3xy9qa3b9&dl=0 Inside the supporting files, there is a test movie. For ease, I would move this to somewhere easily accessible (i.e Desktop). Load and run the example project. Click 'Select Video' Select that video you placed on your desktop It will now output a new video next to the selected one, named "Output.mov" The new video should just be scaled at 50%, but the colorspace is all wrong. Below is a photo of before and after the metal performance shader.

1.In the FxRemoteWindowAPI protocol, there is no way to set window.frame.origin. 2.When using NSWindow, you cannot set [Window setLevel:NSFloatingWindowLevel]. 3.How can I keep the window in front of Final Cut Pro without affecting the normal use of Final Cut Pro?

How is it possible to enable EDR on Apple TV without AVFoundation for custom HDR video playback? The use case is a custom video player for HDR playback via VideoToolbox and Metal, which seem to render colors correctly on iOS but not on tvOS. All related documentation and WWDC sessions describe APIs that are unavailable for tvOS: let metalLayer = CAMetalLayer() metalLayer.wantsExtendedDynamicRangeContent = true metalLayer.edrMetadata = CAEDRMetadata.hdr10(minLuminance: 0.0, maxLuminance: 1000, opticalOutputScale: 100) What's the alternative path for tvOS to have correct system tone mapping for a setup like: metalLayer.pixelFormat = .rgba16Float // (or .bgr10_xr) metalLayer.colorspace = CGColorSpace(name: CGColorSpace.itur_2100_PQ) Video format: HEVC, YUV 4:2:0 10bit, BT.2020 PQ. We do set the preferredDisplayCriteria on AVDisplayManager and thus video range matching is in place. WWDC Ref: https://developer.apple.com/videos/play/wwdc2022/110565?time=557

I am have implemented PIP in my app. When user goes to background while video was playing. App opens PIP and everything works fine. But when user locks the device , PIP is not stopping/Pausing the video. Is there a way I can pause the Video when user locks the device?

I'm trying to secure my m3u8 streaming link with a token. To achieve this, I'm using AVAssetResourceLoaderDelegate in my SwiftUI app. However, the video doesn't play in AVPlayer when I'm using the AVAssetResourceLoaderDelegate. I can see that data is being received in the resourceLoader, but the player does not start playback. Here's the code I'm using: @State private var player: AVPlayer? @EnvironmentObject var pilot: UIPilot<AppRoute> var body: some View { VStack { VerticalSpacer(height: 50) HStack { Image(systemName: "arrow.left") .onTapGesture { pilot.pop() } Spacer() Text("liveStreamData.titleShort") .font(.poppins(.semibold, size: 18)) .lineLimit(1) HorizontalSpacer(width: 16) Spacer() } .padding(.horizontal) if let player = player { VideoPlayer(player: player) .onAppear { player.play() } .onDisappear { player.pause() } } else { Text("Loading video...") } } .onAppear { setupPlayer() } } private func setupPlayer() { guard let url = URL(string: "https://assets.afcdn.com/video49/20210722/v_645516.m3u8") else { print("Invalid URL") return } // Replace the scheme with a custom scheme var components = URLComponents(url: url, resolvingAgainstBaseURL: false) components?.scheme = "customscheme" // Change the scheme to a custom one guard let customURL = components?.url else { print("Failed to create custom URL") return } let asset = AVURLAsset(url: customURL) // Set the resource loader delegate let resourceLoaderDelegate = VideoResourceLoaderDelegate() asset.resourceLoader.setDelegate(resourceLoaderDelegate, queue: DispatchQueue.main) let playerItem = AVPlayerItem(asset: asset) player = AVPlayer(playerItem: playerItem) } } class VideoResourceLoaderDelegate: NSObject, AVAssetResourceLoaderDelegate { func resourceLoader(_ resourceLoader: AVAssetResourceLoader, shouldWaitForLoadingOfRequestedResource loadingRequest: AVAssetResourceLoadingRequest) -> Bool { guard let url = loadingRequest.request.url else { print("Invalid request URL") return false } // Replace the custom scheme with the original HTTP/HTTPS scheme var components = URLComponents(url: url, resolvingAgainstBaseURL: false) components?.scheme = "https" // Change the scheme back to HTTP/HTTPS guard let originalURL = components?.url else { print("Failed to convert URL back to HTTPS") return false } // Fetch the data from the original URL let urlSession = URLSession.shared let task = urlSession.dataTask(with: originalURL) { data, response, error in if let error = error { print("Error loading resource: \(error)") loadingRequest.finishLoading(with: error) return } if let data = data, let dataRequest = loadingRequest.dataRequest { print("Data loaded: \(data.count) bytes") dataRequest.respond(with: data) loadingRequest.finishLoading() } else { print("No data received") loadingRequest.finishLoading(with: NSError(domain: "VideoResourceLoader", code: -1, userInfo: nil)) } } task.resume() return true } func resourceLoader(_ resourceLoader: AVAssetResourceLoader, didCancel loadingRequest: AVAssetResourceLoadingRequest) { print("Loading request was canceled") } } Problem: The video does not play when using AVAssetResourceLoaderDelegate. The data is being loaded correctly as confirmed by the logs, but AVPlayer fails to start playback. Without the resource loader, the video plays without any issues. Question: What could be causing the player to not play the video when using AVAssetResourceLoaderDelegate? Are there any additional steps or configurations I need to ensure smooth playback while using a resource loader? Any help would be greatly appreciated!

I'm using the "Converting side-by-side 3D video to multiview HEVC and spatial video" sample code on iOS. It takes about 8 seconds to convert a 6-second video. At this rate, a 1-hour video would take 1.3 hours to convert. How can I speed up the conversion? BTW, are there solutions to convert side-by-side 3D video to spatial video for Windows?

I was trying to migrate Core Image based code that's rotating an image in a CVPixelBuffer to the newer VTPixelRotationSession from Video Toolbox. Hoping to increase performance. The original code does: let rotatedImage = CIImage(cvPixelBuffer: origPixelBuffer).oriented(.left) context.render(rotatedImage, to: newPixelBuffer) The new code uses a session: _ = VTPixelRotationSessionRotateImage(rotationSession, origPixelBuffer, newPixelBuffer) However I immediately ran into memory limitations, since my code has to be able to run in an iOS extension. It seems VTPixelRotationSessionRotateImage easily lets memory usage spike over the 50MB of allowed memory. While the CIImage based implementation has no such high memory usage at all. Is this expected? Does the VTPixelRotationSession implementation gain more performance by sacrificing memory? Or is there something I'm overlooking? I was expecting the VTPixelRotationSession at worst to be on par in terms of memory usage and processing speed compared to CIImage. At this moment it seems VTPixelRotationSession is unusable in extensions. See also Feedback: FB14977240

When displaying and playing multiple HLS videos (4 or 6 screens) side by side using AVPlayer on iPad devices running iOS 17 or later, even though the videos are set to play at normal speed, some frames appear to be skipped, causing the videos to play faster than intended. This issue occasionally occurs when repeatedly playing and pausing the videos, and the more screens there are, the more frequently it happens. However, the occurrence rate is not very high (about 1 in 50 times). This phenomenon has been reproduced on iPad devices running iOS 17 or later and does not occur on devices running iOS 16 or earlier. Devices where the issue has been confirmed: iPad 6th generation / iOS ver 17.6.1 iPad 9th generation / iOS ver 17.6.1 iPad Pro 11-inch 1st generation / iOS ver 17.4.1 I have tried implementing countermeasures based on information from similar issues, such as those mentioned on the following website, but the problem remains unresolved: https://stackoverflow.com/questions/77224167/avplayer-unexpected-behaviour-after-ios-and-tvos-update-to-17-0 From the console logs, I observed that on devices running iOS 17 or later, the following log was output: AppleD5500: Bad NAL type 10 I suspect that some kind of decoding failure may be occurring, leading to the issue described above. If you have any information or can provide support on this matter, I would greatly appreciate it.

Hi, Im trying to use this example (https://developer.apple.com/documentation/avfoundation/media_reading_and_writing/converting_side-by-side_3d_video_to_multiview_hevc_and_spatial_video) to encode a stereoscopic (left eye right eye) video frame using MVHEVC. The sample project creates tagged buffers for left and right eye, and uses a writer to write the MVHEC encoded video buffers. But i after i get right and left tagged buffers, i want to use VideoMaterial and its AVSampleBufferVideoRenderer to enqueue these video frames. If i render MVHEVC encoded left eye sample buffer, and right eye sample buffer, sequentially will the AVSampleBufferVideoRenderer render it as a stereoscopic view? How does this work with VideoMaterial and AVSampleBufferVideoRenderer ? Thanks!

Hi, Recording Videos with AVAssetWriter, capture fps(camera output fps) is ok, but final result video fps was lower, the reason is AVAssetWriterInput.isReadyForMoreMediaData is false sometimes. Yes, I have read document many times, it said need to set expectsMediaDataInRealTime to true and balabala... I really be tortured by this problem for a long time, can I debug this problem? or any advice?

I work on a video editing app that composes multiple small video clips, sometimes hundreds or thousands. For one user in particular, attempting to export causes a failure 100% of the time. The failure occurs in the initialization of AVAssetReader, and is in the AVFoundationErrorDomain with code -11819 (AVErrorMediaServicesWereReset.) We've done everything we can think of, including quitting other running apps, enabling airplane mode, and even performing the flow on an identical device using the customer's data, and have had no luck pinning down the cause of the error. Does anyone have any suggestion for how we might go about debugging this? Getting ready to file a TSI but thought I should ask here first.

When setting the now playing info for playing media in MPNowPlayingInfoCenter we can set artwork. But it seems the Apple API for creating the artwork is crashing on iOS 18 (FB15145734). On iOS 17 this gave the warning that the completion handler was not run on the main thread. I've tried to seek help here: https://stackoverflow.com/questions/78989543/swift-data-race-with-appkit-mpmediaitemartwork-function/78990231?noredirect=1#comment139277425_78990231 but it seems that it's not possible to override the completion handler and therefor it's up to Apple to fix this issue. .task { await MainActor.run { let nowPlayingInfoCenter = MPNowPlayingInfoCenter.default() var nowPlayingInfo = [String: Any]() let image = NSImage(named: "image")! // warning: data race detected: @MainActor function at MPMediaItemArtwork/ContentView.swift:22 was not called on the main thread nowPlayingInfo[MPMediaItemPropertyArtwork] = MPMediaItemArtwork(boundsSize: image.size, requestHandler: { _ in // Not on main thread here! return image }) nowPlayingInfoCenter.nowPlayingInfo = nowPlayingInfo } } I'm wondering if there is an alternative method to set the now playing artwork?

I updated macOS to 15.0 yesterday, and I found some floating point value support under CMFormatDescriptionExtensions and CVPixelBuffer's Attachment seems to be broken. When I call CMSampleBufferCreateReadyWithImageBuffer() from CVPixelBuffer, macOS 15.0 always fail with floating point values. a. kCMFormatDescriptionExtension_GammaLevel Previous macOS 14.x works with double value like NSString* keyGamma = (__bridge NSString*)kCMFormatDescriptionExtension_GammaLevel; extensions[keyGamma] = @(2.2); b. kCMFormatDescriptionExtension_CleanAperture I am not sure yet but such non-integer value issue also seems to be applied to CleanAperture. kCMFormatDescriptionKey_CleanApertureWidth kCMFormatDescriptionKey_CleanApertureHeight kCMFormatDescriptionKey_CleanApertureHorizontalOffset kCMFormatDescriptionKey_CleanApertureVerticalOffset Also, When I add rational values to extensions, it cannot pass CMVideoFormatDescriptionMatchesImageBuffer() with: kCMFormatDescriptionKey_CleanApertureWidthRational kCMFormatDescriptionKey_CleanApertureHeightRational kCMFormatDescriptionKey_CleanApertureHorizontalOffsetRational kCMFormatDescriptionKey_CleanApertureVerticalOffsetRational Is there any known workaround?

AVKit provides the SwiftUI view VideoPlayer, and allows you to add an interactive overlay. But that overlay is normally placed behind the system-provided playback controls. Is there any way to suppress those controls, without resorting to wrapping AVPlayerView?

I'm trying to create code to generate an fcpxml file so I can automate Final Cut Pro timeline (project) creation. Here's an xml element that FCP successfully imports (and successfully creates a project/timeline). <project name="2013-08-09 19_23_07 (id).mov"> <sequence format="r1"> <spine> <asset-clip ref="r2" offset="0s" name="2013-08-09 19_23_07 (id).mov" start="146173027/60000s" duration="871871/60000s" tcFormat="DF" audioRole="dialogue"></asset-clip> </spine> </sequence> </project> The xml element example above was generated by exporting a simple timeline with a single clip. The problem I'm having is the media asset has timecode that gives a start time in relation to the timecode. When I try to remove timecode attributes and change the start time to "0s" <asset-clip ref="r2" offset="0s" name="2013-08-09 19_23_07 (id).mov" start="0s" duration="871871/60000s" audioRole="dialogue"></asset-clip> FCP complains with the import error: 2013-08-09 19_23_07 (id).fcpxml Invalid edit with no respective media. (/fcpxml[1]/project[1]/sequence[1]/spine[1]/asset-clip[1]) I guess the question is, does AVAsset provide a way to get the timecode information and the timecode based start offset, or is there a way to tell FCP to use a default start time independent of timecode?

Basic iPhone 11 iOS 17.5.1 Main Thread libsystem_kernel.dylib___ulock_wait (in libsystem_kernel.dylib) +8 libdispatch.dylib__dlock_wait (in libdispatch.dylib) +52 libdispatch.dylib__dispatch_thread_event_wait_slow (in libdispatch.dylib) +52 libdispatch.dylib___DISPATCH_WAIT_FOR_QUEUE__ (in libdispatch.dylib) +364 libdispatch.dylib__dispatch_sync_f_slow (in libdispatch.dylib) +144 MediaToolbox_fpic_CopyCurrentEvent (in MediaToolbox) +132 AVFCore___104-[AVPlayer _setRate:withVolumeRampDuration:playImmediately:rateChangeReason:affectsCoordinatedPlayback:]_block_invoke_2 (in AVFCore) +244 AVFCore-[AVPlayer _setRate:withVolumeRampDuration:playImmediately:rateChangeReason:affectsCoordinatedPlayback:] (in AVFCore) +276 AVFCore-[AVPlayer setRate:] (in AVFCore) +56 call AVPlayer pause Thread 81 name: fpic-sync libsystem_kernel.dylib___ulock_wait (in libsystem_kernel.dylib) +8 libdispatch.dylib__dlock_wait (in libdispatch.dylib) +52 libdispatch.dylib__dispatch_thread_event_wait_slow (in libdispatch.dylib) +52 libdispatch.dylib___DISPATCH_WAIT_FOR_QUEUE__ (in libdispatch.dylib) +364 libdispatch.dylib__dispatch_sync_f_slow (in libdispatch.dylib) +144 MediaToolbox_itemasync_CopyProperty (in MediaToolbox) +588 MediaToolbox_fpic_CurrentItemMoment (in MediaToolbox) +184 MediaToolbox___fpic_EstablishCurrentEventForCurrentItem_block_invoke (in MediaToolbox) +136 libdispatch.dylib__dispatch_client_callout (in libdispatch.dylib) +16 libdispatch.dylib__dispatch_lane_barrier_sync_invoke_and_complete (in libdispatch.dylib) +52 MediaToolbox_fpic_ServiceCurrentEvent (in MediaToolbox) +600 MediaToolbox___fpic_NotifyServiceCurrentEvent_block_invoke (in MediaToolbox) +912 libdispatch.dylib__dispatch_call_block_and_release (in libdispatch.dylib) +28 libdispatch.dylib__dispatch_client_callout (in libdispatch.dylib) +16 libdispatch.dylib__dispatch_lane_serial_drain (in libdispatch.dylib) +744 libdispatch.dylib__dispatch_lane_invoke (in libdispatch.dylib) +428 libdispatch.dylib__dispatch_root_queue_drain (in libdispatch.dylib) +388 libdispatch.dylib__dispatch_worker_thread (in libdispatch.dylib) +256 libsystem_pthread.dylib__pthread_start (in libsystem_pthread.dylib) +132 libsystem_pthread.dylib_thread_start (in libsystem_pthread.dylib) +4 Thread 93 name: com.apple.coremedia.player.async.0x303c60240.P/GR libsystem_kernel.dylib_mach_msg2_trap (in libsystem_kernel.dylib) +8 libsystem_kernel.dylib_mach_msg2_internal (in libsystem_kernel.dylib) +76 libsystem_kernel.dylib_mach_msg_overwrite (in libsystem_kernel.dylib) +432 libsystem_kernel.dylib_mach_msg (in libsystem_kernel.dylib) +20 libdispatch.dylib__dispatch_mach_send_and_wait_for_reply (in libdispatch.dylib) +540 libdispatch.dylib_dispatch_mach_send_with_result_and_wait_for_reply (in libdispatch.dylib) +56 libxpc.dylib_xpc_connection_send_message_with_reply_sync (in libxpc.dylib) +260 CoreMedia_FigXPCConnectionSendSyncMessageCreatingReply (in CoreMedia) +288 CoreMedia_FigXPCRemoteClientSendSyncMessageCreatingReply (in CoreMedia) +44 MediaToolbox_remoteXPCPlayer_SetRateWithOptions (in MediaToolbox) +148 MediaToolbox_playerasync_runOneCommand (in MediaToolbox) +768 MediaToolbox_playerasync_runAsynchronousCommandOnQueue (in MediaToolbox) +180 libdispatch.dylib__dispatch_client_callout (in libdispatch.dylib) +16 libdispatch.dylib__dispatch_lane_serial_drain (in libdispatch.dylib) +744 libdispatch.dylib__dispatch_lane_invoke (in libdispatch.dylib) +428 libdispatch.dylib__dispatch_root_queue_drain (in libdispatch.dylib) +388 libdispatch.dylib__dispatch_worker_thread (in libdispatch.dylib) +256 libsystem_pthread.dylib__pthread_start (in libsystem_pthread.dylib) +132 libsystem_pthread.dylib_thread_start (in libsystem_pthread.dylib) +4

When update as IOS18 , Display automatically dims while the iPhone is video calling with LINE and it is stabilizing.

<<<< FigPlayerInterstitial >>>> fpic_ServiceCurrentEvent signalled err=-15671 (kFigPlayerInterstitialError_ClientReleased) (no primary) at FigPlayerInterstitialCoordinator.m:7885 <<<< FigPlayerInterstitial >>>> fpic_ServiceCurrentEvent signalled err=-15671 (kFigPlayerInterstitialError_ClientReleased) (no primary) at FigPlayerInterstitialCoordinator.m:7885 <<<< FigPlayerInterstitial >>>> fpic_ServiceCurrentEvent signalled err=-15671 (kFigPlayerInterstitialError_ClientReleased) (no primary) at FigPlayerInterstitialCoordinator.m:7885 <<<< FigPlayerInterstitial >>>> fpic_ServiceCurrentEvent signalled err=-15671 (kFigPlayerInterstitialError_ClientReleased) (no primary) at FigPlayerInterstitialCoordinator.m:7885 <<<< FigPlayerInterstitial >>>> fpic_ServiceCurrentEvent signalled err=-15671 (kFigPlayerInterstitialError_ClientReleased) (no primary) at FigPlayerInterstitialCoordinator.m:7885 My project uses AVPlayer (AVPlayerViewController) to play video. There are continuous warning logs while playing and when it goes to dealloc, it prints information below. <<<< PlayerRemoteXPC >>>> remoteXPCItem_handleSetProperty signalled err=-12860 (kFigPlayerError_ParamErr) (propertyValue should be MTAudioProcessingTap) at FigPlayer_RemoteXPC.m:2760 This only happens in iOS 18 and I have no idea about this. There is no any information for FigPlayerInterstitial and else.