Hello everyone, with the release of Apple's new Final Cut Camera App, we see the possibility to overlay a Focus Peaking indicator over the camera feed, showing focussed areas. We have already had a contrast based autofocus system for some time via the AVCaptureDevice.Format.AutoFocusSystem.contrastDetection, but I haven't found a way to actually present contrast areas to the user. Given that Apple now natively has such an algorithm for the Final Cut Camera App, I wonder if we devs now also get access to this. If not, does anybody know of implementations of focus peaking out there? Thanks and with best regards

Hello everyone, Is it possible to set AVCaptureMovieFileOutput to record the audio for my HECV video as 44100Hz 16 bit mono PCM. If yes: how does it work? Thank you in advance

Hi Team, I'm using AVPlayer on Apple TV 2nd generation which has that siri click pad with four buttons around and we need to detect where the user clicked on the Siri remote fast forward/backward. I have tested many different approaches to do it, but nothing is working for me. Does anyone have any idea how to resolve it in Swift?

iOS17.4 17.5.1 AVPlayer https://svip.yzzy23-play.com/20240606/14121_017bbbeb/index.m3u8 大概在8:30s时会无声 我测试过系统safari也会出现这样的情况 之前版本都没问题

Hi. I saw that in iOS 18 Beta there is a property "transition" on the Music Kit's ApplicationMusicPlayer. However, in my app I am using MPMusicPlayerApplicationController because I want to play Apple Music songs, local songs and podcasts. But I didn't find an analogue property on MPMusicPlayerApplicationController to specify transitions between songs. Am I missing something? Thanks, Dirk

Well, I will collect a lot of memes from the Internet and save them on my iPhone. I will name and classify them, but I will click on a photo in "All Photos", and its info does not show which album I added to, which makes me very distressed. If I have this function, I will easily manage the memes that I did not correctly add to the corresponding album.

I’m creating a objective C command-line utility to encode RAW image sequences to ProRes 4444, but I’m encountering, blocky compression artifacts in the ProRes 4444 video output. To test the integrity of the image data before encoding to ProRes, I added a snippet in my encoding function that saves a 16-bit PNG before encoding to ProRes and the PNG looks perfect, I can see all detail in every part of the image dynamic range. Here’s a comparison between the 16-bit PNG(on the right) and the ProRes 4444 output. (on the left) As a further test, I re-encoded the ‘test PNG’ to ProRes 4444 using DaVinci Resolve, and the ProRes4444 output video from Resolve doesn’t have any blocky compression artifacts. Looks identical. In short, this is what the utility does: Unpacks the 12-bit raw data into 16-bit values. After unpacking, the raw data is debayered to convert it into a standard color image format (BGR) using OpenCV. Scale the debayered pixel values from their original 12-bit depth to fit into a 16-bit range. Up to this point everything is fine and confirmed by saving 16bit PNGs. The images are encoded to ProRes 4444 using the AVFoundation framework. The pixel buffers are created and managed using dictionary method with ‘kCVPixelFormatType_64RGBALE’. I need help figuring this out, I’m a real novice when it comes to AVfoundation/encoding to ProRes. See relevant parts of my 'encodeToProRes' function: void encodeToProRes(const std::string &outputPath, const std::vector<std::string> &rawPaths, const std::string &proResFlavor) { NSError *error = nil; NSURL *url = [NSURL fileURLWithPath:[NSString stringWithUTF8String:outputPath.c_str()]]; AVAssetWriter *assetWriter = [AVAssetWriter assetWriterWithURL:url fileType:AVFileTypeQuickTimeMovie error:&error]; if (error) { std::cerr << "Error creating AVAssetWriter: " << error.localizedDescription.UTF8String << std::endl; return; } // Load the first image to get the dimensions std::cout << "Debayering the first image to get dimensions..." << std::endl; Mat firstImage; int width = 5320; int height = 3900; if (!debayer_image(rawPaths[0], firstImage, width, height)) { std::cerr << "Error debayering the first image" << std::endl; return; } width = firstImage.cols; height = firstImage.rows; // Save the first frame as a PNG 16-bit image for validation std::string pngFilePath = outputPath + "_frame1.png"; if (!imwrite(pngFilePath, firstImage)) { std::cerr << "Error: Failed to save the first frame as a PNG image" << std::endl; } else { std::cout << "First frame saved as PNG: " << pngFilePath << std::endl; } NSString *codecKey = nil; if (proResFlavor == "4444") { codecKey = AVVideoCodecTypeAppleProRes4444; } else if (proResFlavor == "422HQ") { codecKey = AVVideoCodecTypeAppleProRes422HQ; } else if (proResFlavor == "422") { codecKey = AVVideoCodecTypeAppleProRes422; } else if (proResFlavor == "LT") { codecKey = AVVideoCodecTypeAppleProRes422LT; } else { std::cerr << "Error: Invalid ProRes flavor specified: " << proResFlavor << std::endl; return; } NSDictionary *outputSettings = @{ AVVideoCodecKey: codecKey, AVVideoWidthKey: @(width), AVVideoHeightKey: @(height) }; AVAssetWriterInput *videoInput = [AVAssetWriterInput assetWriterInputWithMediaType:AVMediaTypeVideo outputSettings:outputSettings]; videoInput.expectsMediaDataInRealTime = YES; NSDictionary *pixelBufferAttributes = @{ (id)kCVPixelBufferPixelFormatTypeKey: @(kCVPixelFormatType_64RGBALE), (id)kCVPixelBufferWidthKey: @(width), (id)kCVPixelBufferHeightKey: @(height) }; AVAssetWriterInputPixelBufferAdaptor *adaptor = [AVAssetWriterInputPixelBufferAdaptor assetWriterInputPixelBufferAdaptorWithAssetWriterInput:videoInput sourcePixelBufferAttributes:pixelBufferAttributes]; ... [assetWriter startSessionAtSourceTime:kCMTimeZero]; CMTime frameDuration = CMTimeMake(1, 24); // Frame rate of 24 fps int numFrames = static_cast<int>(rawPaths.size()); ... // Encoding thread std::thread encoderThread([&]() { int frameIndex = 0; std::vector<CVPixelBufferRef> pixelBufferBuffer; while (frameIndex < numFrames) { std::unique_lock<std::mutex> lock(queueMutex); queueCondVar.wait(lock, [&]() { return !frameQueue.empty() || debayeringFinished; }); if (!frameQueue.empty()) { auto [index, debayeredImage] = frameQueue.front(); frameQueue.pop(); lock.unlock(); if (index == frameIndex) { cv::Mat rgbaImage; cv::cvtColor(debayeredImage, rgbaImage, cv::COLOR_BGR2RGBA); CVPixelBufferRef pixelBuffer = NULL; CVReturn result = CVPixelBufferPoolCreatePixelBuffer(NULL, adaptor.pixelBufferPool, &pixelBuffer); if (result != kCVReturnSuccess) { std::cerr << "Error: Could not create pixel buffer" << std::endl; dispatch_group_leave(dispatchGroup); return; } CVPixelBufferLockBaseAddress(pixelBuffer, 0); void *pxdata = CVPixelBufferGetBaseAddress(pixelBuffer); for (int row = 0; row < height; ++row) { memcpy(static_cast<uint8_t*>(pxdata) + row * CVPixelBufferGetBytesPerRow(pixelBuffer), rgbaImage.ptr(row), width * 8); } CVPixelBufferUnlockBaseAddress(pixelBuffer, 0); pixelBufferBuffer.push_back(pixelBuffer); ... Thanks very much!

I'm working on a macOS application that captures audio and video. When the user selects a video capture source (most likely an elgato box), I would like the application to automatically select the audio input from the same device. I was achieving this by pairing video and audio sources that had the same name, but this doesn't work when the user plugs in two capture devices of the same make and model. With the command system_profiler SPUSBDataType I can list all the USB devices, and I can see that the two elgato boxes have different serial numbers. If I could find this serial number, then I could figure out which AVCaptureDevices come from the same hardware. Is there a way to get the manufacturer's serial number from the AVCaptureDevice object? Or a way to identify the USB device for an AVCaptureDevice, and from there I could get the serial or some other unique ID?

I've had an app that edits photos in your library since the PhotoKit API was released in iOS 8. I know it was required if you preserve photo metadata you had to change the value of Orientation to 1 (up), otherwise PhotoKit would fail to perform the asset change request. When I remove this code, I'm seeing Orientation is getting changed to 1 automatically both at root and in the TIFF dictionary (tested with iOS 18). I wanted to confirm this is expected behavior, the system does this for us now? If so, can I remove this code for iOS 15+, or was it a recent iOS version this started happening? Thanks!

Is it possible to sort the user library assets by date captured? The Photos app in iOS 18 lets you choose between Date Captured and Recently Added and I want to offer that same choice in my app. This seems to always sort them by creation date (which I believe is the same as recently added): let assetCollection = PHAssetCollection.fetchAssetCollections(with: .smartAlbum, subtype: .smartAlbumUserLibrary, options: nil).firstObject! let fetchResult = PHAsset.fetchAssets(in: assetCollection, options: PHFetchOptions.imageMediaType())

Playing fMP4 HLS stream on VisionOS beta. This is the stream, HEVC main 10 and EAC3 6 channel: #EXT-X-STREAM-INF:BANDWIDTH=6760793,AVERAGE-BANDWIDTH=6760793,VIDEO-RANGE=PQ,CODECS="hvc1.2.4.L150.B0,mp4a.a6",RESOLUTION=3840x2160,FRAME-RATE=23.976,SUBTITLES="subs" This is what AVPlayer says: Error Domain=AVFoundationErrorDomain Code=-11848 "Cannot Open" UserInfo={NSLocalizedFailureReason=The media cannot be used on this device., NSLocalizedDescription=Cannot Open, NSUnderlyingError=0x3009e37b0 {Error Domain=CoreMediaErrorDomain Code=-15517 "(null)"}} I can't find any documentation for the underlying error 15517. Is it because "mp4a.a6" is declared in the codec list and not "ec-3"? hlsreport has these MUST FIX issues: 1. Measured peak bitrate compared to multivariant playlist declared value exceeds error tolerance Multivariant Playlist Stream Definition for All Variants 2. Stereo audio in AAC-LC, HE-AAC v1, or HE-AAC v2 format MUST be provided Multivariant Playlist 3. If Dolby Digital Plus is provided then Dolby Digital MUST be provided also Multivariant Playlist 4. I-frame playlists ( EXT-X-I-FRAME-STREAM-INF ) MUST be provided to support scrubbing and scanning UI Multivariant Playlist 5. The server MUST deliver playlists using gzip content-encoding All Variants All Renditions Multivariant Playlist 6. You MUST provide multiple bit rates of video Multivariant Playlist 7. Playlist codec type doesn't match content codec type All Variants 8. (Segment) The operation couldn’t be completed. (HTTPPumpErrorDomain error -16845 - HTTP 400: (unhandled)) (list of subtitle renditions) 9. (Segment) HTTP 400 - HTTP/2.0 400 Bad Request (list of subtitle renditions) 10. Multichannel audio MUST be separate audio stream All Variants 11. If EXT-X-INDEPENDENT-SEGMENTS is not in the multivariant playlist, then you MUST use the EXT-X-INDEPENDENT-SEGMENTS tag in all video media playlists All Variants 12. The CODECS attribute MUST include every media format present All Variants, does not declare EC-3

I generated an asset in the photolibrary by adding the unedited image, adjustmentData, and edited image with PHAssetCreationRequest.addResouce(). The image is saved in the photolibrary as HEIF. (Check with the photolibrary) Then, when I save the image generated with PHAssetCreationRequest.addResouce() to the Files app, it becomes JPEG. On the other hand, even if I edit an image taken with the camera and save it to the Files app, it still maintains the HEIF. Do you know why this happens? Also, how can I maintain the HEIF even when saving it in the Files app? Thanks.

Hello, Notice that when using h.265 VideoToolBox encoding with Handbrake (latest snapshots) the resulting output is larger than the 264 source. Obviously this was the case for OSX 14.x and below. Is this a known "regression" ? Thanks.

The probability of this issue occurring is very high when retrieving a newly taken Live Photo. This leads to failure in determining the Live Photo type. How can this issue be resolved?

Hello, We are seeing some strange behaviour when using MPRemoteCommandCenter as well as UIKeyCommand in our app on iOS 17. We are finding that when a UIKeyCommand is trigerred via external keyboard to start playing some music (via our own custom CoreAudio driver), the keyboard becomes unresponsive for a few seconds before UIKeyCommands are triggered again. Strangely enough, if we comment out all our MPRemoteCommandCenter code, the UIKeyCommands work without going into the unresponsive state for a few seconds. ie UIKeyCommands: override open var keyCommands: [UIKeyCommand] { let commands = [UIKeyCommand(title: String(localized: "__PLAY_STOP__"), action: #selector(shortcutPlayStop(_:)), input: " "), UIKeyCommand(title: String(localized: "__PAUSE__"), action: #selector(shortcutPause(_:)), input: "."), /** etc **/] commands.forEach { $0.wantsPriorityOverSystemBehavior = true } return commands } and MPRemoteCommands: MPRemoteCommandCenter.shared().playCommand.addTarget { [weak self] _ in self?.doStuff() // etc return .success } Note this issue did not occur prior to iOS17 🤨 Any ideas what could be going on? Thank you!

Hi, I'm working on a web project that uses the MediaSession API to interface with the media notification on iOS. The issue that I'm experiencing occurs after pressing the play button in the media session modal where the session seems to NOT fire the event handler callback and also kill the media session itself. It's a strange behaviour considering that the pause callback works fine. audio_source = new Audio(url); navigator.mediasession.metadata = { ... // Metadata here }; navigator.mediasession.setActionHandler('play', (details) => { audio_source.play(); } ); navigator.mediasession.setActionHandler('pause', (details) => { audio_source.pause(); } );

Hello, I'm having a problem with my Iphone 12: iOS version: 17.5.1 Model name: Iphone 12 When I call someone it's impossible to communicate, it seems that the microphone is deactivated or that it doesn't respond. I was able to extract the ips file from the "data": stacks+audiomxd-.... Here's an extract: {"bug_type":"288","timestamp":"2024-06-16 22:44:25.00 +0200","os_version":"iPhone OS 17.5.1 (21F90)","roots_installed":0,"incident_id":"7B8604DF-3863-4760-806C-591A90A7A7A4"} { "crashReporterKey" : "6928f591dd9e4d26541855e6d4b6a20d408bdfd1", "exception" : "0xbe18d1ee", "frontmostPids" : [ 34 ], "tuning" : { }, "absoluteTime" : 2954071508880, "product" : "iPhone13,2", "kernel" : "Darwin Kernel Version 23.5.0: Wed May 1 20:35:15 PDT 2024; root:xnu-10063.122.3~3\/RELEASE_ARM64_T8101", "date" : "2024-06-16 22:44:25.08 +0200", "reason" : "XPC message timeout in -[AVAudioSessionRemoteXPCClient getProperty:propertyName:MXProperty:reply:], probably deadlocked. Writing a stackshot and terminating.", "codeSigningMonitor" : 1, "incident" : "7B8604DF-3863-4760-806C-591A90A7A7A4", "build" : "iPhone OS 17.5.1 (21F90)", "roots_installed" : 0, "bug_type" : "288", "pid" : 102, "memoryStatus" : {"compressorSize":38088,"compressions":25391066,"decompressions":20835948,"busyBufferCount":3,"memoryPressureDetails":{"pagesWanted":467,"pagesReclaimed":2085},"pageSize":16384,"memoryPressure":false,"memoryPages":{"active":67555,"throttled":0,"fileBacked":60679,"wired":57187,"purgeable":3679,"inactive":65541,"free":1533,"speculative":2364}}, As you can see, one line states : "reason" : "XPC message timeout in -[AVAudioSessionRemoteXPCClient getProperty:propertyName:MXProperty:reply:], probably deadlocked. Writing a stackshot and terminating.", Deadlock occurs when two or more processes are waiting for resources held by the others, creating a situation where none of the processes can progress. In this case, it seems that the process has tried to recover an audio property, but has remained blocked indefinitely. What reinforces my doubts is that at the end of the .ips is a line that seems to give some informations as to where the problem is : "notes" : ["Requested by audiomxd","_dyld_process_info_create(623) for resampling UUIDs failed with 1","_dyld_process_info_create(2535) for resampling UUIDs failed with 1","_dyld_process_info_create(3181) for resampling UUIDs failed with 1","_dyld_process_info_create(3183) for resampling UUIDs failed with 1","_dyld_process_info_create(3503) for resampling UUIDs failed with 1","resampled 409 of 1813 threads with truncated backtraces from 0 pids: ","resampled 625 of 37 images missing from 175 pids: 75,93,98,178,190,210,...,3627"], Here's what my research yielded. Could you please help me? All calls are useless because the microphone no longer works. Regards

We have an app with a broadcast extension with a RPBroadcastSampleHandler. The implementation is working fine, however for quite some users the extension suddenly crashes during the broadcast. The stacktrace stacktrace of the crashing thread always looks like the shortened sample below. (Full crash reports and stack traces are attached to the submitted Feedbacks.) Looking at the stacktrace none of our code is running, just ReplayKit code handling XPC messages at that moment: Thread: #0 0x00000001e2cf342c in __pthread_kill () #1 0x00000001f6a51c0c in pthread_kill () #2 0x00000001a1bfaba0 in abort () #3 0x00000001a9e38588 in malloc_vreport () #4 0x00000001a9e35430 in malloc_zone_error () [...] #18 0x0000000218ac91bc in -[RPBroadcastSampleHandler processPayload:completion:] () #19 0x0000000198b81360 in __NSXPCCONNECTION_IS_CALLING_OUT_TO_EXPORTED_OBJECT_S2__ () Is anyone aware of there issues with ReplayKit? Are there known workarounds? Could anything we're doing affect crashes like this? Would greatly appreciate it if anyone from Apple DTS could look into this and flag the below Feedbacks to the relevant teams! Feedback IDs: FB13949098, FB13949188

I film and edit content using my iPhone and PremierePro. I've been doing this since 2017, so I'd like to say I know my way around an iPhone and the PremierePro software enough to get a finished product to my client. Before I send content to clients, I airdrop it to my phone to make sure that the quality of it is up to par. On this most recent project, I've had issues airdropping it to myself. Each time I do so, my iPhone prompts which 3rd party app I would like to open the video in, rather than automatically opening or saving the video into my photo library. I will list the specs below: Filmed on iPhone 15 Plus iOS Version 17.5.1 (at the moment this is the most up to date software update) Filmed in 4K at 60 FPS I have ample storage space on my phone and the video file size is 220MB Premiere Pro Export Settings: Video Settings - H.264, Field Order: Progressive, Bit Rate Encoding: CBR I will say that I purchased a transition and burns bundle and used it for the first time on this project. All materials used are in an .mp4 format and blend mode was set to overlay. Nothing out of pocket. I figured it wouldn't be a problem since my client would just be downloading it via dropbox, but there was an issue there as well. My client received an error message saying, "Sorry, this type of video cannot be saved to this device". The back road workaround was to take the saved video, plug it into a separate PremierePro project, and export it with the preset: Match Source - Adaptive High Bitrate. I was able to airdrop it to myself, it saved in my photo album and my client was able to download it without receiving that error message. If there is an explanation as to why I am having this issue and how I can avoid it, I would really appreciate it as I have never had this problem in the past.

My app stores and transports lots of groups of similar PNGs. These aren't compressed well by official algorithms like .lzfse, .lz4, .lzbitmap... not even bz2, but I realized that they are well-suited for compression by video codecs since they're highly similar to one another. I ran an experiment where I compressed a dozen images into an HEVCWithAlpha .mov via AVAssetWriter, and the compression ratio was fantastic, but when I retrieved the PNGs via AVAssetImageGenerator there were lots of artifacts which simply wasn't acceptable. Maybe I'm doing something wrong, or maybe I'm chasing something that doesn't exist. Is there a way to use video compression like a specialized archive to store and retrieve PNGs losslessly while retaining alpha? I have no intention of using the videos except as condensed storage. Any suggestions on how to reduce storage size of many large PNGs are also welcome. I also tried using HEVC instead of PNG via the new UIImage.hevcData(), but the decompression/processing times were just insane (5000%+ increase), on top of there being fatal errors when using async.