Hi, I'm trying to modify the ScreenCaptureKit Sample code by implementing an actor for Metal rendering, but I'm experiencing issues with frame rendering sequence. My app workflow is: ScreenCapture -> createFrame -> setRenderData Metal draw callback -> renderAsync (getData from renderData) I've added timestamps to verify frame ordering, I also using binarySearch to insert the frame with timestamp, and while the timestamps appear to be in sequence, the actual rendering output seems out of order. // ScreenCaptureKit sample func createFrame(for sampleBuffer: CMSampleBuffer) async { if let surface: IOSurface = getIOSurface(for: sampleBuffer) { await renderer.setRenderData(surface, timeStamp: sampleBuffer.presentationTimeStamp.seconds) } } class Renderer { ... func setRenderData(surface: IOSurface, timeStamp: Double) async { _ = await renderSemaphore.getSetBuffers( isGet: false, surface: surface, timeStamp: timeStamp ) } func draw(in view: MTKView) { Task { await renderAsync(view) } } func renderAsync(_ view: MTKView) async { guard await renderSemaphore.beginRender() else { return } guard let frame = await renderSemaphore.getSetBuffers( isGet: true, surface: nil, timeStamp: nil ) else { await renderSemaphore.endRender() return } guard let texture = await renderSemaphore.getRenderData( device: self.device, surface: frame.surface) else { await renderSemaphore.endRender() return } guard let commandBuffer = _commandQueue.makeCommandBuffer(), let renderPassDescriptor = await view.currentRenderPassDescriptor, let renderEncoder = commandBuffer.makeRenderCommandEncoder(descriptor: renderPassDescriptor) else { await renderSemaphore.endRender() return } // Shaders .. renderEncoder.endEncoding() commandBuffer.addCompletedHandler() { @Sendable (_ commandBuffer)-> Swift.Void in updateFPS() } // commit frame in actor let success = await renderSemaphore.commitFrame( timeStamp: frame.timeStamp, commandBuffer: commandBuffer, drawable: view.currentDrawable! ) if !success { print("Frame dropped due to out-of-order timestamp") } await renderSemaphore.endRender() } } actor RenderSemaphore { private var frameBuffers: [FrameData] = [] private var lastReadTimeStamp: Double = 0.0 private var lastCommittedTimeStamp: Double = 0 private var activeTaskCount = 0 private var activeRenderCount = 0 private let maxTasks = 3 private var textureCache: CVMetalTextureCache? init() { } func initTextureCache(device: MTLDevice) { CVMetalTextureCacheCreate(kCFAllocatorDefault, nil, device, nil, &self.textureCache) } func beginRender() -> Bool { guard activeRenderCount < maxTasks else { return false } activeRenderCount += 1 return true } func endRender() { if activeRenderCount > 0 { activeRenderCount -= 1 } } func setTextureLoaded(_ loaded: Bool) { isTextureLoaded = loaded } func getSetBuffers(isGet: Bool, surface: IOSurface?, timeStamp: Double?) -> FrameData? { if isGet { if !frameBuffers.isEmpty { let frame = frameBuffers.removeFirst() if frame.timeStamp > lastReadTimeStamp { lastReadTimeStamp = frame.timeStamp print(frame.timeStamp) return frame } } return nil } else { // Set let frameData = FrameData( surface: surface!, timeStamp: timeStamp! ) // insert to the right position let insertIndex = binarySearch(for: timeStamp!) frameBuffers.insert(frameData, at: insertIndex) return frameData } } private func binarySearch(for timeStamp: Double) -> Int { var left = 0 var right = frameBuffers.count while left < right { let mid = (left + right) / 2 if frameBuffers[mid].timeStamp > timeStamp { right = mid } else { left = mid + 1 } } return left } // for setRenderDataNormalized func tryEnterTask() -> Bool { guard activeTaskCount < maxTasks else { return false } activeTaskCount += 1 return true } func exitTask() { activeTaskCount -= 1 } func commitFrame(timeStamp: Double, commandBuffer: MTLCommandBuffer, drawable: MTLDrawable) async -> Bool { guard timeStamp > lastCommittedTimeStamp else { print("Drop frame at commit: \(timeStamp) <= \(lastCommittedTimeStamp)") return false } commandBuffer.present(drawable) commandBuffer.commit() lastCommittedTimeStamp = timeStamp return true } func getRenderData( device: MTLDevice, surface: IOSurface, depthData: [Float] ) -> (MTLTexture, MTLBuffer)? { let _textureName = "RenderData" var px: Unmanaged<CVPixelBuffer>? let status = CVPixelBufferCreateWithIOSurface(kCFAllocatorDefault, surface, nil, &px) guard status == kCVReturnSuccess, let screenImage = px?.takeRetainedValue() else { return nil } CVMetalTextureCacheFlush(textureCache!, 0) var texture: CVMetalTexture? = nil let width = CVPixelBufferGetWidthOfPlane(screenImage, 0) let height = CVPixelBufferGetHeightOfPlane(screenImage, 0) let result2 = CVMetalTextureCacheCreateTextureFromImage( kCFAllocatorDefault, self.textureCache!, screenImage, nil, MTLPixelFormat.bgra8Unorm, width, height, 0, &texture) guard result2 == kCVReturnSuccess, let cvTexture = texture, let mtlTexture = CVMetalTextureGetTexture(cvTexture) else { return nil } mtlTexture.label = _textureName let depthBuffer = device.makeBuffer(bytes: depthData, length: depthData.count * MemoryLayout<Float>.stride)! return (mtlTexture, depthBuffer) } } Above's my code - could someone point out what might be wrong?

When I tried to launch my application from non-gui process (from launch daemon) NSworkspace openApplicationAtURL failed if I tried to run it when my device on the login screen. Everything is working if someone logged in, but on the login screen I have the error The application “TestApp” could not be launched because a miscellaneous error occurred. with code 256 NSWorkspace* workspace = [NSWorkspace sharedWorkspace]; NSWorkspaceOpenConfiguration* config = [NSWorkspaceOpenConfiguration configuration]; config.createsNewApplicationInstance = YES; config.activates = NO; config.promptsUserIfNeeded = NO; config.addsToRecentItems = NO; [workspace openApplicationAtURL: appURL configuration: config completionHandler:^(NSRunningApplication *app, NSError *error) { }]; Sometimes after the third try it works, sometimes not at all. I try to use "open" command, it works on MacOS Sequoia, but not working for operating systems below, I see this error The application cannot be opened for an unexpected reason, error=Error Domain=RBSRequestErrorDomain Code=5 "Launch failed." UserInfo={NSLocalizedFailureReason=Launch failed., NSUnderlyingError=0x600002998120 {Error Domain=OSLaunchdErrorDomain Code=125 "Domain does not support specified action" UserInfo={NSLocalizedFailureReason=Domain does not support specified action}}} All these problems occur only on the login screen. I'm developing screen share utility, so I need somehow to launch my application on the login screen. Could someone please help me understand what is recommended way to launch application on the login screen?

I understand that there are no delegate methods for this. But to determine a positive consent from user to Record Screen needs to be evaluated via block parameter. When user denies the permission by mistake, and, if user tries again, the alert is not showing up. How do I reset the permission and throw the below alert again?

I've made a simple command line app that requires Screen recording permission. When I ran it from Xcode, it prompts for a permission and once I allowed it from the settings, it runs well. #include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <CoreGraphics/CGDisplayStream.h> int main() { printf("# Start #\n"); if (CGPreflightScreenCaptureAccess()) { printf("# Permitted.\n"); } else { printf("# Not permitted.\n"); if (CGRequestScreenCaptureAccess() == false) { printf("# CGRequestScreenCaptureAccess() returning false\n"); } } size_t output_width = 1280; size_t output_height = 720; dispatch_queue_t dq = dispatch_queue_create("com.domain.screengrabber", DISPATCH_QUEUE_SERIAL); CGError err; CGDisplayStreamRef sref = CGDisplayStreamCreateWithDispatchQueue( 1, output_width, output_height, 'BGRA', NULL, dq, ^( CGDisplayStreamFrameStatus status, uint64_t time, IOSurfaceRef frame, CGDisplayStreamUpdateRef ref ) { printf("Got frame: %llu, FrameStatus:%d \n", time, status); } ); err = CGDisplayStreamStart(sref); if (kCGErrorSuccess != err) { printf("Error: failed to start streaming the display. %d\n", err); exit(EXIT_FAILURE); } while (true) { usleep(1e5); } CGDisplayStreamStop(sref); printf("\n\n"); return 0; } Now I want to execute this from terminal, so I went to the build folder and typed the app name. cd /Users/klee/Library/Developer/Xcode/DerivedData/ScreenStreamTest-ezddqbkzhndhakadslymnvpowtig/Build/Products/Debug ./ScreenStreamTest But I am getting following output without any prompt for permission. # Start # # Not permitted. # CGRequestScreenCaptureAccess() returning false Error: failed to start streaming the display. 1001 Is there a something I need to consider for this type of command line app?

I've been using CGWindowListCreateImage which automatically creates an image with the size of the captured window. But SCScreenshotManager.captureImage(contentFilter:configuration:) always creates images with the width and height specified in the provided SCStreamConfiguration. I could be setting the size explicitly by reading SCWindow.frame or SCContentFilter.contentRect and multiplying the width and height by SCContentFilter.pointPixelScale , but it won't work if I want to keep the window shadow with SCStreamConfiguration.ignoreShadowsSingleWindow = false. Is there a way and what's the best way to take full-resolution screenshots of the correct size? import Cocoa import ScreenCaptureKit class ViewController: NSViewController { @IBOutlet weak var imageView: NSImageView! override func viewDidAppear() { imageView.imageScaling = .scaleProportionallyUpOrDown view.wantsLayer = true view.layer!.backgroundColor = .init(red: 1, green: 0, blue: 0, alpha: 1) Task { let windows = try await SCShareableContent.excludingDesktopWindows(false, onScreenWindowsOnly: true).windows let window = windows[0] let filter = SCContentFilter(desktopIndependentWindow: window) let configuration = SCStreamConfiguration() configuration.ignoreShadowsSingleWindow = false configuration.showsCursor = false configuration.width = Int(Float(filter.contentRect.width) * filter.pointPixelScale) configuration.height = Int(Float(filter.contentRect.height) * filter.pointPixelScale) print(filter.contentRect) let windowImage = try await SCScreenshotManager.captureImage(contentFilter: filter, configuration: configuration) imageView.image = NSImage(cgImage: windowImage, size: CGSize(width: windowImage.width, height: windowImage.height)) } } }

I have a macOS app in production, supporting all macOS versions since 10.15 (Catalina) thru Sequoia. One aspect of the app's functionality is to screen capture the entire screen, including all windows. Starting with Sequoia, my users are receiving a scary system alert saying: "SomeApp" is requesting to bypass the system private window picker and directly access your screen and audio. This will allow SomeApp to record your screen and system audio, including personal or sensitive information that may be visible or audible. I have several questions and concerns about this alert. First of all, as a developer, this is frustrating, as I am using documented, long-standing system APIs, and made no change to my code to cause this warning. Second, nothing in my app records audio in any fashion, and yet the user is made to think I am trying to furtively bypass security controls to record audio, which is absolutely false. The alert seems to be due to the screen capture feature, which is one of the main features of the app, which the user explicitly requests and grants permission for. But to get to the point of the question: is there any definitive documentation anywhere describing exactly which API's trigger this alert? I can't find first-party information from Apple, so I'm kind of guessing in the dark. Searching the internet for all the info I can find (mostly from blog posts of developers and beta-testers), it seemed like the culprit in my code was probably a call to CGWindowListCreateImage, so I spent some time forking the code paths in my app (since I still support back to 10.15) to use the more modern ScreenCaptureKit APIs on systems that support it. But the alert is still appearing, despite not calling into that API at all. Is there a way of calling the modern ScreenCaptureKit APIs that also triggers this alert? As an example, I'm using a snippet like this to get the shareable displays I need do { try await SCShareableContent.excludingDesktopWindows( false, onScreenWindowsOnly: false ) return true } catch { return false } is it possible that this code is triggering the alert because I'm not excluding desktop windows and asking for all windows? to sum up, I (and I'm guessing others) could really use some definitive guidelines on exactly which APIs trigger this alert, so that we can migrate and avoid them if possible. can anyone provide any guidance on this? Thanks in advance!

Our remote access application uses ScreenCaptureKit for capturing the screen in the user context and CGDisplayStream API as a fallback when running in the context of the Login Window. Environment: macOS 15.0; macOS 15.1 beta; Xcode 16 The application is authorized by the user for System Screen recording. The GUI process runs under the root user over the Login Screen. The calling thread gets stuck on CGDisplayStreamCreateWithDispatchQueue() for exactly 30 seconds. The method returns NULL afterward. The same code worked fine on Sonoma

I'm running a launch agent in a CI node. The agent is responsible for launching CI build/test jobs. The agent, being the responsible process, has been granted kTCCServiceScreenCapture permission. With this in place I can run /usr/sbin/screencapture during CI test jobs, archiving the visual state of the CI machine if a test fails, which makes it easier to diagnose why the test failed. However with macOS 15 I get weekly/monthly notifications about the agent being able to record the screen. The general advice for this is that apps should migrate to ScreenCaptureKit, but I'm using a built in tool in macOS, /usr/sbin/screencapture, so how am I supposed to deal with that?

I'm using screenCaptureKit for winodow capture. I build a filter like follow code, (I'm not usng independent window filter, because sometime I need capture multi windows in the same time) filter = [[SCContentFilter alloc] initWithDisplay:displayID includingWindows:includingWindows]; At begining, the capture works OK. When the target window's size or position changed, my code monitored this change and called updateConfiguration like below , I got completionHandler without error. [streamConfiguration setSourceRect:newRect]; [streamConfiguration setWidth:newWidth]; [streamConfiguration setHeight:newHeight]; [scStream updateConfiguration:streamConfiguration completionHandler:^(NSError *_Nullable error) { if (error) { // some error log } else { // update done } }]; But sometimes, it still output frame with old size, and the rect is still the old. And int some other cases, it works fine..... Is there any special work before call updateConfiguration to make it work ?

I want to capture windows of other applications for sharing during the use of SharePlay in Vision Pro. Can I use screencapturekit or is there another method?

I had no luck to compile a sample code provided by apple with Xcode 16.0 beta 5. ScreenCaptureKit demo (https://developer.apple.com/documentation/screencapturekit/capturing_screen_content_in_macos) The part it is failling is, streamOutput.capturedFrameHandler = { continuation.yield($0) } And the error message is Sending '$0' risks causing data races Task-isolated '$0' is passed as a 'sending' parameter; Uses in callee may race with later task-isolated uses Please enlighten me why this is an issue and how to avoid? Thanks in advance!

Hello, I am trying to make use of SCContentSharingPicker for my app and I wonder how I can detect a close event of SCContentSharingPicker. I could open the picker screen with following simple code: SCContentSharingPicker.shared.isActive = true SCContentSharingPicker.shared.add(self) SCContentSharingPicker.shared.present() And I closed it with "Cancel" button located at the top right corner. Initially I was expecting to get a event through an observer like below but realised that it's called when a stream is canceled. extension ContentPickerButton: SCContentSharingPickerObserver { func contentSharingPicker(_ picker: SCContentSharingPicker, didCancelFor stream: SCStream?) { logger.info("Picker canceled for stream \(stream)") } I would like to get a picker close event so that I can deactivate the picker. (Otherwise, camera icon will stay alive at the tray.) How do we get a close event?

One of my apps, Default Folder X, is an unconventional user of screen recording (and now ScreenCaptureKit). Part of its functionality is to add navigation controls to the Open and Save dialogs of other applications. It does this via the Accessibility API, and because of the limitations of that API, it sometimes has to actually pop up a menu in the target app's file dialog. To hide this from the user, it takes a screenshot of the Open or Save dialog and displays it in front of the dialog as a façade while it does its menu manipulation. Here's an example without the use of a captured image: https://www.stclairsoft.com/blog/wp-content/uploads/2024/08/EmptyFolderBehindTheCurtain.mov And an example with the façade: https://www.stclairsoft.com/blog/wp-content/uploads/2024/08/EmptyFolderWithScreenshot.mov This use case prevents me from using SCContentSharingPicker, as it's not a user-driven screen capture. Moreover, Sequoia b5's weekly screen recording reminders are popping up while the user is interacting with an Open or Save dialog, severely impacting his / her workflow. It appears that the Persistent Content Capture entitlement may prevent Sequoia from putting up the weekly warnings, though there's no documentation of the entitlement other than it being listed here: https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_developer_persistent-content-capture. So my questions: Is that what the Persistent Content Capture entitlement does? Where is the form to request this entitlement? Without the entitlement, I can't see continuing the use of screen captures. And eliminating that will compromise the UI in my app in the way I've described above. It will also make Default Folder X unable to tailor its UI to match the Open and Save dialogs of the app it's enhancing (there's no API for it to use to get the light / dark mode of the window of another app, so it currently captures an image of the target file dialog to determine its UI mode). Thanks - Jon

The scenario is quite simple run an application which uses [SCShareableContent getShareableContentExcludingDesktopWindows] and invoke captureImageWithFilter in completionHandler. delay invoking captureImageWithFilter for several seconds and switch user session before call it. The WindowServer crashes if app runs in inactive session. How to manage this issue correctly? Are there any way to avoid this crash?

Hello, I have a few apps that I use for screen recording/streaming like OBS as well as capturing screens to project into a VR headset (Immersed), and they use ScreenCaptureKit to record the full displays/all content. But when capturing that display, some application windows or UI elements, like in Microsoft Teams when you begin a screen sharing session and you get a control-bar overlay to manage sharing options (stop, start, etc), that particular element does not get captured by the recording app's capture (though other MS Teams windows and all the other applications on screen do). Another app that has this problem is the CleanShot X screen capturing app, where it's overlay UI elements don't get captured, but are still on the physical screen. This of course when using Mac displays in VR causes in issue where you can't see these particular CleanShot controls but they are there and intercepting mouse clicks/input. What would be causing only certain elements to not get captured when the recording app is telling ScreenCaptureKit to not exclude anything, and is there a property on these UI elements that the developer can "opt in" to get SCK to pick them up? I am trying to figure out what feedback to give to developers of programs that have this issue/if it's possible for them to modify their apps to change this behavior? Thanks!

I have a mac os app that uses screen capture logic. It was originally coded using the Quartz CG api: if let cgimage = CGDisplayCreateImage(CGMainDisplayID(), rect: cgRect) {...} and this worked as expected even when capturing a screen rect that began on secondary monitor and ended on primary monitor (or was entirely contained on secondary monitor). However now that API is deprecated and you're supposed to use ScreenCaptureKit instead. So I have attempted to convert the code. The trial code is: let scConfig = SCStreamConfiguration() scConfig.sourceRect = drect scConfig.width = Int(drect.width) scConfig.height = Int(drect.height) SCScreenshotManager.captureImage(contentFilter: sFilter, configuration: scConfig) {any,error in if let cgim = any { print("image dims \(cgim.width), \(cgim.height), requested: \(drect)") self.writeToFile2(cgim) } else { print("SCREEN CAP failed") } } ... where sFilter was previously set based on main screen display (with no exclusions). This code also "works" as long as the capture rect is entirely on primary monitor. But it fails if the rect spans both monitors or is fully contained on secondary monitor. (By fails I mean it produces empty image) So my question is: How to use ScreenCaptureKit to obtain screen shot of rectangle that spans dual monitors?

Hi, I am developing a background screen recording app and macOS Sequoia is constantly blocking the app after restart with a nag screen "Confirm to continue the screen recording". That's looking insane! I know that the built-in Grab Screenshot app is not requesting any permission and that is an anti-competitive beaviour! I have found inside the compiled code the following plist XML code that disables the software firewall TCC, implemented by Apple. It's undocumented for no reason. Anyone knows how to repeat this unblocking tweak?

Hi, my app ScreenFloat can capture screenshots and record the screen (along with system- and microphone audio). It does this in an XPC service. On macOS Sequoia b1-3, recording does not work anymore (although permissions are granted to the app in System Preferences > Privacy & Security). Instead, I keep getting an error that my XPC service can access this computer's screen and audio. (of course, that's the point!) First of all, the screen is locked when the warning appears, clicks anywhere on the screen are not recognized. I have to hit Escape (or wait about a minute, at which point it resolves itself), to be able to click anywhere. Clicking on Continue To Allow doesn't do anything, either. The warning just re-appears every time. Do I need to add a new entitlement to my main app or the XPC service, or any new NSUsage strings to the InfoPlist.strings? How can I resolve this? Thank you, Matthias

I tried the ScreenCaptureKit sample code from Apple: ScreenCaptureKit Sample Code When I ran it for a while, it crash at the strange position as attached screenshot. The value array is not empty and has value at index 0 but it crashed.

What framework to use to capture screen of a device connected to the Mac in the way OBS or QuickTime Player does when an iOS device is connected to Mac via USB. I tried to list devices with AVFoundation and ScreenCaptureKit but only Mac camera, mic and displays are listed. When you select New Movie Recording in the QuickTime Player you can choose an Connected iPad or iPhone to record it's screan. Same with OBS. What is the way to do it in my own MacOS app?