I have an app that records a 32 x 32 rect under the cursor as the user moves it around and it sends it to Flutter. It suffers from major lag. Instead of getting 30 fps, I get about 7 fps. That is, there are significant lags between screen grabs. This on an Intel Mac mini x64 with 15.7.3 and one display. flutter: NATIVE: ExplodedView framesIn=2 timeSinceStart=1115.7ms gapSinceLastFrame=838.8ms flutter: NATIVE: ExplodedView framesIn=4 timeSinceStart=1382.6ms gapSinceLastFrame=149.9ms flutter: NATIVE: ExplodedView framesIn=5 timeSinceStart=1511.0ms gapSinceLastFrame=128.4ms flutter: NATIVE: ExplodedView framesIn=7 timeSinceStart=1698.3ms gapSinceLastFrame=102.9ms flutter: NATIVE: ExplodedView STOP polling totalTime=4482.6ms framesIn=28 framesSent=28 acks=28 Here's a testable excerpt: import ScreenCaptureKit import CoreMedia import CoreVideo import QuartzCore final class Test: NSObject, SCStreamOutput, SCStreamDelegate { private let q = DispatchQueue(label: "cap.q") private var stream: SCStream? private var lastFrameAt: CFTimeInterval = 0 private var frames = 0 func start() { SCShareableContent.getExcludingDesktopWindows(false, onScreenWindowsOnly: true) { content, err in guard err == nil, let display = content?.displays.first else { print("shareableContent error: \(String(describing: err))"); return } let filter = SCContentFilter(display: display, excludingWindows: []) let config = SCStreamConfiguration() config.showsCursor = false config.queueDepth = 1 config.minimumFrameInterval = CMTime(value: 1, timescale: 30) config.pixelFormat = kCVPixelFormatType_32BGRA config.width = 32 config.height = 32 config.sourceRect = CGRect(x: 100, y: 100, width: 32, height: 32) let s = SCStream(filter: filter, configuration: config, delegate: self) try! s.addStreamOutput(self, type: .screen, sampleHandlerQueue: self.q) self.stream = s s.startCapture { startErr in print("startCapture err=\(String(describing: startErr))") } // Optional: move sourceRect at 30Hz (cursor-follow simulation) Timer.scheduledTimer(withTimeInterval: 1.0/30.0, repeats: true) { _ in let c2 = SCStreamConfiguration() c2.showsCursor = false c2.queueDepth = 1 c2.minimumFrameInterval = CMTime(value: 1, timescale: 30) c2.pixelFormat = kCVPixelFormatType_32BGRA c2.width = 32 c2.height = 32 let t = CACurrentMediaTime() c2.sourceRect = CGRect(x: 100 + (sin(t) * 50), y: 100, width: 32, height: 32) s.updateConfiguration(c2) { _ in } } } } func stream(_ stream: SCStream, didOutputSampleBuffer sb: CMSampleBuffer, of type: SCStreamOutputType) { guard type == .screen else { return } let now = CACurrentMediaTime() let gapMs = (lastFrameAt == 0) ? 0 : (now - lastFrameAt) * 1000 lastFrameAt = now frames += 1 if frames <= 10 || frames % 60 == 0 { print("frames=\(frames) gapMs=\(String(format: "%.1f", gapMs))") } } }

We're troubleshooting SCK issues. They occur with a relatively small amount of sessions, but lack of context and/or ability to advise the customer on how they could make behavior more predictable and reliable is problematic. Generally, there is 2 distinct issues which may or may not have the same root cause: Failure to establish SCK session. Usually manifests within the app as SCShareableContent.getWithCompletionHandler call either never invoking the completion handler, or taking prohibitively long time (we usually give it 3-10 sec before giving up). In the system log it may look like this: (log omitted - suspecting it triggers the content filter) Note the 6 seconds delay to completion of fetchShareableContentWithOption (normally it's a 30-40ms operation). Sometime, we'd see the stream established, but some minutes (or even hours) into the recording we'd stop receiving frames. Both scenarios are likely to occur when the disk space is low, with reliable repro of the problem #2 at below 8gb of free space (in that case, we've seen replayd silently dropping the session, without ever notifying the client ... improving API could go a long way there). However, out of recent occurrences, while most have less than 100GB available, we've seen it on machines with as much as 500GB free. Unfortunately, it's almost never reproducible in dev environment, so we have to rely on diagnostics we're able to collect in the field -- which nothing obvious yet. I'd like to understand the root cause of both scenarios better and/or how what specific frameworks can cause these behaviors.

I am developing a remote support tool for macOS. While we have successfully implemented a Privileged Helper Tool and LaunchDaemon architecture that works within an active Aqua session, we have observed a total failure to capture the screen buffer or receive input at the macOS Login Window. Our observation of competitor software (AnyDesk, TeamViewer) shows they maintain graphical continuity through logout/restart. We are seeking the official architectural path to replicate this system-level access. Current Technical Implementation Architecture: A root-level LaunchDaemon manages the persistent network connection. A PrivilegedHelperTool (installed in /Library/PrivilegedHelperTools/) is used for elevated tasks. Environment: Tested on macOS 14.x (Sonoma) and macOS 15.x (Sequoia) on Apple Silicon. Capture Methods: We have implemented ScreenCaptureKit (SCK) as the primary engine and CGDisplayCreateImage as a fallback. Binary Status: All components are signed with a Developer ID and have been successfully Notarized. Observed Behavior & Blockers The "Aqua" Success: Within a logged-in user session, our CGI correctly identifies Display IDs and initializes the capture stream. Remote control is fully functional. The "Pre-Login" Failure: When the Mac is at the Login Window (no user logged in), the following occurs: The Daemon remains active, but the screen capture buffer returns NULL or an empty frame. ScreenCaptureKit fails to initialize, citing a lack of graphical context. No TCC (Transparency, Consent, and Control) prompt can appear because no user session exists. The "Bootstrap" Observation: We have identified that the loginwindow process exists in a restricted Mach bootstrap namespace that our Daemon (running in the System domain) cannot natively bridge. Comparative Analysis (Competitor Benchmarking) We have analyzed established remote desktop solutions like AnyDesk and Jump Desktop to understand their success at the login screen. Our findings suggest: Dual-Context Execution: They appear to use a Global LaunchAgent with LimitLoadToSessionType = ["LoginWindow"]. This allows a child process to run as root inside the login window’s graphical domain. Specialized Entitlements: These apps have migrated to the com.apple.developer.persistent-content-capture entitlement. This restricted capability allows them to bypass the weekly/monthly TCC re-authorization prompts and function in unattended scenarios where a user cannot click "Allow." Questions Entitlement Requirement: Is the persistent-content-capture entitlement the only supported way for a third-party app to capture the LoginWindow buffer without manual user intervention? LaunchAgent Strategy: To gain a graphical context at the login screen, is it recommended to load a specialized agent into the loginwindow domain via launchctl bootstrap loginwindow ...? ScreenCaptureKit vs. Legacy: Does ScreenCaptureKit officially support the LoginWindow session, or does it require an active Aqua session to initialize? MDM Bypass: For Enterprise environments, can a Privacy Preferences Policy Control (PPPC) payload grant "Screen Recording" to a non-entitled Daemon specifically for the login window context?

With the help of Claude and Codex, I've tried upgrading a screen magnification, capture, and pixel peeping app (SnoopX, from Stanford's software repository) to be HDR-aware. For images that are truly HDR (displayed in Apple Photos, Google Photos in Chrome, or Mac Preview), and on an HDR monitor with plenty of EDR headroom (Apple XDR display), I should see pixel values well above 1.0. Apple's Digital Color Meter does. However, my app does not. I think the code is doing all the right things with ScreenCaptureKit, but it never returns values above 1.0. Has anybody gotten this to work? Here is what Codex says about the code it helped me build in my app: • Set captureDynamicRange: We set config​.capture​Dynamic​Range = ​SCCapture​Dynamic​Range​HDRLocal​Display (when available). Note: Gemini’s SCCapture​Mode​HDRLocal​Display isn’t a real symbol — the actual API is SCCapture​Dynamic​Range. • Use preset: We try init​With​Preset: ​SCStream​Configuration​Preset​Capture​HDRStream​Local​Display, but your runtime crashes on init​With​Preset: (unrecognized selector), so we fall back to manual config. That means your system SDK headers mention the preset, but the runtime doesn’t implement it. • Pixel format: We explicitly set k​CVPixel​Format​Type​_64​RGBAHalf. Your logs show the stream buffer is 64RGBAHalf. • Enable HDR on display: EDR headroom logs show current=6.40, so HDR is active. • Color space: We set Extended Linear Display P3 for stream, and we forced the screenshot sampler to convert into extended‑linear P3 before reading. The screenshot itself reports Extended sRGB, but conversion doesn’t yield >1.0 either. Despite all of that, the OS is still delivering tone‑mapped values (max 1.0000) for both the stream and the screenshot path. So I don’t think there’s a missing knob here; it looks like a system‑level limitation on Tahoe for desktop capture.

Two months ago I requested the subject entitlement. I'm still waiting for it to be added to our account. Who or how can I find out what going on with it. I have no correspondence from Apple yet saying it was denied and why. https://developer.apple.com/documentation/bundleresources/entitlements/com.apple.developer.persistent-content-capture?language=objc Thank you.

On Tahoe and earlier, ScreenCapture permissions can disappear and not return. Customers are having an issue with this disappearing and when our code executes CGRequestScreenCaptureAccess() nothing happens, the prompt does not appear. I can reproduce this by using the "-" button and removing the entry in the settings, then adding it back with the "+" button. CGPreflightScreenCaptureAccess() always returns the correct value but once the entry has been removed, CGRequestScreenCaptureAccess() requires a reboot before it will work again.

Sharing a solution for a problem that took me a while to figure out. Problem: During development of a macOS 26 app that uses ScreenCaptureKit, the screen capture permissions were being reset after every build. Each time I compiled and ran the app from Xcode, I had to re-authorize screen capture in System Settings. CGPreflightScreenCaptureAccess() would return false even though I'd just granted permission minutes ago. Root cause: I was using ad-hoc code signing during development. macOS ties screen capture permissions to the app's code signing identity. With ad-hoc signing, the identity changes on every build, so the system treats each build as a "new" app. Solution: Switch to an Apple Development certificate for debug builds. In Xcode: Build Settings → Code Signing Identity → Debug → set to "Apple Development" Make sure your development team is selected After this change, the signing identity remains stable across builds, and screen capture permissions persist. This might be related to the broader issue discussed in this forum about ScreenCapture permissions disappearing — if other developers are seeing permissions vanish, it's worth checking whether the code signing identity is changing between sessions.

We built an open-source macOS menu bar app that turns speech into text and pastes it into the active app — using SpeechAnalyzer for on-device transcription, ScreenCaptureKit + Vision for screen-aware context, and FluidAudio for speaker diarization in meeting mode. Here's what we learned shipping it on macOS 26. GitHub: github.com/Marvinngg/ambient-voice Architecture The app has two modes: hotkey dictation (press to talk, release to inject) and meeting recording (continuous transcription with a floating panel). Dictation Mode Audio capture uses AVCaptureSession (more on why below). The captured audio feeds into SpeechAnalyzer via an AsyncStream: let transcriber = SpeechTranscriber( locale: locale, transcriptionOptions: [], reportingOptions: [.volatileResults, .alternativeTranscriptions], attributeOptions: [.audioTimeRange, .transcriptionConfidence] ) let analyzer = SpeechAnalyzer(modules: [transcriber]) let (inputSequence, inputBuilder) = AsyncStream.makeStream() try await analyzer.start(inputSequence: inputSequence) While recording, we capture a screenshot of the focused window using ScreenCaptureKit, run Vision OCR (VNRecognizeTextRequest), extract keywords, and inject them into SpeechAnalyzer as contextual bias: let context = AnalysisContext() context.contextualStrings[.general] = ocrKeywords try await analyzer.setContext(context) This improves accuracy for technical terms and proper nouns visible on screen. If your screen shows "SpeechAnalyzer", saying it out loud is more likely to be transcribed correctly. After transcription, an optional L2 step sends the text through a local LLM (ollama) for spoken-to-written cleanup, then CGEvent simulates Cmd+V to paste into the active app. Meeting Mode Meeting mode forks the same audio stream to two consumers: SpeechAnalyzer — real-time streaming transcription, displayed in a floating NSPanel FluidAudio buffer — accumulates 16kHz Float32 mono samples for batch speaker diarization after recording stops When the user ends the meeting, FluidAudio's performCompleteDiarization() runs on the accumulated audio. We align transcription segments with speaker segments using audioTimeRange overlap matching — each transcription segment gets assigned the speaker ID with the most time overlap. Results export to Markdown. Pitfalls We Hit on macOS 26 1. AVAudioEngine installTap doesn't fire with Bluetooth devices We started with AVAudioEngine.inputNode.installTap() for audio capture. It worked fine with built-in mics but the tap callback never fired with Bluetooth devices (tested with vivo TWS 4 Hi-Fi). Fix: switched to AVCaptureSession. The delegate callback captureOutput(_:didOutput:from:) fires reliably regardless of audio device. The tradeoff is you get CMSampleBuffer instead of AVAudioPCMBuffer, so you need a conversion step. 2. NSEvent addGlobalMonitorForEvents crashes Our global hotkey listener used NSEvent.addGlobalMonitorForEvents. On macOS 26, this crashes with a Bus error inside GlobalObserverHandler — appears to be a Swift actor runtime issue. Fix: switched to CGEventTap. Works reliably, but the callback runs on a CFRunLoop context, which Swift doesn't recognize as MainActor. 3. CGEventTap callbacks aren't on MainActor If your CGEventTap callback touches any @MainActor state, you'll get concurrency violations. The callback runs on whatever thread owns the CFRunLoop. Fix: bridge with DispatchQueue.main.async {} inside the tap callback before touching any MainActor state. 4. CGPreflightScreenCaptureAccess doesn't request permission We used CGPreflightScreenCaptureAccess() as a guard before calling ScreenCaptureKit. If it returned false, we'd bail out. The problem: this function only checks — it never triggers macOS to add your app to the Screen Recording permission list. Chicken-and-egg: you can't get permission because you never ask for it. Fix: call CGRequestScreenCaptureAccess() at app startup. This adds your app to System Settings → Screen Recording. Then let ScreenCaptureKit calls proceed without the preflight guard — SCShareableContent will also trigger the permission prompt on first use. 5. Ad-hoc signing breaks TCC permissions on every rebuild During development, codesign --sign - (ad-hoc) generates a different code directory hash on every build. macOS TCC tracks permissions by this hash, so every rebuild = new app identity = all permissions reset. Fix: sign with a stable certificate. If you have an Apple Development certificate, use that. The TeamIdentifier stays constant across rebuilds, so TCC permissions persist. We also discovered that launching via open WE.app (LaunchServices) instead of directly executing the binary is required — otherwise macOS attributes TCC permissions to Terminal, not your app. Benchmarks We ran end-to-end benchmarks on public datasets (Mac Mini M4 16GB, macOS 26): Transcription (SpeechAnalyzer, AliMeeting Chinese): • Near-field CER 34% (excluding outliers ~25%) • Far-field CER 40% (single channel, no beamforming, >30% overlap) • Processing speed 74-89x real-time Speaker diarization (FluidAudio offline): • AMI English 16 meetings: avg DER 23.2% (collar=0.25s, ignoreOverlap=True) • AliMeeting Chinese 8 meetings: DER 48.5% (including overlap regions) • Memory: RSS ~500MB, peak 730-930MB Full evaluation methodology, scripts, and raw results are in the repo. Open Source The project is MIT licensed: github.com/Marvinngg/ambient-voice It includes the macOS client (Swift 6.2, SPM), server-side distillation/training scripts (Python), and a complete evaluation framework with reproducible benchmarks. Feedback and contributions welcome.

We built an open-source macOS menu bar app that turns speech into text and pastes it into the active app — using SpeechAnalyzer for on-device transcription, ScreenCaptureKit + Vision for screen-aware context, and FluidAudio for speaker diarization in meeting mode. Here's what we learned shipping it on macOS 26. GitHub: github.com/Marvinngg/ambient-voice Architecture The app has two modes: hotkey dictation (press to talk, release to inject) and meeting recording (continuous transcription with a floating panel). Dictation Mode Audio capture uses AVCaptureSession (more on why below). The captured audio feeds into SpeechAnalyzer via an AsyncStream: let transcriber = SpeechTranscriber( locale: locale, transcriptionOptions: [], reportingOptions: [.volatileResults, .alternativeTranscriptions], attributeOptions: [.audioTimeRange, .transcriptionConfidence] ) let analyzer = SpeechAnalyzer(modules: [transcriber]) let (inputSequence, inputBuilder) = AsyncStream.makeStream() try await analyzer.start(inputSequence: inputSequence) While recording, we capture a screenshot of the focused window using ScreenCaptureKit, run Vision OCR (VNRecognizeTextRequest), extract keywords, and inject them into SpeechAnalyzer as contextual bias: let context = AnalysisContext() context.contextualStrings[.general] = ocrKeywords try await analyzer.setContext(context) This improves accuracy for technical terms and proper nouns visible on screen. If your screen shows "SpeechAnalyzer", saying it out loud is more likely to be transcribed correctly. After transcription, an optional L2 step sends the text through a local LLM (ollama) for spoken-to-written cleanup, then CGEvent simulates Cmd+V to paste into the active app. Meeting Mode Meeting mode forks the same audio stream to two consumers: SpeechAnalyzer — real-time streaming transcription, displayed in a floating NSPanel FluidAudio buffer — accumulates 16kHz Float32 mono samples for batch speaker diarization after recording stops When the user ends the meeting, FluidAudio's performCompleteDiarization() runs on the accumulated audio. We align transcription segments with speaker segments using audioTimeRange overlap matching — each transcription segment gets assigned the speaker ID with the most time overlap. Results export to Markdown. Pitfalls We Hit on macOS 26 1. AVAudioEngine installTap doesn't fire with Bluetooth devices We started with AVAudioEngine.inputNode.installTap() for audio capture. It worked fine with built-in mics but the tap callback never fired with Bluetooth devices (tested with vivo TWS 4 Hi-Fi). Fix: switched to AVCaptureSession. The delegate callback captureOutput(_:didOutput:from:) fires reliably regardless of audio device. The tradeoff is you get CMSampleBuffer instead of AVAudioPCMBuffer, so you need a conversion step. 2. NSEvent addGlobalMonitorForEvents crashes Our global hotkey listener used NSEvent.addGlobalMonitorForEvents. On macOS 26, this crashes with a Bus error inside GlobalObserverHandler — appears to be a Swift actor runtime issue. Fix: switched to CGEventTap. Works reliably, but the callback runs on a CFRunLoop context, which Swift doesn't recognize as MainActor. 3. CGEventTap callbacks aren't on MainActor If your CGEventTap callback touches any @MainActor state, you'll get concurrency violations. The callback runs on whatever thread owns the CFRunLoop. Fix: bridge with DispatchQueue.main.async {} inside the tap callback before touching any MainActor state. 4. CGPreflightScreenCaptureAccess doesn't request permission We used CGPreflightScreenCaptureAccess() as a guard before calling ScreenCaptureKit. If it returned false, we'd bail out. The problem: this function only checks — it never triggers macOS to add your app to the Screen Recording permission list. Chicken-and-egg: you can't get permission because you never ask for it. Fix: call CGRequestScreenCaptureAccess() at app startup. This adds your app to System Settings → Screen Recording. Then let ScreenCaptureKit calls proceed without the preflight guard — SCShareableContent will also trigger the permission prompt on first use. 5. Ad-hoc signing breaks TCC permissions on every rebuild During development, codesign --sign - (ad-hoc) generates a different code directory hash on every build. macOS TCC tracks permissions by this hash, so every rebuild = new app identity = all permissions reset. Fix: sign with a stable certificate. If you have an Apple Development certificate, use that. The TeamIdentifier stays constant across rebuilds, so TCC permissions persist. We also discovered that launching via open WE.app (LaunchServices) instead of directly executing the binary is required — otherwise macOS attributes TCC permissions to Terminal, not your app. Benchmarks We ran end-to-end benchmarks on public datasets (Mac Mini M4 16GB, macOS 26): Transcription (SpeechAnalyzer, AliMeeting Chinese): • Near-field CER 34% (excluding outliers ~25%) • Far-field CER 40% (single channel, no beamforming, >30% overlap) • Processing speed 74-89x real-time Speaker diarization (FluidAudio offline): • AMI English 16 meetings: avg DER 23.2% (collar=0.25s, ignoreOverlap=True) • AliMeeting Chinese 8 meetings: DER 48.5% (including overlap regions) • Memory: RSS ~500MB, peak 730-930MB Full evaluation methodology, scripts, and raw results are in the repo. Open Source The project is MIT licensed: github.com/Marvinngg/ambient-voice It includes the macOS client (Swift 6.2, SPM), server-side distillation/training scripts (Python), and a complete evaluation framework with reproducible benchmarks. Feedback and contributions welcome.

Precondition: In system settings, scale the pointer size up to the max. Our SCScreenshotManager code currently works in macOS 15 and earlier to capture the cursor at it's larger size, but broke in one of the minor releases of macOS Tahoe. The error it produces now is "Failed to start stream due to audio/video capture failure". This only seems to happen with the cursor window, not any others. Another way to get the cursor is with https://developer.apple.com/documentation/appkit/nscursor/currentsystem, but that is now deprecated, which makes me think the capture of the cursor is being blocked deliberately. We see this as a critical loss of functionality for our apps, and could use guidance on what to use instead.

Hello Apple Developer Support, I am reaching out for technical guidance regarding screen capture protection behavior on macOS. We are building a desktop application using Flutter running on macOS, and we have implemented native Swift code inside the macOS Runner in order to protect sensitive content from screen recording and screen sharing. Our current implementation relies on native window-level protection and display state handling from Swift, while the main UI remains rendered by Flutter. The main challenge we are facing is the following: we need to keep a strong native anti-recording protection on macOS the application is heavily used with AirPlay and screen mirroring currently, AirPlay / mirroring is often interpreted by the system similarly to screen capture or screen recording this causes our protected content to be replaced by a gray or blank area even during legitimate AirPlay usage In practice, we would like to allow: AirPlay legitimate external display / mirroring usage while still preventing: screen recording screen sharing unauthorized screen capture We would like to know whether Apple recommends an official supported approach for this use case, preferably using public APIs. More specifically: Is there an officially supported way on macOS to distinguish AirPlay mirroring from screen recording / screen sharing? Is "NSWindow.sharingType" the recommended public API for this scenario? Is there a recommended approach when the UI surface is rendered through Flutter / Metal? Are there any best practices with ScreenCaptureKit for protecting content without affecting AirPlay? We understand that some lower-level APIs may not be officially supported, so we would greatly appreciate guidance toward a public and future-proof implementation path. Thank you very much for your time and support. Best regards, Tony

When using the built-in macOS screen recording feature, the recording stops automatically after approximately 10–15 minutes without any warning or error message. No manual stop action is performed. The recording simply ends silently. The same issue also occurs when using ScreenCaptureKit in a custom application, which suggests this may be a system-level issue related to screen capture rather than an app-specific problem. This issue is reproducible and happens consistently after running for a period of time.

When using the built-in macOS screen recording feature, the recording stops automatically after approximately 10–15 minutes without any warning or error message. No manual stop action is performed. The recording simply ends silently. The same issue also occurs when using ScreenCaptureKit in a custom application, which suggests this may be a system-level issue related to screen capture rather than an app-specific problem. This issue is reproducible and happens consistently after running for a period of time.

I am using SCScreenshotManager.captureImage to capture image of an app window. i have the sleep settings that sleeps the display but not the system. when the display sleeps taking screenshot using SCScreenshotManager.captureImage does not work. Is there a way i can take screenshot of window when the display sleeps? Thanks