Is it possible to detect the Window that is making an app?

I am trying to use ScreenCaptureKit to select this window automatically rather than having the user do it.

Answered by DTS Engineer in 819708022

This is just something I'm noticing, but it appears Chrome will append a sound icon to the title of its window when it makes sound

Sure. Safari does something similar and I'm sure other apps do as well. The issue here is the difference between what an app "knows" about it's on activity and what the system "knows". Taking a web browser as an example, "generating audio" is basically just another data rendering task (download data, process data, "render" data into final form) that's not all that different than any other kinds of data it process. The final output is something you hear instead of seeing, but the broad mechanics are the same.

A web browser "knows" that audio is associated with a particular view/window because that's an associate it was "already" managing as part of the broader task of tying "all" of it's activity to a particular source. In that context, appending the sounds icon is a simple interface choice just like displaying the favicon is.

The system's view of this process is totally different. All it really "sees" is the connection(s) a process creates "into" it and the memory buffers passed into it over that connection. None of that activity is has any connection to the window system because that connection simply does not exist and never has as the system simply does not work that way.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Making a sound*, not making an app.

Accepted Answer

Is it possible to detect the Window that is making a sound?

The short answer is no, not really. First off, from the system perspective, processes play audio, not windows. Obviously many apps do present a connection between a window and a given audio stream, but that's something the app is imposing on it's own content not something the system was actually "aware" of.

Making that concrete, Safari will let you mute/unmute individual windows but what Safari is actually doing is "manually" muting the individual streams* it's feeding into the system. The system has no idea that there's ANY connection between those streams and specific windows.

*Assuming it's even giving the system multiple streams. It's an app can also be doing it's own mixing, then passing a single audio stream "out" to the system.

So, changing the question "is it possible to detect which processes are producing audio" the answer is yes... but not really. More specifically:

  1. CoreAudio's "kAudioDevicePropertyHogMode" will let you detect what (if any) process has taken exclusive control of the audio interface. That's straightforward to use, however, I don't think it'll be all that useful. Seizing the interface like this is relatively rare and the kind of apps that do it (basically "games" and some media players) aren't the kind I'd typically think of trying to capture.

  2. If you look at the broader app ecosystem, you'll find that there are apps that clearly "know" which apps are playing audio, as they're able to do things like alter the volume or mute individual processes. These apps actually work by installing a system wide audio driver which lets them act as the "target" output device that other apps are then sending audio to. They could easily be bypassed either by the user (who could change to a different output device) or individual app (if the app chooses to take control of it's own routing), but in practice that's not really a problem since the user "wants" to be using them and mosts apps just use the default route.

The second approach is certainly workable (as many apps obviously do this), but the work involved is... quite significant. Most of the apps that do this sort of thing are in fact "dedicated audio apps" as the work involved in making something like this "from scratch" basically requires that kind of effort and expertise.

That leads to the other option here, which would be to use someone else's code/library. Some commercial developers also license their routing engines and there are some GPL projects as well.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

This is just something I'm noticing, but it appears Chrome will append a sound icon to the title of its window when it makes sound

This is just something I'm noticing, but it appears Chrome will append a sound icon to the title of its window when it makes sound

Sure. Safari does something similar and I'm sure other apps do as well. The issue here is the difference between what an app "knows" about it's on activity and what the system "knows". Taking a web browser as an example, "generating audio" is basically just another data rendering task (download data, process data, "render" data into final form) that's not all that different than any other kinds of data it process. The final output is something you hear instead of seeing, but the broad mechanics are the same.

A web browser "knows" that audio is associated with a particular view/window because that's an associate it was "already" managing as part of the broader task of tying "all" of it's activity to a particular source. In that context, appending the sounds icon is a simple interface choice just like displaying the favicon is.

The system's view of this process is totally different. All it really "sees" is the connection(s) a process creates "into" it and the memory buffers passed into it over that connection. None of that activity is has any connection to the window system because that connection simply does not exist and never has as the system simply does not work that way.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Is it possible to detect the Window that is making an app?
 
 
Q