No prompt showing for WKWebView getUserMedia({audio: true})

A JavaScript library implementation (Twilio Voice JS) on MacOS requires media permissions, specifically microphone permissions.

To do so in web browsers, one typical calls:

navigator.mediaDevices.getUserMedia({audio: true})
    .then((e) => alert("Permissions granted"))
    .catch((e) => alert("Permissions Denied"))

The same could be done in a WKWebView either by evaluateJavaScript or by loading a script (or html file) from local resources. In these cases navigator.mediaDevices would only be made available if called from a "secure" site (i.e. https) or loaded locally e.g. via an html file from bundle resources. See secure note

The issue is quite simple:

Calling getUserMedia via Safari WKWebView inspector console hangs in terminal. It doesn't complete. It doesn't show a prompt. The promise doesn't resolve, reject or throw an error, nothing.

Context

  • Environment: Flutter MacOS plugin, XCode 14.2. Targeting macOS +11.0 with a Headless WKWebView.
  • Info.plist includes Microphone permissions as required
  • Hardened Runtime includes Audio Input, Sandbox includes bluetooth, Audio Input & outgoing connections.

  • Manually prompting permissions documented here shows prompt.
  • (System) Preferences > Security & Privacy > Microphone lists app with permissions.
  • Safari (console) > Preferences > Websites > Microphone does not show any allowed permissions, with Ask required

Media & supporting docs

Current output

Current output for enumeratedDevices and getUserMedia in WKWebView:

Expected output

Safari output, showing enumeratedDevices before and after requesting permissions:

The issue is quite simple: calling getUserMedia via Safari WKWebView inspector console hangs in terminal. It doesn't complete. It doesn't show a prompt. The promise doesn't resolve, reject or throw an error, nothing.

Question

What is causing this hanging/what am I missing?

Up vote post of BearMan Down vote post of BearMan
813 views
Posted by
BearMan
Reply
Add a Comment

Replies

UPDATE 1

I came across this link: https://stackoverflow.com/questions/66048995/avaudioengine-halc-shellobject-errors-no-audio-what-can-i-even-look-for

enumerateDevices shows:

0: InputDeviceInfo {deviceId: 'default', kind: 'audioinput', label: 'Default - Internal Microphone (Built-in)', groupId: '...'}
1: InputDeviceInfo {deviceId: '...', kind: 'audioinput', label: 'Internal Microphone (Built-in)', groupId: '...'}
2: InputDeviceInfo {deviceId: '...', kind: 'audioinput', label: 'Microsoft Teams Audio Device (Virtual)', groupId: '...'}
3: InputDeviceInfo {deviceId: '', kind: 'videoinput', label: '', groupId: ''}
4: MediaDeviceInfo {deviceId: 'default', kind: 'audiooutput', label: 'Default - Internal Speakers (Built-in)', groupId: '...'}
5: MediaDeviceInfo {deviceId: '...', kind: 'audiooutput', label: 'Internal Speakers (Built-in)', groupId: '...'}
6: MediaDeviceInfo {deviceId: '...', kind: 'audiooutput', label: 'Microsoft Teams Audio Device (Virtual)', groupId: '...'}

Hardened Runtime Permissions:

  • Audio Input
  • Camera

Below is the output for both with and without the above permissions:

WITH

getDevices: nil // JS response

2023-07-25 14:32:19.758054+0200 WKWebView[60215:1070748] [plugin] AddInstanceForFactory: No factory registered for id <CFUUID 0x600000266820> F8BB1C28-BAE8-11D6-9C31-00039315CD46
2023-07-25 14:32:19.786676+0200 WKWebView[60215:1070748]   saved enable noise cancellation setting is the same as the default (=1)
2023-07-25 14:32:19.833763+0200 WKWebView[60215:1070748] [plugin] AddInstanceForFactory: No factory registered for id <CFUUID 0x600000271760> 30010C1C-93BF-11D8-8B5B-000A95AF9C6A
2023-07-25 14:32:19.855809+0200 WKWebView[60215:1070748] [plugin] AddInstanceForFactory: No factory registered for id <CFUUID 0x600000271760> 30010C1C-93BF-11D8-8B5B-000A95AF9C6A
2023-07-25 14:32:19.862903+0200 WKWebView[60215:1070703]  HALC_ShellObject::GetPropertyData: call to the proxy failed, Error: 1852797029 (nope)
2023-07-25 14:32:19.862932+0200 WKWebView[60215:1070703]  HALPlugIn::ObjectGetPropertyData: got an error from the plug-in routine, Error: 1852797029 (nope)
2023-07-25 14:32:19.864197+0200 WKWebView[60215:1070703]  HALC_ShellObject::GetPropertyData: call to the proxy failed, Error: 1852797029 (nope)
2023-07-25 14:32:19.864239+0200 WKWebView[60215:1070703]  HALPlugIn::ObjectGetPropertyData: got an error from the plug-in routine, Error: 1852797029 (nope)
2023-07-25 14:32:19.865446+0200 WKWebView[60215:1070703]  HALC_ShellObject::GetPropertyData: call to the proxy failed, Error: 1852797029 (nope)
2023-07-25 14:32:19.865476+0200 WKWebView[60215:1070703]  HALPlugIn::ObjectGetPropertyData: got an error from the plug-in routine, Error: 1852797029 (nope)
2023-07-25 14:32:19.867520+0200 WKWebView[60215:1070703]  HALC_ShellObject::GetPropertyData: call to the proxy failed, Error: 1852797029 (nope)
2023-07-25 14:32:19.867541+0200 WKWebView[60215:1070703]  HALPlugIn::ObjectGetPropertyData: got an error from the plug-in routine, Error: 1852797029 (nope)

and,

WITHOUT

navigator.mediaDevices is undefined (as expected by Mozilla's documentation on secure environments & mediaDevices), resulting in the following output:

getDevices error: Optional(Error Domain=WKErrorDomain Code=4 "A JavaScript exception occurred" UserInfo={WKJavaScriptExceptionLineNumber=14, WKJavaScriptExceptionMessage=TypeError: undefined is not an object (evaluating 'navigator.mediaDevices.enumerateDevices'), WKJavaScriptExceptionColumnNumber=39, WKJavaScriptExceptionSourceURL=file:///Users/mac/Library/Caches/JetBrains/AppCode2023.1/DerivedData/WKWebView-gziqxifpucvqvngtmvedcjalfcqj/Build/Products/Debug/WKWebView.app/Contents/Resources/index.html, NSLocalizedDescription=A JavaScript exception occurred})
Posted by
BearMan
Up vote reply of BearMan Down vote reply of BearMan
Add a Comment