Is Braille Screen Input not passed to keyboard extensions?

Question

Created Jun ’20

Replies 2

Boosts 0

Participants 2

Braille Screen Input does not appear to be passed to third party keyboard extensions, so they are unable to use such input even when they're the active keyboard. Visually impaired users can't use the features of our keyboard, which seems to be a significant accessibility shortcoming.

Could I be missing something, or is there no way for a keyboard extension to detect and respond to Braille Screen Input?

To see what I mean (FB7741375):

On an iPad (we used an iPad Pro 11” 2018 model) running iOS 13.5 and an English (US) soft keyboard
Install TextExpander from the App Store

https://apps.apple.com/us/app/textexpander-keyboard/id1075927186

3. Launch TextExpander
4. Sign in to TextExpander, and ensure you have an abbreviation 'txp' which expands to 'TextExpander'
5. Add TextExpander English keyboard:

textexpander.com/kb/ios/how-to-textexpander-custom-keyboard/

6. Launch the Notes app
7. Select the “English – TextExpander” keyboard
8. Type: txp

Expected: txp becomes TextExpander
Actual: txp becomes TextExpander

9. Start a new line (so that you're not typing immediately following TextExpander)
10. Engage Braille Screen Input (via the Rotor)

Hold 2,3,4,5 then release to type lowercase t
Hold 1,3,4,6 then release to type lowercase x
Hold 1,2,3,4 then release to type lowercase p

Expected: txp becomes TextExpander
Actual: txp remains txp

Nothing gets called in our keyboard extension. There are no touch events reported (which is understandable given that Braille Screen Input is basically a "curtain" designed not to interfere with any underlying apps or extensions). Our UIInputViewController subclass' UITextInputDelegate methods aren't called either. This is less understandable, as it seems that textWilllChange: and textDidChange: could be called.

Could I be missing something, or is there no way for a keyboard extension to detect and respond to Braille Screen Input?

Commentary:
If there's no way to detect input made via Braille Screen Input when the TextExpander keyboard is in use, there's no way to support users of Braille Screen Input who wish to type abbreviations TextExpander would normally expand.

Boost

Answer 1

Apple Designer OP

Apple

Jun ’20

BSI is like a keyboard itself. It manipulates the content of the text field rather than inputting text through another input mechanism. This lets us enter contracted braille for example which requires us to do our own transforms.

Architecturally, we also don't have a way to pipe the text through a keyboard plugin that takes touch input as its input mechanism.

So unfortunately at this time there's no way to make these work together without new API contracts I suspect.

0

Answer 2

macgreg OP

Jun ’20

What is the best way to go about attending Labs and/or filing additional Feedback requests to try to move this forward?

In researching this, I've learned that visually impaired users also make use of overlays on hardware keyboards for their iPads. Keyboard extensions also do not have access to hardware keyboard input. I had not realized lack of access to that input is an accessibility issue for a portion of our users.

I understand that presently, this is not possible. What I'm asking is: what can I learn or do during WWDC week to help lobby on behalf of our visually impaired users to get improvements considered and/or made in due course in this area?

Thanks!

0