CGFlag bits gets reset when there is concurrent manual mouse movement

Hello,

There is a problem when querying flag state using the following API CGEventSourceFlagsState(kCGEventSourceStateHIDSystemState). When there is external mouse movement the flag gets reset by a low OS-level event corresponding to mouse move. As a result the modifier keys don’t work as expected when using Java’s Robot which in-turn uses Apple's native CGEvent APIs.

The issue occurs at CRobot.m#L295 the flags gets reset or cleared when mouse is moved physically in unison with Robot's key events. The difference can be seen in the logs without and with mouse move for typing 'A'. (log attached)

Logs

Due to this issue applications that use Java's Robot on Mac don’t work as expected, in particular this behavior breaks the usability of the on-screen accessibility related keyboard application - TouchBoard. More details of this use case here. https://github.com/adoptium/adoptium-support/issues/710

The Robot is initialized with the following initial configurations - https://github.com/openjdk/jdk/blob/ac6af6a64099c182e982a0a718bc1b780cef616e/src/java.desktop/macosx/native/libawt_lwawt/awt/CRobot.m#L125

  • Are we missing anything during initialization of Robot which cause this issue?
  • Why does an external mouse movement cause the event flags to reset?
  • Since a low level OS event corresponding to mouse move is causing the flag to reset, there might be an issue within the CGEventSourceFlagsState() API.
  • Is there a reason behind why an external mouse event causes CGEventFlag state to reset to 0 ?
  • Is there any known issue regarding CGEventSourceFlagsState() and a workaround for it?
Up vote post of harsh11 Down vote post of harsh11
513 views
Posted by
harsh11
Reply
Add a Comment

Replies

I have added steps and reproducer test for verification.

Environment:

  • macOS 12, 13. Not tested on older versions
  • JDK version 11 & above

Steps to reproduce:

  • Open any text editor. Run the example program after installing JDK.
  • After you start the program, you have seven seconds to place the text cursor in the text editor. Then the program will type characters for about 15 seconds and end.
  • When the letters are being typed, run the test for the following 2 cases separately -
    • Case 1: Don't move the mouse when the letters are being typed - Uppercase 'A' is produced as expected.
    • Case 2: Run the test again, this time with manual mouse movement (either using trackpad or external mouse) when letter are being typed. Lowercase 'a' is produced instead of 'A'.

More details about the test case is documented here - https://github.com/adoptium/adoptium-support/issues/710

Root cause:

When debugging native Objective-C code in JDK the flag bits obtained using API - CGEventSourceFlagsState(kCGEventSourceStateHIDSystemState) resets to 0 (cleared) when any external mouse event occurs concurrently to the typing of letters. The reason as to why the flag bits are cleared on mouse move is unclear.

Reproducer.java
Posted by
harsh11
Up vote reply of harsh11 Down vote reply of harsh11
Add a Comment

I have added steps and reproducer test for verification.

It’s not clear whether you’ve filed a bug with Apple about this yet. If not, please do.

Also, please post your bug number, just for the record.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Posted by
eskimo
Up vote reply of eskimo Down vote reply of eskimo
Add a Comment