Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Touch gestures: introduce arrows/navigation mode to simulate pressing arrow keys via flick gestures #4443

Open
nvaccessAuto opened this issue Sep 6, 2014 · 4 comments
Open

Touch gestures: introduce arrows/navigation mode to simulate pressing arrow keys via flick gestures #4443

nvaccessAuto opened this issue Sep 6, 2014 · 4 comments
Assignees
@michaelDCurran
Labels
enhancement feature/touch-screen

Comments

@nvaccessAuto
Copy link

Reported by nvdakor on 2014-09-06 01:52
Hi,
There are certain situations where trying to accomplish something via object navigation may fail on touchscreens. For example, when trying to change languages in general settings dialog, one needs to use physical arrow keys or remember to click expand button. Thus I propose introduction of a mode to allow flicks to emulate pressing physical arrow keys.

Example situation: A user wishes to change keyboard layout.
Scenario 1: when using the keyboard, the user would open NVDA's keyboard settings dialog, then use physical arrow keys to change keyboard layout between desktop and laptop.
Scenario 2: in order to change keyboard layout using touchscreens, one needs to move inside the combo box control until one lands at expand button. Double-tap, move down the object hierarchy (one finger flick down in object mode) in order to land in a list of keyboard layouts. When one wishes to move onto other controls in keyboard settings dialog with touchscreens, one cannot simply move among objects and would resort to using TAB key to move to other settings, or locate the other controls by exploring the screen.
Scenario 3: with the arrows/navigation mode implemented, a touchscreen user would switch to arrows mode and use up and down finger flicks to select the desired keyboard layout.

Implementation strategies:

  1. Introduce an arrows/navigation mode in touch handler module that'll allow touch gestures to emulate arrow keys. This is similar to how some braille displays emulate arrows using braille commands.
  2. A user would switch to this mode using the existing mode switch command (three finger single tap). By default, NVDA should start in object mode like previous releases.
  3. Either in touch handler or in global commands, add arrows/navigation layout flag for finger flick gestures.
    The commands in arrows mode would be as follows:
  • One finger flick left/right/up/down: left, right, up and down arrows.
  • Two finger flick left/right/up/down: control+arrow keys.

Possible concerns:

  • Some may ask, "doesn't NVDA support touch keyboard and can the touch arrow keys be used instead?" Yes, which requires the user remember how to activate touch keyboard (located on the bottom of the screen between taskbar and notification area) and the location of the touch arrow keys. The new arrows mode would simplify this approach, in that the commands can be executed from virtually anywhere.
  • Doesn't some braille displays emulate arrow keys using their braille keyboards? Yes, and the new touch arrows mode would utilize this functionality but implemented in a different module.
  • Some would also ask for functionality to let touch gestures emulate TAB key. This should be discussed on another ticket, as this ticket is meant to talk about emulating just the arrow keys.

Thanks.
@nvaccessAuto
Copy link
Author

Comment 1 by jteh on 2014-09-06 09:56
While simulating the arrow keys is one possible solution to the problem of navigating combo boxes, it's more of a hack than an elegant solution.

  • A user shouldn't have to think about using a keyboard when using a touch screen. It's a keyboard after all, not a touch screen.
  • It doesn't scale well. Arrow keys might work for combo boxes, but other controls might need other keyboard commands; e.g. alt+arrows, page up and page down. Even if there were enough gestures to cover all of the needed commands, the experience would become unintuitive very quickly.
  • Mode switching is cumbersome. (In fact, I'm leaning towards dropping our current touch modes in favour something else, but that is a separate issue.)

The correct solution here is probably to make double tapping the combo box expand it and to improve the experience of expanding/collapsing Win32 ComboBox controls.

Btw, this isn't to say that we shouldn't implement keyboard simulation on a touch screen at all--there are doubtless times where it could be useful--but it should be a last resort, rather than the primary means of achieving something.

@nvaccessAuto
Copy link
Author

Comment 2 by nvdakor (in reply to comment 1) on 2014-09-06 11:49
Replying to jteh:

  • It doesn't scale well. Arrow keys might work for combo boxes, but other controls might need other keyboard commands; e.g. alt+arrows, page up and page down. Even if there were enough gestures to cover all of the needed commands, the experience would become unintuitive very quickly.

I agree. In the end, one may resort to three or four finger flicks to perform some of these commands such as page up/down.

  • Mode switching is cumbersome. (In fact, I'm leaning towards dropping our current touch modes in favour something else, but that is a separate issue.)

What ideas do you (and Mick) have in mind? Should we talk about dropping touch modes on the mailing list first to gather feedback before opening a ticket regarding your suggestion?

The correct solution here is probably to make double tapping the combo box expand it and to improve the experience of expanding/collapsing Win32 ComboBox controls.

Right, because when I perform review_activate, it seems to do nothing (in fact, for NvDA's preferences dialogs, it just closes the dialog).

Btw, this isn't to say that we shouldn't implement keyboard simulation on a touch screen at all--there are doubtless times where it could be useful--but it should be a last resort, rather than the primary means of achieving something.

I see. I do agree that there are times when emulating keyboard commands via touchscreens would be desirable. My reason for the suggestion was to provide a way for a touchscreen user who doesn't have access to physical keyboard to "press" at least the arrow keys to navigate controls which supports this such as combo boxes.
Thanks for your input.

@LeonarddeR
Copy link
Collaborator

@josephsl: any recent thoughts about this issue?

@Adriani90
Copy link
Collaborator

how about integrating and improving the quicknav addon from Tuukka and assigning touch gestures to its functions?

QuickNav

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@michaelDCurran michaelDCurran

Labels
enhancement feature/touch-screen
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@LeonarddeR @michaelDCurran @nvaccessAuto @Adriani90