Comment 1 by jteh on 2015-02-09 02:21
Implementing this using mouse clicks would be potentially problematic. We'd have to override mouse clicks and we can't know for certain whether clicking there would normally activate something. I agree that the current solution is tedious, but I don't think clicking is the answer.

Comment 2 by steverep80 on 2015-02-09 03:18
I definitely see your point on many users, especially those who are totally blind, running into a dangerous situation of not knowing whether the click is to link to or activate something or not. I can't say I fully understand the scenarios you'd be afraid of though since numpad-divide and numpad-multiply are already linked to mouse clicks. Could you give a couple examples? I'm sure there's a compromise here.

If the problem is just mouse vs. keyboard control, then a first step might just be a new NVDA+something command to combine steps 2 and 3 after placing the mouse (e.g. NVDA+shift+numpad-multiply might make sense for the desktop scheme).

Comment 3 by jteh on 2015-02-09 03:33
Currently, we don't ever override the function of the mouse buttons. If I understand you correctly, you're suggesting that we do; i.e. clicking should move the browse mode cursor. In a web browser, clicking normally activates things. The only time it moves the caret is in an editable text field. If we overrode this, clicking might not activate something when it was intended to by the author.

Comment 4 by steverep80 on 2015-02-09 04:03
Ah, there might be a misunderstanding here. I'm not suggesting changing what a click would do in any way for a webpage (it should activate a button, link, or whatever as the author intends). In fact, NVDA will already do what I'm suggesting in those cases. For example, if I go to the NVDA User Guide page and left-click and hold on a link, but then move the mouse away so that Firefox doesn't actually open the page, the result is that the focus and cursor are now moved to the link location, and I can arrow down the page from there.

What I'm suggesting is that the same thing should happen when I left-click on a paragraph, heading, graphic, or any other inactive element, analogous to being in an editable field or document. It would allow low vision users to pick a focus point to start reading from on particularly difficult to navigate pages.

Comment 5 by jteh on 2015-02-09 19:59
I understand what you're suggesting. The problem is that clicking just about anything can activate it. That is, anything can be "clickable", even some things that don't report as clickable for various reasons. This is why enter will always click in browse mode if all else fails.

So, doing as you suggest could break some of these edge cases. It might also unintentionally cause problems for other mouse functionality; e.g. scrolling or dragging.

Aside from using keyboard commands, the only thing we could perhaps consider is something like NVDA+click; i.e. hold down the NVDA key and click. That way, we can be certain it was intended to affect NVDA and only override in that case.

Comment 6 by steverep80 on 2015-02-10 00:05
I may not fully understand those edge cases, but I'm the beginner at understanding the NVDA core and you're obviously the expert, so I'll heed to your judgement. Let's focus on the possible compromises. So we've got two so far:

1. NVDA+mouse-click overriding to move the focus/carat to the clicked position. In fact, you might extend that and say NVDA+left-click moves the focus/carat, and NVDA+right-click moves the navigator position.
2. Move the mouse to the desired position, and rely on a new keyboard shortcut to move the focus/carat in less steps.

I really like option #1, but I'll play devil's advocate and say it's sort of a conflicting paradigm considering NVDA+numpad-divide is already mapped to move the mouse to the current navigator object. On the other hand, it makes perfect sense since NVDA+numpad-multiply already moves the navigator to the mouse (and numpad-multiply is a right click. What do you think?

Comment 7 by steverep80 on 2015-02-10 17:17
Sorry to not wait for a reply, but I was thinking about this overnight and I think a really consistent set of commands would be:

I guess the only factor here is whether you guys would be willing to change the function of NVDA+numpad-divide, but this table seems like the most consistent approach and would really help out users who can still see and/or use the mouse.

Comment 8 by jteh on 2015-02-10 22:29
Changing these commands like this would be fairly dangerous, since users who are already used to them would end up clicking when they didn't intend to, which could have disasterous results. I also don't follow how adding a mouse click solves your problem. You want to move the browse mode cursor, and as I've pointed out, clicking cannot and will not do this in many cases.

As I understand your request, what you want is just a single command to route the caret to the mouse, preferably something you can activate from the mouse itself.

Comment 9 by steverep80 on 2015-02-10 23:31
Yes, I believe you understand the request to quickly move the carat to the mouse. I do not at all understand how you think the table of commands I proposed could have "disastrous" results. The only command that currently has a function that I'm proposing to change is NVDA+numpad-divide, and the worst that happens is someone moves the carat instead of moving the mouse, which might result in confusion unless the user read's the release notes, updated user guide, and/or pop-up after installing the version with the change, and simply adapts (something the visually-impaired are quite good at). The commands in that table are mostly as they are today, with functionality changes as follows:

1. Remap current NVDA+numpad-divide to NVDA+shift+numpad-multiply (an unassigned key)
2. Enhance current NVDA+numpad-multiply functionality by also allowing NVDA+right-mouse-click to do the same thing
3. Add a new function to move the carat to the mouse position, either via NVDA+numpad-divide (change) or NVDA+left-mouse-click (new)
4. Add new function to move the mouse to the carat/focus via NVDA+shift+numpad-divide (an unassigned key)

I would suggest sharing this ticket and our long list of comments with other users and developers to get additional perspectives, perhaps by posting to the mailing list?

Comment 10 by nvdakor on 2015-02-11 00:11
Hi,
A few thoughts:

• Some users could have reassigned these commands and may wish to do the same for the new commands, making the proposed commands invalid - invalid as in becoming invalid or unassigned.
• Some add-ons may use these commands in the future (I know one add-on which took over NVDA+Numpad Minus command for itself), potentially opening doors for more confusion - even more so if a user wishes to reassign commands for them on the main keyboard section, particularly on small laptops. One major concern we as NVDA contributors have is the fact that we're running out of commands, even more so when more add-ons are developed which doesn't check to see if a command is taken by the core or another add-on. Lastly, different languages have different commands for some NVDA core features, so it might not be a good idea to assume that the proposed commands will be accepted for now.
• What if the user may need to click (or route the mouse to) a control that is off-screen or invisible? This is more so on a site where a visitor may need to scroll to the invisible or object located off screen, particularly for controls which require mouse interactions nad if the interaction causes side effects (which isn't desirable (however, more and more sites are using dynamic contents and mouse-based interactions, especially gesture and touch-based interactions are gaining in popularity).
  One thing that caught my attention was the phrase which may imply (not to me for now) that visually impaired computer users are good at adopting to change. I'd like to kindly disagree: with limited resources and with "limited view and knowledge", visually impaired computer users have harder time adjusting to change, as our recent discussion on the NVDA users list regarding operating systems demonstrate. Part of the reason for some people to not adopt latest versions of software (particularly for professional applications) is the tendency for quicker changes - both outward (user interface, layout, etc.) and inwards (code, taking advantage of accessibility frameworks, etc.), and with current trends, this will accelerate, putting visually impaired users at a disadvantage. Although it may seem that changing the behavior of a command that seems trivial to implement is a good thing, when we go deeper into this issue (based on Jamie's comments above), it is a major change (take a look at ticket 804 (I believe) where users and developers had a heated debate on new laptop commands). But I can see why the proposed change might be good for low vision users (if we look at the current blindness statistics, we'll be able to serve more and more low vision users), but because of the fact that NVDA is primarily coded by two blind programmers with help from other visually impaired programmers around the world, I'd say that, for now, let NVDA be good at what it does now (screen reading), as well as have some room for collaboration from low vision programmers and users in hopes of asking NVDA and perhaps a magnification tool (built-in or third party ones) to cooperate with each other in the future.
  Thanks.

Comment 11 by jteh (in reply to comment 9) on 2015-02-11 00:19
I think I misunderstood your intention. You noted "Same keyboard command with addition of the mouse clicking ability". I thought you meant you wanted the command to click the mouse as well, when I now think you just meant an additional (mouse) gesture for the same command.

Can you explain the need to change NVDA+numpadDivide rather than just using new keys for new commands?

It's worth noting that too many routing commands may actually confuse things further. Right now, everything goes via the NVDA review cursor. That is, you can route the review cursor to any other cursor and vice versa. That does sometimes mean multiple commands, but it also makes for a consistent experience.

It's also worth noting that you can set the review cursor to follow the mouse cursor (NVDA menu -> Preferences -> Follow mouse cursor). That way, you should just be able to move the mouse and then route the caret to the mouse, since the review cursor will always be at your mouse position unless you explicitly move the review cursor after you move the mouse.

Comment 12 by steverep80 on 2015-02-11 03:30

To nvdakor

I'm not really sure how your comments contribute to the actual feature being discussed. The comments about addons or users changing commands applies so generally and is a separate topic if you want a feature to prevent or better handle this. And I'm not sure what off-screen or dynamic content has to do with this.

I do, however, understand the constrained number of key commands, and, yes, I certainly understand how it can be difficult to adapt to new versions of software, operating systems, etc. Give me a break though and be a little more positive.

Lastly, and this is where we really are going to disagree, is that "letting NVDA be" is not exactly a fantastic sales pitch to gain more users and grow the community into a compromised set of visually-impaired users, from starting to lose sight to never having known it at all. So, yes, I'm proposing a change that is primarily for low vision users, but so what? That may mean more donations and faster adaption.

Now, the one thing you did say that has to do specifically with this request is the mention of keys in other languages or on a laptop layout. I made no assumptions there, I'm simply not an expert in other language's keyboard layouts and don't use the laptop layout nearly as much as the desktop one. Please do chime in on how my proposal breaks those though. I'm an engineer and programmer by trade, and would like to solve that problem if there is one.

To Jaime

I think you hit a misunderstanding on the money again. I was not proposing a click anymore than already exists in the current command set.

The reason for changing the function of NVDA+numpad-divide rather than a new set of keys is what I was trying to illustrate with the wiki table. Right now, numpad-divide and numpad-multiply are left and right clicks, respectively, and NVDA+numpad-multiply moves the navigator to the mouse. To me, it makes perfect sense to add shift to that to do the reverse, i.e. move the mouse to the navigator. Then, you'd have numpad-divide to do the same things for the carat/focus, i.e. by itself is a click, adding NVDA moves the carat/focus to the mouse, and adding shift reverses it by moving the mouse to the focus/carat. Having NVDA plus actual mouse clicks be the same as NVDA+numpad-divide or multiply then follows directly since those are left and right clicks by themselves.

Your point about solely using the review cursor as the mover and move-to thing (for lack of a better term) is a good one, but I strongly feel it breaks down significantly in browse mode. If I move the navigator, single letter navigation and tabbing doesn't care until I get the carat there too. I just tested turning on having the review cursor follow the mouse. On the bright side, this does actually make it one less step, combining steps 1 and 2 from my original request description, but I still think my proposal is much better and obviously shorter. Keep in mind there's always a step 4 which is to either begin a "say all" or continue other arrowing or first-letter navigation from that point.

Comment 13 by jteh (in reply to comment 12) on 2015-02-11 03:39
Replying to steverep80:
Thanks for the explanation regarding the movement of commands. While it was never our intention that NVDA+numpadMultiply be associated with a left mouse click, etc., I understand that you're saying this would help with remembering the commands. I'm pretty reluctant to change commands users already know, though.

I just tested turning on having the review cursor follow the mouse. On the bright side, this does actually make it one less step, combining steps 1 and 2 from my original request description, but I still think my proposal is much better and obviously shorter.

I don't follow why it's much shorter. With what you're proposing, you'd move the mouse, then hit NVDA+numpadDivide or NVDA+leftMouse. With things as they currently are and follow mouse enabled, you move the mouse, then hit NVDA+shift+numpadMinus twice. It's a double press instead of a single press, but it's the same command, so I'd argue that's negligible.

Comment 14 by steverep80 on 2015-02-11 05:47
Well, it must be getting too late here in the east USA, because I cannot seem to repeat my previous statement, or maybe I never truly tested it. I have the checkbox to have the review cursor follow the mouse enabled right now, and it it seems to be following correctly, i.e. I can use the text review commands to see that the points are the same. However, when performing a NVDA+shift+numpad-minus double press, the carat doesn't move. The only way I can get it to move is by doing a NVDA+numpad-multiply first, as in the original description. Could someone else please test this? I've tried it on 3 different websites and same lack of result. I even resorted to restarting my PC thinking I royally screwed up something.

Comment 15 by steverep80 on 2015-02-15 23:54
Have we given up on discussing this? Regarding having the review cursor follow the mouse, and then double pressing NVDA+shift+numpad-minus not working, I'm prepared to open that as a separate issue as a bug instead of a feature request. That is, unless someone else tells me I'm crazy and it works fine. It seems to be a bug to me at this point.

Regarding the original feature request, if you are totally against the table of commands I proposed because it would change one existing command, that's fine I guess. I still think there is value in adding a feature to more quickly move the carat in webpages by clicking though, so what about just implementing the NVDA+mouse-click commands?

This is a friendly reminder for @jcsteh, and maybe @Qchristensen and @feerrenrut also (as sighted individuals), to kindly consider responding to #4896 (comment).

Since its been so long since there was activity on this issue, and it's quite long and time consuming to get familiar with, I'll attempt to summarise:

Use case:

For low vision nvda users who do not use mouse tracking and are familiar with
using nvda with the keyboard, being able to easily (with a shortcut like
nvda+click) move the focus and caret to the current mouse location could provide an
improved (and faster) navigation UX.

Currently this is possible and requires the following:

1. Place the mouse at the desired position, making sure to be in "document
   review"
2. Use NVDA+numpad-multiply to move the navigator object to the mouse
3. Use NVDA+shift+numpad-minus twice to move the focus and carat

Benefits

Improved navigation of pages with poor semantics or few landmarks. It's worth noting that mouse users (mostly) have their right hand on the mouse, and their left hand free to access the keyboard. This should be considered when picking the keys that should be pressed in combination with mouse use.

Related issues

• Move virtual buffer position to mouse position in browse mode feature #3004 (Move virtual buffer position to mouse position in browse mode feature)
• Use braille display keys as modifier keys #6213 (Use braille display keys as modifier keys)
• Add "report formatting under cursor routing key" command #7106 (Add "report formatting under cursor routing key" command)

Technical considerations

• We would have to override mouse clicks
• We would have to allow gestures made by multiple input sources, this is something that has been requested for braille as well (EG NVDA+Braille Key)
  • Use braille display keys as modifier keys #6213 (Use braille display keys as modifier keys)
  • Add "report formatting under cursor routing key" command #7106 (Add "report formatting under cursor routing key" command)
• There is a risk that clicking would activate something unintentional.

Further work

• Also providing NVDA+right mouse to move the Navigator object to the mouse.
• Provide a keyboard shortcut to move the mouse to the current focus/caret
  position.

This work would result in the following behaviour:

This ignores some of the more controversial new / changed keyboard shortcuts
suggested in
#4896 (comment) :

• Add a keyboard shortcut for "Move mouse to current focus/caret position" as NVDA+shift+numpad-divide
• Change keyboard shortcut for "Move mouse to current navigator position" to NVDA+shift+numpad-multiply
  • This would be a change from the keyboard shortcut which is NVDA+numpad-divide
  • This provides nice symmetry between this and moving focus/caret with the same keyboard shortcut except using divide instead of multiply.
• Change keyboard shortcut for "Move focus/caret to mouse" to NVDA+numpad-divide
• Keep existing keyboard shortcut for "Move navigator object to mouse" as NVDA+numpad-multiply

There are some benefits to changing these keyboard shortcuts, especially in terms of the symmetry. The major disadvantage is that its a subtle change that is likely to confuse existing users.

Questions

What is the disadvantage to using mouse tracking? Or to put it another way, what is the advantage of this solution over mouse tracking?
A low vision user could turn on mouse tracking and move the mouse to have things reported by NVDA.

@feerrenrut, I think if NVDA could provide a key command to move the focus to the mouse (i.e. NVDA + left mouse click), maybe this would simplify and solve this problem. I hope I didn't miss anything important from this discussion. In browse mode, the virtual cursor would be moved with NVDA + left mouse click. In focus mode, NVDA + left mouse click would move the focus.

Comming back to this, I think it would be much easier now to implement since we can split the virtual cursor from system focus with NVDA+8.
cc: @LeonarddeR

Revisiting this again, this is definitely something we get asked about regularly. What I thought should work:

1. EITHER In NVDA's review cursor settings, have "Follow mouse cursor checked" OR move to desired location on page and press NVDA+numpad multiply (NVDA+shift+n) to set the navigator object to the object under the mouse.

2. Press NVDA+shift+numpad minus (or NVDA+shift+backspace) twice to move the system focus or caret to the position of the review cursor.

3. Press NVDA+down arrow (NVDA+a) to say all.

Step 1 works, but step 2 doesn't seem to do anything.

I found setting the review cursor to follow the mouse, hovering the mouse over the text to read and then pressing numpad plus (NVDA+shift+a) to say all in review works - but only in the current element, because the next element is a new object. I wonder if we could make that a command you could press twice? EG: "Pressed once, reads from the current position of the review cursor to the end of the current object, moving the review cursor as it goes. Pressed twice, reads from the current position of the review cursor to the end of the document, moving the system caret as it goes"

That way the workflow for someone wanting to read the text under the mouse would be (having setup the review cursor to follow the mouse):

1. Hover the mouse over the text to read
2. Press numpad plus twice to read to the ned of the document

I believe an argument could be made for changing the numpad plus command to simply move the caret and do a regular say all with one press (most times when people initiate any kind of "read from here" command, they generally mean "read from here to the end of the document or until I stop it reading".