Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When performing a say all on webpages, the screen should scroll along with the speech #904

Open
nvaccessAuto opened this issue Sep 13, 2010 · 11 comments
Labels
audience/low-vision PR or issue is relevant to sighted or low vision users enhancement feature/browse-mode

Comments

@nvaccessAuto
Copy link

Reported by mike.reiser on 2010-09-13 20:25
I was showing my sighted friend a news story, and while doing a sayall, he said that it didn't follow on the screen while the speech was going. It would be nice if this could be done so that my friend could follow along while I'm reading.

@nvaccessAuto
Copy link
Author

Comment 1 by jteh on 2010-09-13 22:21
Changes:
Milestone changed from None to None

@nvaccessAuto
Copy link
Author

Comment 2 by steverep80 on 2015-02-05 20:53
As a low vision user of NVDA, I would also greatly appreciate this enhancement in order to follow along with what I am reading (not to mention for the same reason as pointed out by the originator - reading simultaneously with my sighted wife). g. The only way I see to do that now (as of 2014.4) is to arrow along in browse mode, which is horrible for just sitting back and reading articles or other documents. If this enhancement were added, add-ons such as Focus Highlight might also more easily be able to fix issues with following along during scrolling.

@nvaccessAuto
Copy link
Author

Comment 3 by briang1 (in reply to comment 2) on 2015-02-06 08:50
Replying to steverep80:

As a low vision user of NVDA, I would also greatly appreciate this enhancement in order to follow along with what I am reading (not to mention for the same reason as pointed out by the originator - reading simultaneously with my sighted wife). g. The only way I see to do that now (as of 2014.4) is to arrow along in browse mode, which is horrible for just sitting back and reading articles or other documents. If this enhancement were added, add-ons such as Focus Highlight might also more easily be able to fix issues with following along during scrolling.

I agree on this one. I get quite frustrated sometimes when I have an issue on a web site, wheel in a sighted person and when I'm in the area where I encounter a problem, the sighted person cannot see it as the screen has not scrolled to where it is. I imagine there could be some overhead here with speed if the screen has to be scrolled after a while. So I also think it shoud highlight the word its reading as well, so at least the user might be able to show the person ho elping what is going on.
Is the problem with this on various browsers a performance one? I note that it has a recent ticket to refactor the way speech is handled and thought that while this was being condi sidered, this function in browse mode would need to be considered as well.

@LeonarddeR
Copy link
Collaborator

I agree there are some quirks that need to be addressed regarding browse mode and sighted use in general. See also #6460 and #6382.

CC @Brian1Gaff @jcsteh @Qchristensen @feerrenrut

@Qchristensen
Copy link
Member

There are two issues I think - knowing exactly where the element is that is being read is in the visual representation of a web page, and then deciding how to scroll the page (if at all) when the element being read is not visible.

Using Say all, I found that none of the current browser versions (Firefox, Chrome, IE or Edge) on Windows 10 moved the focus at all during reading.

For a "control", I went and had a look at Word. Doing a Say All on a long document, NVDA moves the focus to the start of each sentence (not line), scrolling the page so the cursor is on the third line from the top if the sentence being read starts off the bottom of the page. Visually, I find that a little distracting, as when it scrolls, I have to look for the first sentence on the third line. There was a slight lag as well

Next, I looked at Notepad. It moves the focus to each line as it reads it. If a line is off screen, it scrolls just enough to bring it on screen - so after the first screen of text, it scrolls every line.

Assuming it can be implemented without too much trouble, I'm in favour of having the ability to follow the text being read.

I think the two main options are:

  • Try and follow the current line. Either keep it at the top, middle or bottom of the screen once moving past the first screen.
  • Read through the current screen, then scroll one full screen.
    I'm not sure we'd get 100% agreement from users on either of those (or any of those since the first is really three options). Assuming it was all implementable and not too much harder to do the rest after doing one, I'd be tempted to put an option maybe in "object presentation"? something like: "Scroll during say all:" with a drop down list of options: "focus at top of screen, focus at middle of screen, focus at bottom of screen, scroll full screen, and, none" (the last particularly if there are performance issues with implementation. Ideally then use the chosen implementation across different programs.

@Brian1Gaff
Copy link

Brian1Gaff commented Jul 19, 2017 via email

@LeonarddeR
Copy link
Collaborator

@Qchristensen commented on 19 jul. 2017 07:09 CEST:

I'm not sure we'd get 100% agreement from users on either of those (or any of those since the first is really three options). Assuming it was all implementable and not too much harder to do the rest after doing one, I'd be tempted to put an option maybe in "object presentation"? something like: "Scroll during say all:" with a drop down list of options: "focus at top of screen, focus at middle of screen, focus at bottom of screen, scroll full screen, and, none" (the last particularly if there are performance issues with implementation. Ideally then use the chosen implementation across different programs.

I don't think this should be limited to say all. IN my opinion, scrolling in say all and manual scrolling with arrow keys/quick nav should look equal. Furthermore, there is indeed some difference in what supports what, see #6382 (comment)

@jcsteh
Copy link
Contributor

jcsteh commented Jul 19, 2017

As discussed at length in #6382, the exact way we scroll is largely dependent on what the app facilitates and it's difficult to come up with a UX that suits everyone. For the purposes of this issue, I think we can all agree that say all should scroll.

Technical: The reason this doesn't currently happen is that say all calls updateCaret on the TextInfo. However, for virtual buffers, that just updates the buffer's internal caret offset. In order to trigger scrolling, we'd need to somehow have say all (whether directly or indirectly) use _set_selection on the CursorManager. (This is why braille has a CursorManagerRegion.) I guess we might need to special case this in sayAllHandler too.

Note that I'm not as certain we want to move the focus while doing a say all, as that could cause changes to the page on poorly authored sites. At present, it can also cause weirdness with auto focus mode switching (#2039). To avoid issues like this, we should pass REASON_SAYALL to _set_selection and handle it accordingly in the browse mode implementation.

@Qchristensen
Copy link
Member

Nothing to add, except we've had a few requests for this still lately.

@Adriani90
Copy link
Collaborator

Actually, is it really that important to scroll in real time while sayall? Maybe it would suffice to provide a keystroke which moves the screen to the current virtual cursor without interupting the sayall, so that people can scroll on demand.

@feerrenrut
Copy link
Contributor

@Adriani90 for low vision users this is a fairly important feature.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
audience/low-vision PR or issue is relevant to sighted or low vision users enhancement feature/browse-mode
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants