Comment 2 by nvdakor on 2014-04-13 10:51
Hi Jamie,
Thanks for locating the correct ticket for this request.

Comment 3 by nvdakor on 2014-04-14 23:51
Hi,
Initial work is ready for review:

• Branch: t2559
• Commit ID: d994dc3cb
  I decided to implement this on next nav object command for now to gather feedback. Borrowed play audio coordinates method from mouse handler unless if there is a better version.
  Thanks.

Comment 4 by nvdakor on 2014-04-15 00:45
Further to the commit, I have a design suggestion:
If audio coordinate beep is to be heard while moving the mouse, flicking on the touchscreen and via keyboard-based object navigation, then I propose that a new function be created in the API module. The specs would be as follows:

• api.playObjectCoordinates(obj)
• Purpose: given an object, play its XY coordinate.
• Implementation: essentially move mouseHandler.playAudioCoordinate to API module so it can be invoked in touchHandler, globalCommands, mouseHandler and other modules.
  Algorithm:
• The new function retrieves the coordinates for the obj via obj.location.
• Retrieve the "screen resolution" i.e. desktop object's coordinate.
• After this, it'll be same as the current mouseHandler.playAudioCoordinates function.
  Usage from other modules:
• When the user sets play audio coordinates to yes, the routine which calls this function should pass the desired obj to the API function if the object has location.
  Example:

  if config.conf.PlayAudioCoordiante: api.playObjectCoordinates(curObject)
  
  

Thanks.

Comment 5 by nvdakor on 2014-04-15 07:12
Hi,
A preliminary implementation is ready for review:

• URL: https://bitbucket.org/nvdaaddonteam/nvda
• Branch: t2559APIExperiment
  All I did was copied mouseHandler.playAudioCoordinates to api.py for now and added a new function called api.playObjectCoordinates. The global commands commands for next/previous object now calls api.playObjectCoordinates(curObject) to play the object coordinates.
  I'm sure a better solution exisets. Thanks.

Comment 6 by jteh on 2014-04-15 09:00
In general, new functions should not be added to the api module. It's poorly named, and in an ideal world, everything would be in more specific modules and the api module wouldn't exist. :) Perhaps the screenExplorer module is a better place for this.

Comment 7 by nvdakor on 2014-04-15 09:19
Hi,
I think that's reasonable to move the audio playback function to screen explorer.
as for the original question: I think playObjectCoordinates should be moved into screen explorer so other modules may use them instead of adding object coordinate beep code in the global command routines. What's your take on that?
Thanks.

Comment 8 by nvdakor on 2014-04-15 10:06
Hi,
Okay, a major refactor has been made thanks to the suggestion:

• URL: https://bitbucket.org/nvdaaddonteam/nvda
• Branch: t2559APIExperiment
• Commit ID: f342f74f
  In this refactor, we have:
• Coordinate playback functions were removed from API and were transfered to screen explorer which also now has copyright header.
• playAudioCoordinates has been renamed to playScreenCoordinates to indicate the fact that we're broadening the scope from mouse to touch exploration as well (sorry if I should have kept it as playAudioCoordinates).
• mouseHandler.playAudioCoordinates is kept back in case we need to revert this change.
• Global commands: object nav commands now calls screenExplorer.playObjectCoordinates(curObject).
• Screen explorer now imports some needed modules (mouseHandler, screen bitmap, config) used in audio coordinates method. Also added mouseHandler.scrBmpObj instead of scrBmpObj as Python complained about not finding global variable.
  I'm going to bed. Thanks.

Comment 9 by nvdakor on 2014-04-18 02:43
Hi,
A major flaw with object location sound was found when navigating large websites via object navigation:

• If you move to the middle of the page and if your navigaor object is located towards the top of the page i.e. offscreen but on top, NVDA will emit higher pitch than expectedd. in other words, if the top location value is negative, this means the navigator object is located offscreen.
• If the navigator object moves to the end of the page i.e. offscreen towards the bottom, Python throws overflow error. That is, if the top location of the object returns more than 100 percent, nVDA will play negative pitch, which is impossible.
  This would be problematic for those using touchscreens to navigate a webpage.

Comment 10 by jteh (in reply to comment 9) on 2014-12-01 06:37
Replying to nvdakor:

• If you move to the middle of the page and if your navigaor object is located towards the top of the page i.e. offscreen but on top, NVDA will emit higher pitch than expectedd. in other words, if the top location value is negative, this means the navigator object is located offscreen.
• If the navigator object moves to the end of the page i.e. offscreen towards the bottom, Python throws overflow error. That is, if the top location of the object returns more than 100 percent, nVDA will play negative pitch, which is impossible.

The coordinates should be restricted to the screen size, which you can get with api.getDesktopObject().location.

Comment 11 by nvdakor (in reply to comment 10) on 2014-12-01 06:52
Replying to jteh:

Replying to nvdakor:

The coordinates should be restricted to the screen size, which you can get with api.getDesktopObject().location.

Right. Should the user be alerted that the object is offscreen? I'm thinking no.
Thanks for the review and pointing out something I haven't thought about.

Comment 12 by nvdakor on 2014-12-02 03:38
Hi,
Hopefully this commit should be enough - I think I've got everything except for the GUI option to configure this behavior:

• RURL: http://bitbucket.org/nvdaaddonteam/nvda
• Branch: t2559APIExperiment
• Commit ID: 5f5e042
  I'm willing to create a t2559 branch based on current work.
  For mouse coordinates, would you like to retain the coordinates function in mouse handler or move it to screen explorer? If moving to screen explorer, I propose creating a master coordinate player method that is really the coordinates function from mouse handler and which the new object coordinate can call. Thus when moving to next/previous object, screenExplorer.PlayObjectCoordinates will call screenExplorer.playLocationCoordinates (or perhaps call it play screen coordinates), with mouse handler calling playScreenCoordinates in screen explorer.
  For the config section, I'm thinking it should live in object presentation settings with the setting set to off.
  Thanks.

Comment 13 by jteh (in reply to comment 12) on 2014-12-02 04:02
Replying to nvdakor:

For mouse coordinates, would you like to retain the coordinates function in mouse handler or move it to screen explorer?

Move it to screenExplorer.

If moving to screen explorer, I propose creating a master coordinate player method that is really the coordinates function from mouse handler and which the new object coordinate can call. Thus when moving to next/previous object, screenExplorer.PlayObjectCoordinates will call screenExplorer.playLocationCoordinates (or perhaps call it play screen coordinates), with mouse handler calling playScreenCoordinates in screen explorer.

This makes sense. I think playLocationCoordinates and playObjectCoordinates.

For the config section, I'm thinking it should live in object presentation settings with the setting set to off.

Agreed.

Comment 14 by Joseph Lee <joseph.lee22590@... on 2014-12-02 05:45
In [9e5eb65]:

It is now possible to hear object coordinates when using object navigation commands via keyboard or mouse and moving via touch gestures. re #2559
Audio coordinate announcement function now lives in screen explorer. Essentially the new play location coordinates function is same as old mouse handler version.
In screen explorer, added a new function to play coordinates for a given NVDA object. This function in turn calls location coordinate function in screen explorer.
In object presentation settings, there is now a new option to toggle announcement of object location coordinates when using object navigation commands. By default this checkbox is not checked.

Hi,

Coming back to this issue...

about to rebase it based on the current master branch and moving the branch to my own forked repo so people can start testing this.

A possible follow-up issue might be asking NVDA to play coordinates when one uses focus command such as tab, arrows and so on (for this, an even simpler solution is to expand what becomeNVDAObject event can do by playing coordinate beeps).

Thanks.

Hi,

Blocked by #5906 due to the following:

If indent announcement is set to beep, moving to an edit field that indicates indents (such as Notepad++) will mean one of the new beeps will not play. Will wait until a decision on what to give priority is made.

Thanks.

@josephsl @michaelDCurran I'm curious if simply playing indent beeps 80 MS or so in the future would help alleviate this bug

Hi,

Coming back to this...

Speech refactor may open up possibilities for this work, although lately I've been thinking about withdrawing this issue in favor of Audio Themes or similar mechanism.

Thanks.