Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Windows 10 start menu: Search suggestions are not announced automatically #5049

Closed
nvaccessAuto opened this issue Apr 23, 2015 · 6 comments
Closed

Windows 10 start menu: Search suggestions are not announced automatically #5049

nvaccessAuto opened this issue Apr 23, 2015 · 6 comments
Labels
enhancement

Comments

@nvaccessAuto
Copy link

Reported by nvdakor on 2015-04-23 05:22
Hi,
Hopefully this would be first of a series of proposals on supporting Windows 10. This is spin-off from a ticket on supporting search results in Windows 8.1 start screen:
If you search for items in Windows 10 Start Menu, NVDA does not announce search results automatically.

STR:

  1. Try using Windows 10 Tech Preview build 10041 or later with latest NVDA snapshots installed (I don't recommend using official NvDA releases in Windows 10 yet).
  2. From Start Menu, try searching for something (such as Project Spartan).
    Expected: NvDA announces search results as they come in or with some delay.
    Actual: Results are not announced.

Technical:

  • We need a new app module (searchui.py) to handle the new search routine in Windows 10. This could also provide basis for working on code to let NVDA cooperate with Cortana.
  • The search results are presented as edit fields when they may not be what they claim to be. Dev info shows the following:
name: u'Search box'
role: ROLE_EDITABLETEXT
states: STATE_FOCUSABLE, STATE_FOCUSED
isFocusable: True
hasFocus: 1
Python object: <NVDAObjects.Dynamic_EditableTextWithoutAutoSelectDetectionUIA object at 0x04E62ED0>
Python class mro: (<class 'NVDAObjects.Dynamic_EditableTextWithoutAutoSelectDetectionUIA'>, <class 'NVDAObjects.behaviors.EditableTextWithoutAutoSelectDetection'>, <class 'editableText.EditableTextWithoutAutoSelectDetection'>, <class 'NVDAObjects.behaviors.EditableText'>, <class 'editableText.EditableText'>, <class 'NVDAObjects.UIA.UIA'>, <class 'NVDAObjects.window.Window'>, <class 'NVDAObjects.NVDAObject'>, <class 'baseObject.ScriptableObject'>, <class 'baseObject.AutoPropertyObject'>, <type 'object'>)
description: u''
location: (48, 728, 320, 40)
value: u'project Spartan'
appModule: <'appModuleHandler' (appName u'searchui', process ID 4060) at address 4cf9330>
appModule.productName: exception: No version information
appModule.productVersion: exception: No version information
TextInfo: <class 'NVDAObjects.UIA.UIATextInfo'>
windowHandle: 66106
windowClassName: u'Windows.UI.Core.CoreWindow'
windowControlID: 0
windowStyle: -1811939328
windowThreadID: 4488
windowText: u'Search'
displayText: u''
UIAElement: <POINTER(IUIAutomationElement) ptr=0x6312f68 at 4cfbc60>
UIA automationID: SearchTextBox
UIA frameworkID: XAML
UIA runtimeID: (42, 66106, 2, 1)
UIA providerDescription: [Main(parent link):Unidentified Provider (unmanaged:Windows.UI.Xaml.dll)](pid:4060,hwnd:0x0)
UIA className: TextBox

Tested on Windows 10 Tech Preview build 10061 running as a VM guest in VMWare.

  • As the user searches for items, this field is updated, firing value change events. The issue is that this event is fired more than once by the search result object when new results are available. Thankfully, search results are stored as value for this object, so it might be possible to catch this. This is reproducible when using accevent.exe from Windows 8.1 SDK installed in Windows 10.
  • Ideally, we should try to provide similar user experience as that of Windows 8.1: when new results come in, there should be a delay of about half a second before announcing results.

Thanks.
@nvaccessAuto
Copy link
Author

Comment 1 by nvdakor on 2015-06-02 05:36
Hi,
Further work shows the following:

  • In Build 10130 at least, search results are shown as a list item. This is for desktop variation of Start screen - might be different on tablet mode (hope it is the same over there).
  • Using Windows 10 SDK preview's accevent, it is shown that we're dealing with an updated version of Windows 8's search results list. Thus it might be possible to use the same logic (explorer.SearchListItem) in search UI (searchui.SearchListItem). As for where it should live, I can think of two ways: app module or as part of UIA API level.
    As for resolving this, I can try either:
  • Using the same logic implemented in Explorer app module i.e. copying the search list item class content.
  • Creating a new API level class named SearchListItem under UIA that'll take care of both situations, but they need to be unique and should work under both Windows 8.x and 10.
    I'm leaning towards the first approach for now unless using UIA API level class is better in the long run.
    Thanks.

@nvaccessAuto
Copy link
Author

Comment 2 by Michael Curran <mick@... on 2015-06-09 23:20
In [faffb8a]:

Merge branch 't5049' into next. Incubates #5049

Changes:
Added labels: incubating

@nvaccessAuto
Copy link
Author

Comment 3 by Michael Curran <mick@... on 2015-06-25 09:02
In [6d3ef4b]:

Merge branch 't5049'. Fixes #5049

Changes:
Removed labels: incubating
State: closed

@nvaccessAuto
Copy link
Author

Comment 4 by leonarddr on 2015-07-30 09:07
It seems that search suggestions are read correctly, however multiple times:

Str:

  1. Go to start
  2. Type 'edge'
    Expected: NVDA says Microsoft Edge only once
    Actual: NVDA says Microsoft Edge multiple times, four times in a row in my case.
    It might be desired to open a new ticket for this, but it seemed to me that this ticket is closely related.

@nvaccessAuto
Copy link
Author

Comment 5 by nvdakor on 2015-07-30 13:55
Hi,
Wouldn't caching help? That is, whenever selection event is fired, cache the result and do not announce it if subsequent events return the same text?
Thanks.

@nvaccessAuto
Copy link
Author

Comment 6 by brandon15 on 2015-11-05 12:06
Hello,

While this works most of the time, I recently discovered a bug that may cause this to stop working.
I don't know whether this happens on official builds of windows 10, but on the build 10500.x such as 10547, 10565, or 10576, do the following?

  1. Shutdown the computer if it's running (I can't determine another way to do this unless I do it from first booting)
  2. Open NVDA.
  3. Use windows+R for the run box.
  4. For this to work you should have the very latest version of Skype. So in the run box, type Skype.
  5. Click ok.
  6. When the Skype window opens, select a contact and open their chat window (mine is set to split window view)
  7. Close the chat window, and press alt+f4 to close Skype, and you should land in the task bar. Note that I have my Skype set to land in the tray when minimized.
  8. Now press the windows key, and type something in the search box.
    For example I type e to see results starting with that letter.
    NVDA doesn't read them automatically. Additionally, when I down arrow I might hear, eloquence
    eBay
    etc, but it doesn't tell me the result type. In that example, Eloquence is a desktop app, and eBay is a store app.
    But when this happens, all this information is lost.

Do you by chance know what the problem is?

Thank you very much for your time and I look forward to any assistance you are willing to provide.

Thanks,

Brandon

This was referenced Nov 10, 2015
Closed
Closed
michaelDCurran added a commit that referenced this issue Nov 23, 2015
@michaelDCurran
Merge branch 't5049'. Fixes #5049
6d3ef4b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@nvaccessAuto