Comment 2 by jteh on 2014-10-27 09:29
Scan codes are dependent on the keyboard layout and may even be different for different types of keyboards. Is it expected that the keys produced may be different dependent on the layout being used for the system keyboard?

For these types of keyboards, I think it makes sense to just have them use keyboardHandler.KeyboardInputGesture (which is what is used for the system keyboard), rather than a BrailleDisplayGesture subclass. Is there any reason this isn't acceptable?

Comment 3 by halim on 2014-10-29 15:00
Thx for your answer. Maybe your sugested approach is ok but I need some hints :-).
The keyboard which I am trying to add support provides the following:

• key pressed state
• key is extended scancode
• key is an extended scancode of type 2 or std one
• the scancode itself

How can I use the keyboardinputgesture with this stuff?

Comment 4 by jteh (in reply to comment 3) on 2014-10-30 01:54
Replying to halim:

The keyboard which I am trying to add support provides the following:

• key is an extended scancode of type 2 or std one

Windows keyboard input deals with scan codes from scan code set 1. This means a key is either extended or it isn't. If this display specifies two types of extended key, this is a different scan code set and can't use NVDA's keyboard support.

Regardless, you'll need to know what spec is being used for the scan codes. There are three sets of scan codes for PC keyboards and yet another for USB keyboards. Windows applications see PC keyboard scan code set 1.

How can I use the keyboardinputgesture with this stuff?

If it does use set 1, we'll need to tweak KeyboardInputGesture so it can translate a scan code to a vk code using Windows API calls. Once it can do that, you'd just construct KeyboardInputGesture with the appropriate arguments and execute it as you would any other gesture.

As above, if it doesn't use set 1, you'll have to write your own implementation from scratch.

Comment 5 by halim on 2014-11-07 11:50
It seems the keyboard uses set 1 and doesn't use all mentioned extended flags.
Could you please extend the keyboardinputgesture class as described in your previous comment?
Or is a brailledriver with its own keyboard implementation acceptable???
Thx for your help.

Comment 6 by jteh on 2014-11-11 02:32
Looking at this more deeply, KeyboardInputGesture can't handle this alone. The problem is that KeyboardInputGesture.send deals with complete key combinations, not individual keys. That is, it presses all of the modifiers, then the key, then releases them all. This also means you have to track the modifiers yourself, handle double presses of the NVDA key, handle pass key through, etc. The question is how other displays handle this and whether they will end up using the same code.

I think the best way we can handle this is to introduce a keyboardHandler.injectRawKeyboardInput(isPress, scanCode, isExtended) function. This will use keyboardHandler's internal hook callbacks and send the raw key to the OS if appropriate.

Comment 7 by James Teh <jamie@... on 2014-11-11 03:46
In [27941d8768bf66f8118838962cf0f191171deb92]:

You can now injet raw input from a system keyboard that is not handled natively by Windows (e.g. a QWERTY keyboard on a braille display) using the new keyboardHandler.injectRawKeyboardInput function.

Re #4576.

Comment 8 by jteh on 2014-11-11 03:48
I've done an initial implementation of this in a branch (t4576). However, I'm waiting from feedback from others as to how other braille displays provide keyboard input. If other displays do it in a completely different way, this code isn't really reusable, so probably shouldn't go into the core. It's also possible they do it similarly, but use, say, vk codes instead of scan codes, in which case I'll need to tweak the function to handle this.

Comment 9 by jteh on 2014-11-11 10:32

• HIMS displays send QWERTY input as dots, so they must backtranslate internally.
• HumanWare BrailleNote sends QWERTY keys as modifiers and a vk code for the main key.
  So, neither of these displays can benefit from this proposed function. However, we may still want to add it anyway, especially because it needs to use internal NVDA functions which really shouldn't be called from drivers. I need to think on this.

Comment 10 by halim on 2014-11-12 13:57
t4576 seems ok for me to add the initialy requested keyboard support.
Is there a chance to get a build of t4576?

Comment 11 by halim on 2014-11-25 08:34
Thx a lot for the patch.
I was able to test it with the keyboard of the braillex live+.
What works:
Letters, numbers, f1-f10, esc, shift, ctrl, alt etc.
What doesn't work:
Left windows key, toggling numlock.
It seems the numlock is active and arrowkeys are emulating numpad keys (currently).
Left windows key seems to fire correct code according to
http://www.computer-engineering.org/ps2keyboard/scancodes1.html
Can you help?

Comment 12 by halim on 2014-11-25 13:52
Short update:
My misstake, arrow keys work now. The ext flag was wrong :-(.
Other problems are now:

• right alt key can't be used. nvda says left alt key.
• The function keys were not correctly reported in nvda's input help.
  EG: Arrwo keys are %, ), ',
• left windows doesn't work as well (haven'Ät tested right win key).

Comment 13 by halim on 2014-11-25 15:30
Little progress: Now nvda reports all keys except right alt correct but some of these keys don't work again.
-right alt reported as left alt

• left windows key announced correctly but doesn't open the startmenü etc.
• seems right windows key and applications don't work as well but reported all correctly now.
• don't know howto handly numlock status because it's a fn keycommand

Comment 14 by halim on 2014-11-27 20:59
Numlock will not work with the keyboard.

The bigest issue for now is why nvda reports leftwin rightwin and application key correct but doesn't trigger any aktion in windows?
Any Ideas?

Comment 15 by jteh on 2014-12-02 02:32
Triage note: We decided to merge this once any outstanding issues are fixed.

Comment 16 by halim (in reply to comment 15) on 2014-12-04 09:40
Replying to jteh:

Triage note: We decided to merge this once any outstanding issues are fixed.

Thx!

Here is a new summary of the known issues:

• alt-gr on the right hanside of space: The keyboard fires right code but nvda seems to ignore the extended flag. EG: the difference between left alt and right alt is only the extended flag.
• left windows, right windows, and application keys: They are correctly reported in nvda but don't trigger any action in windows. One difference to my usb keyboard is that the mentioned keys have a slightly other translation.
  EG: anwendung vs anwendungen in german.
  HTH.

Comment 17 by nvdakor on 2015-01-07 10:41
Hi,
I'm testing BrailleNote side of things at the moment, and have confirmed Jamie's observation: BrailleNote does send individual keys, one at a time with a flag right in front of the keys. This gets interesting when PC keyboard emulation is active - BrailleNote sends a flag indicating that emulation is active.
Setup:

• NvDA version: latest master branch source code (I'm working from a branch of my own).
• OS: Windows 8.1 64-bit
• Device: BrailleNote Apex BT32 with USB keyboard connected (Apex will think it is a QT while a keyboard is connected).
• Connection type: Bluetooth.
  For those interested: For BrailleNote, reading packets from Bluetooth com port reports the following:
• For BT mode (braille dots), two bytes are enough (first is the added modifier dots such as dot 7, the second is the actual dot pressed).
• For QWERTY mode (BT with PC keyboard emulation and QT with or without emulation mode on), up to four bytes are sent: flag, modifier, mode, letter. For example, with native QT mode without PC keyboard emulation, if I press the letter A, I get: 0x8D, 0x00, 0x8C, 0x41. With native QT mode with PC keyboard emulation, if I type the letter A, I get two bytes: 0x88, 0x61. These readings suggest that it might be possible to provide a way to construct arbitrary QWERTY commands by looking at the packet header, obtaining correct number of bytes (two or four) and convert the byte(s) to QWERTY equivalents (@ragb, please let me know if you'd like to read modifier to hex map unless you already have it).
  Thanks.

Comment 18 by nvdakor on 2015-01-13 07:22
Hi,
Further to comment:17, current code (which is available as a branch on my Bitbucket account) implements the following:

• Ability to detect QT input, either native mode or as an emulated PC keyboard. The header is distinct from BT (braille keyboard) packet. Also, when QT header is received, read packet is called once more to obtain the letter portion of the packet (the last two bytes, that is).
• QWERTY key echo - if a QT user presses keys on his or her BrailleNote keyboard, one will be able to hear names of keys pressed.
• BrailleNote QT packet constants - the constants named "QT*" are used by BrailleNote QT for sending key presses via serial port. Also a dedicated q key names map has been added to look up modifier keys for use with key echo for now.
• Input and gesture assignments are not possible. Also updated copyright year and authors.
  Issues to be resolved:
• When native QT mode (not PC keyboard emulation) is active, punctuation keys (such as semicolon) uses ASCII range greater than 127, so it is necessary to convert it to correct ASCII range (for these, you need to subtract 144 from ordinal representation).
• For input gesture, whether to use keys keyword or create a dedicated QWERTY keyword for handling QWERTY input. Currently keys argument is used to handle thumb-keys, and this might be extended to cover QWERTY keyboard input (same logic is used for Apex's scroll wheel, which is beyond the scope of this ticket). If QWERTY argument is introduced, this may be used to detect both the native QT mode and PC keyboard emulation on both BT and QT and construct appropriate keyboard gesture.
  Current code can be found at:
• URL: https://bitbucket.org/josephsl/nvda-dev
• Branch: t4576BrailleNote (I guess I should have used t4576/BrailleNote instead).
• Commit: 38074ae5f3
  Thanks.

Comment 19 by jteh on 2015-01-13 07:28
While it's probably not entirely clear from the summary, this ticket is more about the core support needed to allow raw scan code input from braille displays (rather than the changes or specific drivers). BrailleNote doesn't actually use raw scan codes, so it's a totally separate case. Please file a new ticket for the BrailleNote work.

Having said that, it might be better to map the BrailleNote QWERTY stuff to keyboardHandler.KeyboardInputGesture so it can benefit from NVDA's actual keyboard bindings, as this is meant to simulate a true keyboard. This is aprticularly true if BrailleNote uses Windows vk codes (I vaguely seem to recall this), as KeyboardInputGesture can do the mapping for you. Anyway, let's discuss that on the new ticket.

Comment 20 by halim on 2015-01-27 07:49
are you planing to merge the injectRawKeyboardInput for nvda-2015.1?

Comment 21 by jteh on 2015-01-27 08:25
There probably isn't time, unfortunately. There is still some work needed to make this code future proof.

Comment 22 by halim (in reply to comment 7) on 2015-02-25 10:24
Replying to James Teh <jamie@…>:

In [27941d8768bf66f8118838962cf0f191171deb92]:

#!CommitTicketReference repository="" revision="27941d8768bf66f8118838962cf0f191171deb92"
You can now injet raw input from a system keyboard that is not handled natively by Windows (e.g. a QWERTY keyboard on a braille display) using the new keyboardHandler.injectRawKeyboardInput function.

Re #4576.

What about merging that for 2015.2?

Comment 23 by jteh on 2015-02-25 10:50
I'll aim to get it into 2015.2. As I said, there's still work to be done. Specifically, this should probably accept vkCode as well as scanCode, as there may be devices which provide vk codes.

Comment 24 by halim (in reply to comment 23) on 2015-02-27 07:01
Replying to jteh:

I'll aim to get it into 2015.2. As I said, there's still work to be done. Specifically, this should probably accept vkCode as well as scanCode, as there may be devices which provide vk codes.

Ok this makes sense. Perhaps the function needs an aditional boolean parameter which specifies the given code.

Comment 25 by halim on 2015-04-18 12:36
Is there any progress? I must return the display back to papenmeier soon.

Comment 26 by jteh on 2015-04-20 06:32
Sorry. I haven't had a chance to look at this, but am trying to prioritise it now given that this is time sensitive.

I think I've fixed the problems with the alt/Windows/applications keys in b210f340. Note that according to Microsoft documentation, doing this for extended keys only works on Windows Vista and later. There's nothing I can do about this.

Comment 27 by jteh on 2015-04-20 06:36
Btw, I think I was wrong in comment:4 about there only being one type of extended key in set 1. Windows only tells us about one type when we receive keys, but I think we lose some information. However, it seems that there aren't any keys other than the pause key which actually use the second type, and the pause key is a 6 byte sequence anyway, making it impossible to inject like this.

Comment 28 by halim on 2015-04-23 06:59
Thx. It's working now correct for me!.
I.ll open a new ticket and attach the patch for the papenmeier braille driver.

Comment 29 by jteh on 2015-05-11 04:22
Changes:
Milestone changed from None to 2015.3

Comment 33 by James Teh <jamie@... on 2015-06-29 01:46
In [3f71be7]:

Merge branch 't5181' into next

Incubates #5181, #4576.

Changes:
Added labels: incubating

Comment 34 by James Teh <jamie@... on 2015-07-14 06:01
In [1973d33]:

You can now injet raw input from a system keyboard that is not handled natively by Windows (e.g. a QWERTY keyboard on a braille display) using the new keyboardHandler.injectRawKeyboardInput function.

Fixes #4576.

Changes:
Removed labels: incubating
State: closed