Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to automatically update add-ons #3208

Open
nvaccessAuto opened this issue May 8, 2013 · 64 comments
Open

Ability to automatically update add-ons #3208

nvaccessAuto opened this issue May 8, 2013 · 64 comments
Assignees
Labels
Addon/management In NVDA management of addons by users. enhancement feature/addon-store Features / behavior of the add-on Store feature p4 https://github.com/nvaccess/nvda/blob/master/projectDocs/issues/triage.md#priority triaged Has been triaged, issue is waiting for implementation.

Comments

@nvaccessAuto
Copy link

Reported by nvdakor on 2013-05-08 07:08
Hi,
With the launch of the official NvDA add-ons page at addons.nvda-project.org, it became easier to search add-ons from one central location. Although this helps with downloading add-ons for the first time, a user who is running an add-on may find that he or she is running an older version of an add-on. So just like automatic NVDA update feature, there could be a feature where newer versions of add-ons would be downloaded automatically and prompt the user to install it without having to visit the addons site. Thanks.
Blocking #4762

@nvaccessAuto
Copy link
Author

Comment 1 by blindbhavya on 2014-09-07 08:27
This proposal would be very useful if implemented.
Could you (the devs James Sir) share your thoughts on this?

@nvaccessAuto
Copy link
Author

Comment 3 by bhavyashah on 2015-01-01 07:44
Hi,
Please add #4762 to 'blocking'.

@nvaccessAuto
Copy link
Author

Comment 5 by fastfinge on 2015-02-19 16:17
As far as I know, we haven't yet had security issues with an addon. But I'm wondering what the security implications of people running out-of-date addons are? Especially as NVDA needs administrator access. This is another reason why, personally, I think having the ability to check for addon updates from within NVDA is kind of important. Even if the addon itself could specify a URL to check for updates, rather than forcing everything through the centralized NVDA addon website, that might be enough to get people updating old addons when, not if, a serious security hole in an addon is discovered. In it's simplest form, it could be a file with a version number, download URL, and file hash. I don't think addons are currently signed via any kind of private key system at the moment, and I'm not sure they urgently need to be. Maybe this isn't a big deal, though, if addons are sandboxed in some way I don't know about?

@nvaccessAuto
Copy link
Author

Comment 6 by jteh on 2015-02-20 01:17
First and most important, NVDA does not require administrator access. You should absolutely not be running NVDA with administrator privileges. You also should not disable UAC.

For what it's worth, no one is disputing that add-on updates would be nice. It's just a matter of someone having the time to do it, including integration with the add-ons repository.

@nvaccessAuto
Copy link
Author

Comment 7 by nvdakor on 2015-04-26 05:54
Hi,
After some investigation, it appears it might be doable. The biggest challenge is the mechanism to integrate with the add-ons repository. Recording the URL and then downloading and instaling the add-on is easy (we can borrow ideas from update check's UpdateDownloader class); the issue is how to differentiate between current and the newer add-on (hashing may work).

The big picture could be the following:

  1. Add an update button in the add-ons manager if the add-on manifest specifies an update URL. This could be either the repurposed URL field from the current manifest format, or it could be a new field just for this purpose.
  2. When the update button is clicked, a routine similar to update check routine from NVDA core would run, attempting to compare hashes and version and so on, which will result in the update UI similar to that of NVDA core.
  3. Once the update is downloaded, the add-on installation UI should come up, asking users if they wish to install the new add-on.

Issues to be resolved:

  • How can we calculate the first hash for our add-ons?
  • How should the web server which hosts add-ons website communicate with NvDA core as to comparing hashes, add-on version and so on?

I think anyone can implement the client side, and Mesar knows more about the server side unless if Jamie has access to that server. We also need to hear from add-on authors so we can prepare our add-ons to take advantage of this idea in the future.
Thanks.

@nvaccessAuto
Copy link
Author

Comment 8 by jteh (in reply to comment 7) on 2015-04-27 03:50
Replying to nvdakor:

Recording the URL and then downloading and instaling the add-on is easy (we can borrow ideas from update check's UpdateDownloader class);

Ideally, that code should be refactored so that it can be reused rather than duplicating it. I started work on this a while ago (#3504), but never got around to finishing it.

the issue is how to differentiate between current and the newer add-on (hashing may work).

Keep it simple. Just compare the old version with the new version. If it's different, consider it an update. This is all we do for NVDA updates. That doesn't account for someone running a versio that is newer than is available on the site, but if that happens, the user clearly is a developer and/or is working with one.

  1. Add an update button in the add-ons manager

We should also consider automatic updates.

if the add-on manifest specifies an update URL.

I'm leaning towards restricting this functionality to the official add-ons repository, which means we don't need a full update URL, just some key indicating that this can do updates.

  • How can we calculate the first hash for our add-ons?

See above. We don't need hashes for version identification. We do need hashes for verifying file integrity, but those can be generated automatically.

Also, I don't think this needs to magically start working for existing add-ons. I think it's acceptable to require that users download a new version of the add-on initially to have it start updating.

  • How should the web server which hosts add-ons website communicate with NvDA core as to comparing hashes, add-on version and so on?

We could do something similar to the current NVDA update check server script. The bigger issue is actually having a database on the server of add-ons and versions. Currently, as I understand it, version information is only listed in the Markdown for the website, which is not something we can reliably use.

I think anyone can implement the client side, and Mesar knows more about the server side unless if Jamie has access to that server.

The entire translation and add-ons system is now all running on the NV Access server. Mesar is pretty busy with other things these days and I haven't seen much from him lately, so I'm now handling everything related to this (unless Mesar wants to step in). I have a pretty good understanding of some parts of the system, though am still learning my way around others.

@nvaccessAuto
Copy link
Author

Comment 9 by nvdakor (in reply to comment 8) on 2015-04-27 03:59
Replying to jteh:

Replying to nvdakor:

Recording the URL and then downloading and instaling the add-on is easy (we can borrow ideas from update check's UpdateDownloader class);

Ideally, that code should be refactored so that it can be reused rather than duplicating it. I started work on this a while ago (#3504), but never got around to finishing it.

If you've started on this, then I think we should wait until #3504 is in place before returning to this ticket. Besides, update downloader and installer GUI has issues - for instance, the GUI for update downloader doesn't close automatically once an update has been downloaded and multiple update progress GUI may appear after a failed update is attempted.

  1. Add an update button in the add-ons manager

We should also consider automatic updates.

if the add-on manifest specifies an update URL.

I'm leaning towards restricting this functionality to the official add-ons repository, which means we don't need a full update URL, just some key indicating that this can do updates.

Understood.

Also, I don't think this needs to magically start working for existing add-ons. I think it's acceptable to require that users download a new version of the add-on initially to have it start updating.

Sure. I'd say we the authors need at least three months time to adjust to the new routines once this ticket has been implemented; whatever major version of the add-on we were working on, that version or the version after that can implement needed changes, similar to what happened last fall when we started implementing add-on help functionality.
Thanks.

@nvaccessAuto
Copy link
Author

Comment 10 by nvdakor on 2015-10-02 01:40
Hi,
Coming back to this ticket...
Looking at urllib and NvDA Core's update routines ghave me an idea as to how to implement client side, but for version 1, it'll require user intervention.
The proposal is as follows:

  1. Add-ons should specify the download URL (perhaps a new key). This can be done easily, as many add-ons come with readme files written in Markdown that specifies URL's for stable and development versions.
  2. When update button is clicked, NVDA will gather the add-on manifest for the selected add-on, which will return add-on version, URL and so on. Perhaps a call to locate the add-on installer size should be specified for dev versions.
  3. NVDA will open the URL via urllib.urlopen, parse the URL and look at content length header. Thankfully, because of the add-on file naming convention in use, it is easy to parse the address looking for the version information (a couple calls to strip function).
  4. If the version on the server is newer than the installed add-on, assume that a newer version is available and present the update dialog (borrow NVDA Core's update presentation dialog, rewording parts of it to refer to add-ons).
  5. If a development version is being updated, check the content length against the size of the add-on installer for the currently installed add-on (initially the client side is 0), and if they are different, assume that a new version is available and act accordingly.
  6. For steps 4 or 5, if an update is downloaded, save the content length for future update lookup.
  7. Until Split out downloader used by updateCheck so it can be used elsewhere #3504 is implemented, NVDA will call os.startfile.
    I'd be happy to write a proof of concept add-on that demonstrates this algorithm.
    Thanks.

@nvaccessAuto
Copy link
Author

Comment 11 by jteh (in reply to comment 10) on 2015-10-02 08:29
Replying to nvdakor:

  1. Add-ons should specify the download URL (perhaps a new key).

I'm still thinking we should restrict this to the official add-on repository. Also, there are actually two URLs required in your proposed system: the URL for the manifest and the URL for the add-on itself.

  1. When update button is clicked, NVDA will gather the add-on manifest for the selected add-on, which will return add-on version, URL and so on.

As noted in comment:8, I think this is better done with a server script similar to the current update check server. This centralises things and allows for tracking of statistics (though that's of course not required for the first round).

  1. If a development version is being updated, check the content length against the size of the add-on installer for the currently installed add-on (initially the client side is 0), and if they are different, assume that a new version is available and act accordingly.

This is going to be fragile. Honestly, I think dev versions should just increment the version number (like NVDA snapshots do) if users really are supposed to update.

Thanks.

@josephsl
Copy link
Collaborator

Hi all,

@derekriemer and I had a talk about how to implement the server side (client side can be done easily). @nvdaes said on the add-ons mailing list that a file that lists add-on keys and versions would be sufficient, but Derek believes a database would be much better (I agree). The questions are what (database), when (if someone has time), and how (procedure).

CC @jcsteh

@derekriemer
Copy link
Collaborator

Hi,
I can open another ticket for this, I'd like an opinion from @jcsteh on this though.
I think when we do this, signing all community add-ons, and only allowing installation of community add-ons until a box is unchecked authorizing that unreviewed add-ons may be installed.
This has these advantages.

  1. Add-ons can check whether the new bundle is properly signed.
  2. When NVDA Boots, it checks the signature of all add-ons if the allow untrusted add-ons option is unchecked. It refuses to load any untrusted add-ons.
  3. When a user tries to install an add-on, if the add-on is not signed, it tells them this is an untrusted andd-on, and prompts them to change their settings if they really want untrusted add-ons.
  4. When updates occur, the add-on bundle must be signed before it will install.

@derekriemer
Copy link
Collaborator

Also, a post request for this would be desired, with all the add-ons being specified. I.E.
{
"rf": {
"version" : "2.2.2",
"beta" : false,
"hash" : "987987987987987987987987"
},
"ocr" : {
"version" : "2.1",
"beta" : false,
"hash" : "d53929098d550098409834409824..."
},
...
}
with json being returned of the form
{
"rf" : {
"version" : 2.5.0",
"beta" : false,
...
}
}

@rkingett
Copy link

Could all the add on's be hosted on Git Hub and then someone uses the Git Hub server to keep the costs down? & would that be an easier way to have NVDA check for updates?

@LeonarddeR
Copy link
Collaborator

This seemed to be quite an interesting ticket to read. Here are my thoughts.

  1. Although I can see why @jcsteh proposes add-ons can only be updated from a central server, this might cause problems for paid add-ons (such as speech synths) which are not in the community repository since they are not or not entirely open source. Even for non-open source add-ons, I think there should be a pathway for automatic updates. Two alternatives:
    A. Only allow third party unregistered add-ons to be updated from a https source with verified SSL certificate
    B. Cumulatively, Allow add-ons from other sources and their corresponding url's to be registered on the centralised add-on system.

@jcsteh
Copy link
Contributor

jcsteh commented Nov 13, 2016

I agree we want paid add-ons to get automatic updates. I imagined we would have them registered with our central repository, even though you probably wouldn't be able to actually purchase them from there.

@feerrenrut feerrenrut added the p4 https://github.com/nvaccess/nvda/blob/master/projectDocs/issues/triage.md#priority label Nov 14, 2016
@feerrenrut
Copy link
Contributor

P3. This seems like it would take a large amount of time to implement, and is not currently stopping primary use cases. It would however be very convenient.

@jcsteh Perhaps the priority should be higher if this is considered of strategic importance to NVDA?

@jcsteh
Copy link
Contributor

jcsteh commented Nov 14, 2016 via email

@rkingett
Copy link

A lot of people have no idea where to get add on updates, though, or
even that an add on has updated, so this would seriously be more than
just a convenience

On 11/13/2016 11:29 PM, Reef Turner wrote:

P3. This seems like it would take a large amount of time to implement,
and is not currently stopping primary use cases. It would however be
very convenient.

@jcsteh https://github.com/jcsteh Perhaps the priority should be
higher if this is considered of strategic importance to NVDA?


You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#3208 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AMj7zJ0QpOnkrL1fZnuKML9ZGP0oEp5mks5q9_HAgaJpZM4JXkpI.

@josephsl
Copy link
Collaborator

Hi,

After doing some more research and talking with @derekriemer, it appears the best possible server side implementation would be that of a table that includes info such as add-on name, description, version, download link and hash and so on, with another table reserved strictly for add-on updates. When the client looks up add-on updates, the client should specify add-on update channel, then the server should return update info such as version, download link and hash.

For example, suppose we use the URL of the form: https://addons.nvaccess.org/updateCheck?name=addon_channel

For instance, Windows 10 App Essentials dev version could say:
https://addons.nvaccess.org/updateCheck?name=wintenApps-dev

The server should return nothing if the version is up to date or a response similar to how NVDA Core checks for new versions (version, download link, hash).

For add-ons, a new manifest key named "addon_updateChannel" or something similar could be added, with the default value being "None" (no info). However, before the add-on updates database says that the specified add-on does not exist, the server would assume that one wants to update to the latest stable branch. Ideally, a new add-on that does include update name should be used.

Additional comments are appreciated. Thanks.

@derekriemer
Copy link
Collaborator

derekriemer commented Jan 16, 2017 via email

@jcsteh
Copy link
Contributor

jcsteh commented Jan 16, 2017 via email

josephsl added a commit to josephsl/nvda that referenced this issue Aug 23, 2018
…pdated when update dialog is shown. Re nvaccess#3208.

Add checkboxes (list control) to add-on updates dialog so users can select which add-ons should be updated. By default, checkboxes are not checked, but one can check add-ons one wishes to update.
Also, whenever checkboxes are checked, check if update button should be enabled or not based on if at least one add-on is checked for update installation (via Python's any() function).
josephsl added a commit to josephsl/nvda that referenced this issue Aug 27, 2018
josephsl added a commit to josephsl/nvda that referenced this issue Nov 16, 2018
…p if told to do so via a callback. Re nvaccess#3208.

Possibly due to threading issue: GUI works correctly if queued from the main thread. One way to do this is via a callback that is claled by wx.CallAfter. Thus use an internal callback when NVDA needs to display add-on update results dialog at startup.
josephsl added a commit to josephsl/nvda that referenced this issue Nov 16, 2018
…king turned off. Re nvaccess#3208.

Add a new set in add-on state that'll record add-ons for which update checking should not be done. With this active, add-on update check function will skip add-ons listed in that set.
josephsl added a commit to josephsl/nvda that referenced this issue Nov 16, 2018
…various add-ons. Re nvaccess#3208.

A new button in Add-ons Manager will open a checkable list where users can turn off add-on update checks. To turn update checking off, one would check the box for the add-on in question. Once OK button is clicked, a record of checked add-ons will be saved to add-on state and will be written to disk just to make sure the record is saved in the event of disasters.
josephsl added a commit to josephsl/nvda that referenced this issue Nov 16, 2018
@josephsl
Copy link
Collaborator

Hi,

Update for November 2018: I'm slowly incorporating lessons learned from Add-on Updater add-on into this pull request. at the moment almost all features from that add-on are present except for choosing update channel. most notably, two outstanding issues/requests were done:

  1. Better user experience when add-on update results shows up on startup: when add-on updates were found at startup, NvDA didn't focus on update results dialog; now it will thanks to an internal callback.
  2. Users can now turn off update checking for one or more add-ons. As @jcsteh suggested, add-on state will record a set of add-ons where update checking is disabled. Eventually a new property will be added that'll let NVDA know if the add-on supports update checking. Also, enabling/disabling add-on update checks for various add-ons can be configurable via a new dialog in Add-ons Manager.

Thanks.

josephsl added a commit to josephsl/nvda that referenced this issue Dec 6, 2018
josephsl added a commit to josephsl/nvda that referenced this issue Dec 12, 2018
…ded NVDA 'update' is not required. Re nvaccess#3208.

Add a keyword in UpdateDownader class that will ask the downloader to retrieve file version information. If set to yes, file version info will be retrieved, otherwise this step will be skipped.
The only time the new keyword will be set to False is if updating add-ons, as add-on bundles do not come with file version metadata, although this might change in the future.
josephsl added a commit to josephsl/nvda that referenced this issue Dec 12, 2018
…vaccess#3208.

Add-ons Manger's instlal add-on function can now update add-ons, performing different steps along the way. Also, remote add-on installer will be used to perform add-on updates if the newly added fresh instlal argument is false.

Specificlaly:

* Moved add-on update steps (at least the user interface) from add-on update downloader to Add-ons Manager class, thus unifying the routines.
* A new keyword named 'freshInstall' will be used to tell Add-ons Manager if installation wil involve updating (false) or a complete instlal (true).
* Depending on value of fresh install keyword, different messages will be presented when 'instlaling' add-ons while updating.
* Remote add-on installer has gained 'freshInstall' keyword for use in add-on update only (this keyword may become private if necessary).
* During add-on updates, install confirmation message won't be shown. The only interaction will be when the new add-on release is not compatible with current version of NVDA (see issue 6275 for more information).
@feerrenrut feerrenrut added this to To do in Auto updates for add-on via automation Feb 7, 2019
josephsl added a commit to josephsl/nvda that referenced this issue Feb 13, 2019
josephsl added a commit to josephsl/nvda that referenced this issue Feb 13, 2019
…branch. Re nvaccess#3208.

Specifically:
* Instead of calling remote add-on instlal handler, the new install add-on function will be called.
* The add-oin install logic is now part of the new installAddon function that has been moved to outside of add-on GUI class.
* Due to introduction of restart prompt function, add-on update generator will call the new function instead of going through add-ons manager/close event.
josephsl added a commit to josephsl/nvda that referenced this issue Feb 15, 2019
…r enforced. Re nvaccess#3208.

As part of revised add-on compatibility approach, file version check is no longer performed by update downloader, thus remove this flag from update downloader and subclasses.
josephsl added a commit to josephsl/nvda that referenced this issue Jul 15, 2019
josephsl added a commit to josephsl/nvda that referenced this issue Sep 23, 2019
…t least one add-on is installed. Re nvaccess#3208.

Enable add-on update check settings if add-ons are installed (perviously it was showing all the time due to a logic error regarding add-on list refresh).
@feerrenrut feerrenrut mentioned this issue Aug 4, 2022
5 tasks
@seanbudd seanbudd added feature triaged Has been triaged, issue is waiting for implementation. feature/addon-store Features / behavior of the add-on Store labels Jun 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Addon/management In NVDA management of addons by users. enhancement feature/addon-store Features / behavior of the add-on Store feature p4 https://github.com/nvaccess/nvda/blob/master/projectDocs/issues/triage.md#priority triaged Has been triaged, issue is waiting for implementation.
Projects
No open projects
Development

No branches or pull requests