Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

InternalActionID is a hash, which cannot be depended on for equality comparison #13124

Open
jazzdelightsme opened this issue May 18, 2022 · 2 comments
Labels
Area-Settings Issues related to settings and customizability, for console or terminal Issue-Bug It either shouldn't be doing this or needs an investigation. Product-Terminal The new Windows Terminal.
Milestone

Comments

@jazzdelightsme
Copy link
Member

Windows Terminal version

latest

Windows build number

10.0.22621.0

Other Software

No response

Steps to reproduce

From doc\specs\#885 - Terminal Settings Model\Actions Addendum.md:

std::map<KeyChord, InternalActionID> _KeyMap;
std::map<InternalActionID, Command> _ActionMap;

InternalActionID will be a hash of ActionAndArgs such that two ActionAndArgs with the same ShortcutAction and IActionArgs output the same hash value.

A hash function can be used to quickly determine if two items are NOT equal: if they have different hashes, they are not the same.

However, if the hashes are the same, that does not mean the items are equal. It just means that you've had a "hash collision", and need to perform a thorough equality check. But Terminal uses InternalActionID as a dictionary key (and the values are not lists), so if you have a collision, an action gets dropped.

I ran into this trying to add a new action--the hash function was not great, so there were collisions, which resulted in very confusing behavior, since some of my actions appeared to be straight-up do the wrong thing, but not just any wrong thing; but the same thing as some other instance of the command with different arguments!

I was fortunately able to improve the hash function, but this still depends on getting lucky.

Expected Behavior

I expected to be able to add a new ActionAndArgs, even though it had a not-very-good hash function, without some of my actions being mysteriously subsumed by others.

Actual Behavior

Some new actions appeared to do the wrong thing (the same thing as some other action that had different args, but ended up with the same hash value).

@jazzdelightsme jazzdelightsme added the Issue-Bug It either shouldn't be doing this or needs an investigation. label May 18, 2022
@ghost ghost added Needs-Triage It's a new issue that the core contributor team needs to triage at the next triage meeting Needs-Tag-Fix Doesn't match tag requirements labels May 18, 2022
@KalleOlaviNiemitalo
Copy link

Previously discussed in #10341 (comment).

@lhecker
Copy link
Member

lhecker commented May 18, 2022

There's two ways forward IMO:

  • Quick fix: Replace til::hasher's FNV1a algorithm with a modern one. My personal preference is xxhash 3 which has a very low collision rate in line with the birthday paradox. It would effectively reduce the likelihood of this being an issue for a long time with a low development cost. Something more compact, like Murmurhash would largely work the same though...
  • Long term fix: ActionMap.cpp's Hash() functions are already build in terms of til::hasher's generic Write() function, which only accepts binary data. We can make our own, equivalent Write() function which builds a std::vector<std::byte> out of the binary data instead, then we hash the vector and finally keep both, the vector and the memoized hash, around in a struct InternalActionID. The hash would keep most comparisons and hashmap operations cheap, and the vector would serve for quick memcmp-comparisons. InternalActionID can also be aliased as a std::shared_ptr so that the behavior of it as a "cheap" ID would be the same as before.

@lhecker lhecker added Area-Settings Issues related to settings and customizability, for console or terminal Product-Terminal The new Windows Terminal. labels May 19, 2022
@ghost ghost removed the Needs-Tag-Fix Doesn't match tag requirements label May 19, 2022
jazzdelightsme added a commit to jazzdelightsme/terminal that referenced this issue Jul 5, 2022
As described in microsoft#9583, this change implements the legacy conhost "EnableColorSelection" feature.

@zadjii-msft was super nice and provided the outline/plumbing (WinRT classes and such) as a hackathon-type project (thank you!)--a "SelectionColor" runtimeclass, a ColorSelection method on the ControlCore runtimeclass, associated plumbing through the layers; plus the action-and-args plumbing to allow hooking up a basic "ColorSelection" action, which allows you to put actions in your settings JSON like so:

```json
{
    "command":
    {
        "action": "experimental.colorSelection",
        "foreground": "#0f3"
    },
    "keys": "alt+4"
},
```

On top of that foundation, I added a couple of things:
* The ability to specify indexes for colors, in addition to RGB and RRGGBB colors.
  - It's a bit hacky, because there are some conversions that fight against sneaking an "I'm an index" flag in the alpha channel.
* A new "matchMode" parameter on the action, allowing you to say if you want to only color the current selection ("0") or all matches ("1").
  - I made it an int, because I'd like to enable at least one other "match mode" later, but it requires me/someone to fix up search.cpp to handle regex first.
  - Search used an old UIA "ColorSelection" method which was previously `E_NOTIMPL`, but is now wired up. Don't know what/if anything else uses this.
* An uber-toggle setting, "EnableColorSelection", which allows you to set a single `bool` in your settings JSON, to light up all the keybindings you would expect from the legacy "EnableColorSelection" feature:
    - alt+[0..9]: color foreground
    - alt+shift+[0..9]: color foreground, all matches
    - ctrl+[0..9]: color background
    - ctrl+shift+[0..9]: color background, all matches
 * A few of the actions cannot be properly invoked via their keybindings, due to microsoft#13124. `*!*` But they work if you do them from the command palette.
  * If you have "`EnableColorSelection : true`" in your settings JSON, but then specify a different action in your JSON that uses the same key binding as a color selection keybinding, your custom one wins, which I think is the right thing.
* I fixed what I think was a bug in search.cpp, which also affected the legacy EnableColorSelection feature: due to a non-inclusive coordinate comparison, you were not allowed to color a single character; but I see no reason why that should be disallowed. Now you can make all your `X`s red if you like.

"Soft" spots:
* I was a bit surprised at some of the helpers I had to provide in textBuffer.cpp. Perhaps there are existing methods that I didn't find?
* Localization? Because there are so many (40!) actions, I went to some trouble to try to provide nice command/arg descriptions. But I don't know how localization works…
jazzdelightsme added a commit to jazzdelightsme/terminal that referenced this issue Jul 5, 2022
As described in microsoft#9583, this change implements the legacy conhost "EnableColorSelection" feature.

@zadjii-msft was super nice and provided the outline/plumbing (WinRT classes and such) as a hackathon-type project (thank you!)--a "SelectionColor" runtimeclass, a ColorSelection method on the ControlCore runtimeclass, associated plumbing through the layers; plus the action-and-args plumbing to allow hooking up a basic "ColorSelection" action, which allows you to put actions in your settings JSON like so:

```json
{
    "command":
    {
        "action": "experimental.colorSelection",
        "foreground": "#0f3"
    },
    "keys": "alt+4"
},
```

On top of that foundation, I added a couple of things:
* The ability to specify indexes for colors, in addition to RGB and RRGGBB colors.
  - It's a bit hacky, because there are some conversions that fight against sneaking an "I'm an index" flag in the alpha channel.
* A new "matchMode" parameter on the action, allowing you to say if you want to only color the current selection ("0") or all matches ("1").
  - I made it an int, because I'd like to enable at least one other "match mode" later, but it requires me/someone to fix up search.cpp to handle regex first.
  - Search used an old UIA "ColorSelection" method which was previously `E_NOTIMPL`, but is now wired up. Don't know what/if anything else uses this.
* An uber-toggle setting, "EnableColorSelection", which allows you to set a single `bool` in your settings JSON, to light up all the keybindings you would expect from the legacy "EnableColorSelection" feature:
    - alt+[0..9]: color foreground
    - alt+shift+[0..9]: color foreground, all matches
    - ctrl+[0..9]: color background
    - ctrl+shift+[0..9]: color background, all matches
 * A few of the actions cannot be properly invoked via their keybindings, due to microsoft#13124. `*!*` But they work if you do them from the command palette.
  * If you have "`EnableColorSelection : true`" in your settings JSON, but then specify a different action in your JSON that uses the same key binding as a color selection keybinding, your custom one wins, which I think is the right thing.
* I fixed what I think was a bug in search.cpp, which also affected the legacy EnableColorSelection feature: due to a non-inclusive coordinate comparison, you were not allowed to color a single character; but I see no reason why that should be disallowed. Now you can make all your `X`s red if you like.

"Soft" spots:
* I was a bit surprised at some of the helpers I had to provide in textBuffer.cpp. Perhaps there are existing methods that I didn't find?
* Localization? Because there are so many (40!) actions, I went to some trouble to try to provide nice command/arg descriptions. But I don't know how localization works…
jazzdelightsme added a commit to jazzdelightsme/terminal that referenced this issue Jul 9, 2022
As described in microsoft#9583, this change implements the legacy conhost "EnableColorSelection" feature.

@zadjii-msft was super nice and provided the outline/plumbing (WinRT classes and such) as a hackathon-type project (thank you!)--a "SelectionColor" runtimeclass, a ColorSelection method on the ControlCore runtimeclass, associated plumbing through the layers; plus the action-and-args plumbing to allow hooking up a basic "ColorSelection" action, which allows you to put actions in your settings JSON like so:

```json
{
    "command":
    {
        "action": "experimental.colorSelection",
        "foreground": "#0f3"
    },
    "keys": "alt+4"
},
```

On top of that foundation, I added a couple of things:
* The ability to specify indexes for colors, in addition to RGB and RRGGBB colors.
  - It's a bit hacky, because there are some conversions that fight against sneaking an "I'm an index" flag in the alpha channel.
* A new "matchMode" parameter on the action, allowing you to say if you want to only color the current selection ("0") or all matches ("1").
  - I made it an int, because I'd like to enable at least one other "match mode" later, but it requires me/someone to fix up search.cpp to handle regex first.
  - Search used an old UIA "ColorSelection" method which was previously `E_NOTIMPL`, but is now wired up. Don't know what/if anything else uses this.
* An uber-toggle setting, "EnableColorSelection", which allows you to set a single `bool` in your settings JSON, to light up all the keybindings you would expect from the legacy "EnableColorSelection" feature:
    - alt+[0..9]: color foreground
    - alt+shift+[0..9]: color foreground, all matches
    - ctrl+[0..9]: color background
    - ctrl+shift+[0..9]: color background, all matches
 * A few of the actions cannot be properly invoked via their keybindings, due to microsoft#13124. `*!*` But they work if you do them from the command palette.
  * If you have "`EnableColorSelection : true`" in your settings JSON, but then specify a different action in your JSON that uses the same key binding as a color selection keybinding, your custom one wins, which I think is the right thing.
* I fixed what I think was a bug in search.cpp, which also affected the legacy EnableColorSelection feature: due to a non-inclusive coordinate comparison, you were not allowed to color a single character; but I see no reason why that should be disallowed. Now you can make all your `X`s red if you like.

"Soft" spots:
* I was a bit surprised at some of the helpers I had to provide in textBuffer.cpp. Perhaps there are existing methods that I didn't find?
* Localization? Because there are so many (40!) actions, I went to some trouble to try to provide nice command/arg descriptions. But I don't know how localization works…
jazzdelightsme added a commit to jazzdelightsme/terminal that referenced this issue Jul 9, 2022
As described in microsoft#9583, this change implements the legacy conhost "EnableColorSelection" feature.

@zadjii-msft was super nice and provided the outline/plumbing (WinRT classes and such) as a hackathon-type project (thank you!)--a "SelectionColor" runtimeclass, a ColorSelection method on the ControlCore runtimeclass, associated plumbing through the layers; plus the action-and-args plumbing to allow hooking up a basic "ColorSelection" action, which allows you to put actions in your settings JSON like so:

```json
{
    "command":
    {
        "action": "experimental.colorSelection",
        "foreground": "#0f3"
    },
    "keys": "alt+4"
},
```

On top of that foundation, I added a couple of things:
* The ability to specify indexes for colors, in addition to RGB and RRGGBB colors.
  - It's a bit hacky, because there are some conversions that fight against sneaking an "I'm an index" flag in the alpha channel.
* A new "matchMode" parameter on the action, allowing you to say if you want to only color the current selection ("0") or all matches ("1").
  - I made it an int, because I'd like to enable at least one other "match mode" later, but it requires me/someone to fix up search.cpp to handle regex first.
  - Search used an old UIA "ColorSelection" method which was previously `E_NOTIMPL`, but is now wired up. Don't know what/if anything else uses this.
* An uber-toggle setting, "EnableColorSelection", which allows you to set a single `bool` in your settings JSON, to light up all the keybindings you would expect from the legacy "EnableColorSelection" feature:
    - alt+[0..9]: color foreground
    - alt+shift+[0..9]: color foreground, all matches
    - ctrl+[0..9]: color background
    - ctrl+shift+[0..9]: color background, all matches
 * A few of the actions cannot be properly invoked via their keybindings, due to microsoft#13124. `*!*` But they work if you do them from the command palette.
  * If you have "`EnableColorSelection : true`" in your settings JSON, but then specify a different action in your JSON that uses the same key binding as a color selection keybinding, your custom one wins, which I think is the right thing.
* I fixed what I think was a bug in search.cpp, which also affected the legacy EnableColorSelection feature: due to a non-inclusive coordinate comparison, you were not allowed to color a single character; but I see no reason why that should be disallowed. Now you can make all your `X`s red if you like.

"Soft" spots:
* I was a bit surprised at some of the helpers I had to provide in textBuffer.cpp. Perhaps there are existing methods that I didn't find?
* Localization? Because there are so many (40!) actions, I went to some trouble to try to provide nice command/arg descriptions. But I don't know how localization works…
jazzdelightsme added a commit to jazzdelightsme/terminal that referenced this issue Aug 1, 2022
As described in microsoft#9583, this change implements the legacy conhost "EnableColorSelection" feature.

@zadjii-msft was super nice and provided the outline/plumbing (WinRT classes and such) as a hackathon-type project (thank you!)--a "SelectionColor" runtimeclass, a ColorSelection method on the ControlCore runtimeclass, associated plumbing through the layers; plus the action-and-args plumbing to allow hooking up a basic "ColorSelection" action, which allows you to put actions in your settings JSON like so:

```json
{
    "command":
    {
        "action": "experimental.colorSelection",
        "foreground": "#0f3"
    },
    "keys": "alt+4"
},
```

On top of that foundation, I added a couple of things:
* The ability to specify indexes for colors, in addition to RGB and RRGGBB colors.
  - It's a bit hacky, because there are some conversions that fight against sneaking an "I'm an index" flag in the alpha channel.
* A new "matchMode" parameter on the action, allowing you to say if you want to only color the current selection ("0") or all matches ("1").
  - I made it an int, because I'd like to enable at least one other "match mode" later, but it requires me/someone to fix up search.cpp to handle regex first.
  - Search used an old UIA "ColorSelection" method which was previously `E_NOTIMPL`, but is now wired up. Don't know what/if anything else uses this.
* An uber-toggle setting, "EnableColorSelection", which allows you to set a single `bool` in your settings JSON, to light up all the keybindings you would expect from the legacy "EnableColorSelection" feature:
    - alt+[0..9]: color foreground
    - alt+shift+[0..9]: color foreground, all matches
    - ctrl+[0..9]: color background
    - ctrl+shift+[0..9]: color background, all matches
 * A few of the actions cannot be properly invoked via their keybindings, due to microsoft#13124. `*!*` But they work if you do them from the command palette.
  * If you have "`EnableColorSelection : true`" in your settings JSON, but then specify a different action in your JSON that uses the same key binding as a color selection keybinding, your custom one wins, which I think is the right thing.
* I fixed what I think was a bug in search.cpp, which also affected the legacy EnableColorSelection feature: due to a non-inclusive coordinate comparison, you were not allowed to color a single character; but I see no reason why that should be disallowed. Now you can make all your `X`s red if you like.

"Soft" spots:
* I was a bit surprised at some of the helpers I had to provide in textBuffer.cpp. Perhaps there are existing methods that I didn't find?
* Localization? Because there are so many (40!) actions, I went to some trouble to try to provide nice command/arg descriptions. But I don't know how localization works…
jazzdelightsme added a commit to jazzdelightsme/terminal that referenced this issue Aug 4, 2022
As described in microsoft#9583, this change implements the legacy conhost "EnableColorSelection" feature.

@zadjii-msft was super nice and provided the outline/plumbing (WinRT classes and such) as a hackathon-type project (thank you!)--a "SelectionColor" runtimeclass, a ColorSelection method on the ControlCore runtimeclass, associated plumbing through the layers; plus the action-and-args plumbing to allow hooking up a basic "ColorSelection" action, which allows you to put actions in your settings JSON like so:

```json
{
    "command":
    {
        "action": "experimental.colorSelection",
        "foreground": "#0f3"
    },
    "keys": "alt+4"
},
```

On top of that foundation, I added a couple of things:
* The ability to specify indexes for colors, in addition to RGB and RRGGBB colors.
  - It's a bit hacky, because there are some conversions that fight against sneaking an "I'm an index" flag in the alpha channel.
* A new "matchMode" parameter on the action, allowing you to say if you want to only color the current selection ("0") or all matches ("1").
  - I made it an int, because I'd like to enable at least one other "match mode" later, but it requires me/someone to fix up search.cpp to handle regex first.
  - Search used an old UIA "ColorSelection" method which was previously `E_NOTIMPL`, but is now wired up. Don't know what/if anything else uses this.
* An uber-toggle setting, "EnableColorSelection", which allows you to set a single `bool` in your settings JSON, to light up all the keybindings you would expect from the legacy "EnableColorSelection" feature:
    - alt+[0..9]: color foreground
    - alt+shift+[0..9]: color foreground, all matches
    - ctrl+[0..9]: color background
    - ctrl+shift+[0..9]: color background, all matches
 * A few of the actions cannot be properly invoked via their keybindings, due to microsoft#13124. `*!*` But they work if you do them from the command palette.
  * If you have "`EnableColorSelection : true`" in your settings JSON, but then specify a different action in your JSON that uses the same key binding as a color selection keybinding, your custom one wins, which I think is the right thing.
* I fixed what I think was a bug in search.cpp, which also affected the legacy EnableColorSelection feature: due to a non-inclusive coordinate comparison, you were not allowed to color a single character; but I see no reason why that should be disallowed. Now you can make all your `X`s red if you like.

"Soft" spots:
* I was a bit surprised at some of the helpers I had to provide in textBuffer.cpp. Perhaps there are existing methods that I didn't find?
ghost pushed a commit that referenced this issue Aug 10, 2022
wyhash was chosen based on the results found in `smhasher`, were it proved
itself as an algorithm with little flaws and fairly high output quality.

While I have a personal preference for xxhash (XXH3 specifically), wyhash is a
better fit for this project as its source code is multiple magnitudes smaller,
simplifying the review and integration into the header-only `hash.h` file.

For use with hashmaps the hash quality doesn't actually matter much for
optimal performance and instead the binary size usually matters more.
But even in that scenario wyhash is fairly close to FNV1a (aka "FNV64").

The result is that this new hash algorithm will only have little impact on
hashmap performance if used over the standard FNV1a as used in the STL,
while simultaneously offering a vastly better hash quality.

This partially solves #13124.

## Validation Steps Performed
* Added test cases ✅
ghost pushed a commit that referenced this issue Aug 31, 2022
## Summary of the Pull Request

As described in #9583, this change implements the legacy conhost "EnableColorSelection" feature.

## Detailed Description of the Pull Request / Additional comments

@zadjii-msft was super nice and provided the outline/plumbing (WinRT classes and such) as a hackathon-type project (thank you!)--a "SelectionColor" runtimeclass, a ColorSelection method on the ControlCore runtimeclass, associated plumbing through the layers; plus the action-and-args plumbing to allow hooking up a basic "ColorSelection" action, which allows you to put actions in your settings JSON like so:

```json
{
    "command":
    {
        "action": "experimental.colorSelection",
        "foreground": "#0f3"
    },
    "keys": "alt+4"
},
```

On top of that foundation, I added a couple of things:
* The ability to specify indexes for colors, in addition to RGB and RRGGBB colors.
  - It's a bit hacky, because there are some conversions that fight against sneaking an "I'm an index" flag in the alpha channel.
* A new "matchMode" parameter on the action, allowing you to say if you want to only color the current selection ("0") or all matches ("1").
  - I made it an int, because I'd like to enable at least one other "match mode" later, but it requires me/someone to fix up search.cpp to handle regex first.
  - Search used an old UIA "ColorSelection" method which was previously `E_NOTIMPL`, but is now wired up. Don't know what/if anything else uses this.
* An uber-toggle setting, "EnableColorSelection", which allows you to set a single `bool` in your settings JSON, to light up all the keybindings you would expect from the legacy "EnableColorSelection" feature:
    - alt+[0..9]: color foreground
    - alt+shift+[0..9]: color foreground, all matches
    - ctrl+[0..9]: color background
    - ctrl+shift+[0..9]: color background, all matches
 * A few of the actions cannot be properly invoked via their keybindings, due to #13124. `*!*` But they work if you do them from the command palette.
  * If you have "`EnableColorSelection : true`" in your settings JSON, but then specify a different action in your JSON that uses the same key binding as a color selection keybinding, your custom one wins, which I think is the right thing.
* I fixed what I think was a bug in search.cpp, which also affected the legacy EnableColorSelection feature: due to a non-inclusive coordinate comparison, you were not allowed to color a single character; but I see no reason why that should be disallowed. Now you can make all your `X`s red if you like.

"Soft" spots:
* I was a bit surprised at some of the helpers I had to provide in textBuffer.cpp. Perhaps there are existing methods that I didn't find?
* Localization? Because there are so many (40!) actions, I went to some trouble to try to provide nice command/arg descriptions. But I don't know how localization works…


<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [x] Closes #9583
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] Tests added/passed *(what would be the right place to add tests for this?)*
* [ ] Documentation updated. If checked, please file a pull request on [our docs repo](https://github.com/MicrosoftDocs/terminal) and link it here: #xxx
* [ ] Schema updated. *(is this needed?)*
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx

## Validation Steps Performed
Just manual testing.
@carlos-zamora carlos-zamora removed the Needs-Triage It's a new issue that the core contributor team needs to triage at the next triage meeting label Aug 2, 2023
@carlos-zamora carlos-zamora added this to the Backlog milestone Aug 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-Settings Issues related to settings and customizability, for console or terminal Issue-Bug It either shouldn't be doing this or needs an investigation. Product-Terminal The new Windows Terminal.
Projects
None yet
Development

No branches or pull requests

4 participants