Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(deps): bulk add missing dependencies - 2023-11-02 #2857

Closed
petermetz opened this issue Nov 1, 2023 · 0 comments · Fixed by #2859 or #3324
Closed

fix(deps): bulk add missing dependencies - 2023-11-02 #2857

petermetz opened this issue Nov 1, 2023 · 0 comments · Fixed by #2859 or #3324
Assignees
Labels
API_Server dependencies Pull requests that update a dependency file good-first-issue-400-expert P1 Priority 1: Highest
Milestone

Comments

@petermetz
Copy link
Contributor

Description

On top of the obvious solution of adding the missing dependency to the package.json, we also need to make sure that this can be checked against somehow with tooling and automation.
Up until now I thought that disabling hoisting in yarn completely fixed this issue but it turns out that it's only a partial fix.
The edge case where it still fails is when the root package.json declares a dependency that we then import from a sub-package and the compiler walks up the directory tree to the root, finds the package and is happy. This can happen because the package was in the root node_modules with or without hoisting to begin with.

https://yarnpkg.com/package?name=depcheck

Acceptance Criteria

  1. A one liner or script (preferably a script) that finds these issues
  2. The solution to 1) should also be part of the custom-checks
  3. It should clearly list which package json is missing which dependencies so that taking action is trivial without further investigation having to be performed
  4. In addition to fixing the particular missing dependency in the subject, all other missing dependencies should be fixed as well due to the urgent nature of this problem (packages are being released as broken when this happens)
@petermetz petermetz self-assigned this Nov 1, 2023
@petermetz petermetz added API_Server dependencies Pull requests that update a dependency file good-first-issue-400-expert P1 Priority 1: Highest labels Nov 1, 2023
@petermetz petermetz added this to the v2.0.0 milestone Nov 1, 2023
@petermetz petermetz changed the title fix(cactus-common): missing dependency run-time-error-cjs fix(deps): bulk add missing dependencies - 2023-11-02 Nov 2, 2023
petermetz added a commit to petermetz/cacti that referenced this issue Nov 2, 2023
1. Added missing dependencies everywhere to all the packages
2. The exception to the above is test runners and related dependencies.
You can see the detailed exclusion list in the source code of the script
itself [1]. We should make these configurable later on as well. For
reference, this is what the exclusions were declared as when I ran the
tool in order to then proceed to update the package.json files:

```typescript
ignorePatterns: [
    // files matching these patterns will be ignored
    "sandbox",
    "dist",
    "bower_components",
    "node_modules"
],
ignoreMatches: [
    // ignore dependencies that matches these globs
    "grunt-*",
    "jest-extended",
    "tape-promise",
    "tape",
    "tap",
    "@ionic-native/*"
],
```
3. There were instances of missing dependency usages where we did NOT
have to add the dependencies to package.json files because what we could
do instead is just import types at development time by using the
`import type { .. } from "...";`  syntax of Typescript which means that
the import is disappeared during transpilation completely. So this is why
some source code files were also modified and not strictly just package.json
files.
4. Added a script in the ./tools/custom-check directory to audit the
entire mono-repo for missing NodeJS dependencies.

We've had at least a dozen packages that were missing production
dependency declarations from their package.json files. The usual suspects
here are packages that are contained by the root node_modules folder
which masks the problem at development time (e.g. the compiler won't
let you know about the missing dependencies).

For a future-proof solution we should add a commit hook or other validation
that runs the custom check that I added [2] here to verify that there are no
missing dependencies in the project

[1] `./tools/custom-checks/check-missing-node-deps.ts`
[2] `$ yarn tools:check-missing-node-deps`

Fixes hyperledger-cacti#2857

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this issue Nov 2, 2023
1. Added missing dependencies everywhere to all the packages
2. The exception to the above is test runners and related dependencies.
You can see the detailed exclusion list in the source code of the script
itself [1]. We should make these configurable later on as well. For
reference, this is what the exclusions were declared as when I ran the
tool in order to then proceed to update the package.json files:

```typescript
ignorePatterns: [
    // files matching these patterns will be ignored
    "sandbox",
    "dist",
    "bower_components",
    "node_modules"
],
ignoreMatches: [
    // ignore dependencies that matches these globs
    "grunt-*",
    "jest-extended",
    "tape-promise",
    "tape",
    "tap",
    "@ionic-native/*"
],
```
3. There were instances of missing dependency usages where we did NOT
have to add the dependencies to package.json files because what we could
do instead is just import types at development time by using the
`import type { .. } from "...";`  syntax of Typescript which means that
the import is disappeared during transpilation completely. So this is why
some source code files were also modified and not strictly just package.json
files.
4. One exception to the above is the google-sm-keychain plugin's mock
code where minimal code alternations were necessary to satisfy the compiler.
With that said, no behavioral code change was done here either, just the
elimination of some redundant assignments.
5. Added a script in the ./tools/custom-check directory to audit the
entire mono-repo for missing NodeJS dependencies.

We've had at least a dozen packages that were missing production
dependency declarations from their package.json files. The usual suspects
here are packages that are contained by the root node_modules folder
which masks the problem at development time (e.g. the compiler won't
let you know about the missing dependencies).

For a future-proof solution we should add a commit hook or other validation
that runs the custom check that I added [2] here to verify that there are no
missing dependencies in the project

[1] `./tools/custom-checks/check-missing-node-deps.ts`
[2] `$ yarn tools:check-missing-node-deps`

Fixes hyperledger-cacti#2857

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this issue Nov 2, 2023
1. Added missing dependencies everywhere to all the packages
2. The exception to the above is test runners and related dependencies.
You can see the detailed exclusion list in the source code of the script
itself [1]. We should make these configurable later on as well. For
reference, this is what the exclusions were declared as when I ran the
tool in order to then proceed to update the package.json files:

```typescript
ignorePatterns: [
    // files matching these patterns will be ignored
    "sandbox",
    "dist",
    "bower_components",
    "node_modules"
],
ignoreMatches: [
    // ignore dependencies that matches these globs
    "grunt-*",
    "jest-extended",
    "tape-promise",
    "tape",
    "tap",
    "@ionic-native/*"
],
```
3. There were instances of missing dependency usages where we did NOT
have to add the dependencies to package.json files because what we could
do instead is just import types at development time by using the
`import type { .. } from "...";`  syntax of Typescript which means that
the import is disappeared during transpilation completely. So this is why
some source code files were also modified and not strictly just package.json
files.
4. One exception to the above is the google-sm-keychain plugin's mock
code where minimal code alternations were necessary to satisfy the compiler.
With that said, no behavioral code change was done here either, just the
elimination of some redundant assignments.
5. Added a script in the ./tools/custom-check directory to audit the
entire mono-repo for missing NodeJS dependencies.

We've had at least a dozen packages that were missing production
dependency declarations from their package.json files. The usual suspects
here are packages that are contained by the root node_modules folder
which masks the problem at development time (e.g. the compiler won't
let you know about the missing dependencies).

For a future-proof solution we should add a commit hook or other validation
that runs the custom check that I added [2] here to verify that there are no
missing dependencies in the project

[1] `./tools/custom-checks/check-missing-node-deps.ts`
[2] `$ yarn tools:check-missing-node-deps`

Fixes hyperledger-cacti#2857

Signed-off-by: Peter Somogyvari <[email protected]>
outSH pushed a commit to outSH/cactus that referenced this issue Nov 3, 2023
1. Added missing dependencies everywhere to all the packages
2. The exception to the above is test runners and related dependencies.
You can see the detailed exclusion list in the source code of the script
itself [1]. We should make these configurable later on as well. For
reference, this is what the exclusions were declared as when I ran the
tool in order to then proceed to update the package.json files:

```typescript
ignorePatterns: [
    // files matching these patterns will be ignored
    "sandbox",
    "dist",
    "bower_components",
    "node_modules"
],
ignoreMatches: [
    // ignore dependencies that matches these globs
    "grunt-*",
    "jest-extended",
    "tape-promise",
    "tape",
    "tap",
    "@ionic-native/*"
],
```
3. There were instances of missing dependency usages where we did NOT
have to add the dependencies to package.json files because what we could
do instead is just import types at development time by using the
`import type { .. } from "...";`  syntax of Typescript which means that
the import is disappeared during transpilation completely. So this is why
some source code files were also modified and not strictly just package.json
files.
4. One exception to the above is the google-sm-keychain plugin's mock
code where minimal code alternations were necessary to satisfy the compiler.
With that said, no behavioral code change was done here either, just the
elimination of some redundant assignments.
5. Added a script in the ./tools/custom-check directory to audit the
entire mono-repo for missing NodeJS dependencies.

We've had at least a dozen packages that were missing production
dependency declarations from their package.json files. The usual suspects
here are packages that are contained by the root node_modules folder
which masks the problem at development time (e.g. the compiler won't
let you know about the missing dependencies).

For a future-proof solution we should add a commit hook or other validation
that runs the custom check that I added [2] here to verify that there are no
missing dependencies in the project

[1] `./tools/custom-checks/check-missing-node-deps.ts`
[2] `$ yarn tools:check-missing-node-deps`

Fixes hyperledger-cacti#2857

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this issue Nov 7, 2023
1. Added missing dependencies everywhere to all the packages
2. The exception to the above is test runners and related dependencies.
You can see the detailed exclusion list in the source code of the script
itself [1]. We should make these configurable later on as well. For
reference, this is what the exclusions were declared as when I ran the
tool in order to then proceed to update the package.json files:

```typescript
ignorePatterns: [
    // files matching these patterns will be ignored
    "sandbox",
    "dist",
    "bower_components",
    "node_modules"
],
ignoreMatches: [
    // ignore dependencies that matches these globs
    "grunt-*",
    "jest-extended",
    "tape-promise",
    "tape",
    "tap",
    "@ionic-native/*"
],
```
3. There were instances of missing dependency usages where we did NOT
have to add the dependencies to package.json files because what we could
do instead is just import types at development time by using the
`import type { .. } from "...";`  syntax of Typescript which means that
the import is disappeared during transpilation completely. So this is why
some source code files were also modified and not strictly just package.json
files.
4. One exception to the above is the google-sm-keychain plugin's mock
code where minimal code alternations were necessary to satisfy the compiler.
With that said, no behavioral code change was done here either, just the
elimination of some redundant assignments.
5. Added a script in the ./tools/custom-check directory to audit the
entire mono-repo for missing NodeJS dependencies.

We've had at least a dozen packages that were missing production
dependency declarations from their package.json files. The usual suspects
here are packages that are contained by the root node_modules folder
which masks the problem at development time (e.g. the compiler won't
let you know about the missing dependencies).

For a future-proof solution we should add a commit hook or other validation
that runs the custom check that I added [2] here to verify that there are no
missing dependencies in the project

[1] `./tools/custom-checks/check-missing-node-deps.ts`
[2] `$ yarn tools:check-missing-node-deps`

Fixes hyperledger-cacti#2857

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this issue Nov 10, 2023
1. Added missing dependencies everywhere to all the packages
2. The exception to the above is test runners and related dependencies.
You can see the detailed exclusion list in the source code of the script
itself [1]. We should make these configurable later on as well. For
reference, this is what the exclusions were declared as when I ran the
tool in order to then proceed to update the package.json files:

```typescript
ignorePatterns: [
    // files matching these patterns will be ignored
    "sandbox",
    "dist",
    "bower_components",
    "node_modules"
],
ignoreMatches: [
    // ignore dependencies that matches these globs
    "grunt-*",
    "jest-extended",
    "tape-promise",
    "tape",
    "tap",
    "@ionic-native/*"
],
```
3. There were instances of missing dependency usages where we did NOT
have to add the dependencies to package.json files because what we could
do instead is just import types at development time by using the
`import type { .. } from "...";`  syntax of Typescript which means that
the import is disappeared during transpilation completely. So this is why
some source code files were also modified and not strictly just package.json
files.
4. One exception to the above is the google-sm-keychain plugin's mock
code where minimal code alternations were necessary to satisfy the compiler.
With that said, no behavioral code change was done here either, just the
elimination of some redundant assignments.
5. Added a script in the ./tools/custom-check directory to audit the
entire mono-repo for missing NodeJS dependencies.

We've had at least a dozen packages that were missing production
dependency declarations from their package.json files. The usual suspects
here are packages that are contained by the root node_modules folder
which masks the problem at development time (e.g. the compiler won't
let you know about the missing dependencies).

For a future-proof solution we should add a commit hook or other validation
that runs the custom check that I added [2] here to verify that there are no
missing dependencies in the project

[1] `./tools/custom-checks/check-missing-node-deps.ts`
[2] `$ yarn tools:check-missing-node-deps`

Fixes hyperledger-cacti#2857

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this issue Nov 10, 2023
1. Added missing dependencies everywhere to all the packages
2. The exception to the above is test runners and related dependencies.
You can see the detailed exclusion list in the source code of the script
itself [1]. We should make these configurable later on as well. For
reference, this is what the exclusions were declared as when I ran the
tool in order to then proceed to update the package.json files:

```typescript
ignorePatterns: [
    // files matching these patterns will be ignored
    "sandbox",
    "dist",
    "bower_components",
    "node_modules"
],
ignoreMatches: [
    // ignore dependencies that matches these globs
    "grunt-*",
    "jest-extended",
    "tape-promise",
    "tape",
    "tap",
    "@ionic-native/*"
],
```
3. There were instances of missing dependency usages where we did NOT
have to add the dependencies to package.json files because what we could
do instead is just import types at development time by using the
`import type { .. } from "...";`  syntax of Typescript which means that
the import is disappeared during transpilation completely. So this is why
some source code files were also modified and not strictly just package.json
files.
4. One exception to the above is the google-sm-keychain plugin's mock
code where minimal code alternations were necessary to satisfy the compiler.
With that said, no behavioral code change was done here either, just the
elimination of some redundant assignments.
5. Added a script in the ./tools/custom-check directory to audit the
entire mono-repo for missing NodeJS dependencies.

We've had at least a dozen packages that were missing production
dependency declarations from their package.json files. The usual suspects
here are packages that are contained by the root node_modules folder
which masks the problem at development time (e.g. the compiler won't
let you know about the missing dependencies).

For a future-proof solution we should add a commit hook or other validation
that runs the custom check that I added [2] here to verify that there are no
missing dependencies in the project

[1] `./tools/custom-checks/check-missing-node-deps.ts`
[2] `$ yarn tools:check-missing-node-deps`

Fixes hyperledger-cacti#2857

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this issue Nov 10, 2023
1. Added missing dependencies everywhere to all the packages
2. The exception to the above is test runners and related dependencies.
You can see the detailed exclusion list in the source code of the script
itself [1]. We should make these configurable later on as well. For
reference, this is what the exclusions were declared as when I ran the
tool in order to then proceed to update the package.json files:

```typescript
ignorePatterns: [
    // files matching these patterns will be ignored
    "sandbox",
    "dist",
    "bower_components",
    "node_modules"
],
ignoreMatches: [
    // ignore dependencies that matches these globs
    "grunt-*",
    "jest-extended",
    "tape-promise",
    "tape",
    "tap",
    "@ionic-native/*"
],
```
3. There were instances of missing dependency usages where we did NOT
have to add the dependencies to package.json files because what we could
do instead is just import types at development time by using the
`import type { .. } from "...";`  syntax of Typescript which means that
the import is disappeared during transpilation completely. So this is why
some source code files were also modified and not strictly just package.json
files.
4. One exception to the above is the google-sm-keychain plugin's mock
code where minimal code alternations were necessary to satisfy the compiler.
With that said, no behavioral code change was done here either, just the
elimination of some redundant assignments.
5. Added a script in the ./tools/custom-check directory to audit the
entire mono-repo for missing NodeJS dependencies.

We've had at least a dozen packages that were missing production
dependency declarations from their package.json files. The usual suspects
here are packages that are contained by the root node_modules folder
which masks the problem at development time (e.g. the compiler won't
let you know about the missing dependencies).

For a future-proof solution we should add a commit hook or other validation
that runs the custom check that I added [2] here to verify that there are no
missing dependencies in the project

[1] `./tools/custom-checks/check-missing-node-deps.ts`
[2] `$ yarn tools:check-missing-node-deps`

Fixes hyperledger-cacti#2857

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this issue Nov 14, 2023
1. Added missing dependencies everywhere to all the packages
2. The exception to the above is test runners and related dependencies.
You can see the detailed exclusion list in the source code of the script
itself [1]. We should make these configurable later on as well. For
reference, this is what the exclusions were declared as when I ran the
tool in order to then proceed to update the package.json files:

```typescript
ignorePatterns: [
    // files matching these patterns will be ignored
    "sandbox",
    "dist",
    "bower_components",
    "node_modules"
],
ignoreMatches: [
    // ignore dependencies that matches these globs
    "grunt-*",
    "jest-extended",
    "tape-promise",
    "tape",
    "tap",
    "@ionic-native/*"
],
```
3. There were instances of missing dependency usages where we did NOT
have to add the dependencies to package.json files because what we could
do instead is just import types at development time by using the
`import type { .. } from "...";`  syntax of Typescript which means that
the import is disappeared during transpilation completely. So this is why
some source code files were also modified and not strictly just package.json
files.
4. One exception to the above is the google-sm-keychain plugin's mock
code where minimal code alternations were necessary to satisfy the compiler.
With that said, no behavioral code change was done here either, just the
elimination of some redundant assignments.
5. Added a script in the ./tools/custom-check directory to audit the
entire mono-repo for missing NodeJS dependencies.

We've had at least a dozen packages that were missing production
dependency declarations from their package.json files. The usual suspects
here are packages that are contained by the root node_modules folder
which masks the problem at development time (e.g. the compiler won't
let you know about the missing dependencies).

For a future-proof solution we should add a commit hook or other validation
that runs the custom check that I added [2] here to verify that there are no
missing dependencies in the project

[1] `./tools/custom-checks/check-missing-node-deps.ts`
[2] `$ yarn tools:check-missing-node-deps`

Fixes hyperledger-cacti#2857

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit that referenced this issue Nov 14, 2023
1. Added missing dependencies everywhere to all the packages
2. The exception to the above is test runners and related dependencies.
You can see the detailed exclusion list in the source code of the script
itself [1]. We should make these configurable later on as well. For
reference, this is what the exclusions were declared as when I ran the
tool in order to then proceed to update the package.json files:

```typescript
ignorePatterns: [
    // files matching these patterns will be ignored
    "sandbox",
    "dist",
    "bower_components",
    "node_modules"
],
ignoreMatches: [
    // ignore dependencies that matches these globs
    "grunt-*",
    "jest-extended",
    "tape-promise",
    "tape",
    "tap",
    "@ionic-native/*"
],
```
3. There were instances of missing dependency usages where we did NOT
have to add the dependencies to package.json files because what we could
do instead is just import types at development time by using the
`import type { .. } from "...";`  syntax of Typescript which means that
the import is disappeared during transpilation completely. So this is why
some source code files were also modified and not strictly just package.json
files.
4. One exception to the above is the google-sm-keychain plugin's mock
code where minimal code alternations were necessary to satisfy the compiler.
With that said, no behavioral code change was done here either, just the
elimination of some redundant assignments.
5. Added a script in the ./tools/custom-check directory to audit the
entire mono-repo for missing NodeJS dependencies.

We've had at least a dozen packages that were missing production
dependency declarations from their package.json files. The usual suspects
here are packages that are contained by the root node_modules folder
which masks the problem at development time (e.g. the compiler won't
let you know about the missing dependencies).

For a future-proof solution we should add a commit hook or other validation
that runs the custom check that I added [2] here to verify that there are no
missing dependencies in the project

[1] `./tools/custom-checks/check-missing-node-deps.ts`
[2] `$ yarn tools:check-missing-node-deps`

Fixes #2857

Signed-off-by: Peter Somogyvari <[email protected]>
@github-project-automation github-project-automation bot moved this from In Progress to Done in Cacti_Scrum_Project_v2_Release Nov 14, 2023
sandeepnRES pushed a commit to sandeepnRES/cacti that referenced this issue Dec 21, 2023
1. Added missing dependencies everywhere to all the packages
2. The exception to the above is test runners and related dependencies.
You can see the detailed exclusion list in the source code of the script
itself [1]. We should make these configurable later on as well. For
reference, this is what the exclusions were declared as when I ran the
tool in order to then proceed to update the package.json files:

```typescript
ignorePatterns: [
    // files matching these patterns will be ignored
    "sandbox",
    "dist",
    "bower_components",
    "node_modules"
],
ignoreMatches: [
    // ignore dependencies that matches these globs
    "grunt-*",
    "jest-extended",
    "tape-promise",
    "tape",
    "tap",
    "@ionic-native/*"
],
```
3. There were instances of missing dependency usages where we did NOT
have to add the dependencies to package.json files because what we could
do instead is just import types at development time by using the
`import type { .. } from "...";`  syntax of Typescript which means that
the import is disappeared during transpilation completely. So this is why
some source code files were also modified and not strictly just package.json
files.
4. One exception to the above is the google-sm-keychain plugin's mock
code where minimal code alternations were necessary to satisfy the compiler.
With that said, no behavioral code change was done here either, just the
elimination of some redundant assignments.
5. Added a script in the ./tools/custom-check directory to audit the
entire mono-repo for missing NodeJS dependencies.

We've had at least a dozen packages that were missing production
dependency declarations from their package.json files. The usual suspects
here are packages that are contained by the root node_modules folder
which masks the problem at development time (e.g. the compiler won't
let you know about the missing dependencies).

For a future-proof solution we should add a commit hook or other validation
that runs the custom check that I added [2] here to verify that there are no
missing dependencies in the project

[1] `./tools/custom-checks/check-missing-node-deps.ts`
[2] `$ yarn tools:check-missing-node-deps`

Fixes hyperledger-cacti#2857

Signed-off-by: Peter Somogyvari <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API_Server dependencies Pull requests that update a dependency file good-first-issue-400-expert P1 Priority 1: Highest
Projects
None yet
1 participant