Globs on Windows PowerShell not working correctly?

[watson]

I'm trying to run the following command in Windows PowerShell:

dependency-check . --unused --no-dev --entry lib/instrumentation/modules/*

But I get the following error:

Cannot find module 'C:\path\to\cwd\lib\instrumentation\modules\*' from 'C:\path\to\cwd\lib\instrumentation\modules'

[watson]

I've tried different the following variations of the glob to see if it might be related to that:

• lib\instrumentation\modules\* - backslash instead of slash
• lib\instrumentation\modules\*.js - append .js

But I get the same error

[watson]

Forgot to mention, that if I change the * to a file that exists in that folder, it works fine. So it is somehow related to the glob

[watson]

Ok, this actually seems to be an issue with Windows not properly supporting globs - at least when npm scripts are involved. Tools like eslint are getting around this problem by using node-glob to parse the path, which means the developer should supply it like a string instead to be cross platform:

dependency-check . --unused --no-dev --entry 'lib/instrumentation/modules/*'

For more info see:

• https://stackoverflow.com/questions/27594550/glob-wildcards-in-windows-npm/30114333
• https://medium.com/@jakubsynowiec/you-should-always-quote-your-globs-in-npm-scripts-621887a2a784

[voxpelli]

A note on the use of --entry here – it isn't needed when adding multiple files in the way you currently do, one can just do dependency-check package.json entry1.js entry2.js and so on. The --entry will only apply to the first file resolved by the lib/instrumentation/modules/* glob pattern as it is now.

As dependency-check itself doesn't do any or support any globbing today, but instead rather support an input format that's compatible with the shells own globbing support, I'm not sure how to best solve this.

Ideally it could be solved independently of this module, but from the articles you link I get the feeling that it's not really a possibility?

[watson]

Yes, it seems that this is an issue that tools like eslint and the like have all encountered and the preferred solution seems to be to resolve globs manually inside the module instead of relying on the OS to do it as this unfortunately doesn't work reliably across different OS'es/shells (e.g. zsh vs bash vs PowerShell).

If there is an external way to do this that would of course be better, but so far I haven't found one.

[voxpelli]

I guess one possibility could be to accept the list of files from stdin in addition to the current way? So one can resolve the file list through another tool and then just pipe it to this one?

But it would be somewhat unorthodox and break against the pattern established by ESLint etc so maybe we should just implement globbing internally. I'll have to think this through from my side.

Any thoughts @blakeembrey @maxogden?

[watson]

It seems you can use this tool: https://www.npmjs.com/package/glob-cli2

If Windows support something similar to backticks from Linux then you could even run:

dependency-check . --unused --no-dev --entry `glob 'lib/instrumentation/modules/*'`

If not, you can use stdin as you suggested:

glob 'lib/instrumentation/modules/*' | dependency-check . --unused --no-dev

[voxpelli]

@watson One thought – maybe you can save the result of glob in a variable and use that variable?

I often use an npm-script like this:

JS_FILES="*.js lib/*.js" && dependency-check . $JS_FILES && dependency-check . $JS_FILES --unused --no-dev

Perhaps that would be doable in PowerShell as well? Looks like it: https://stackoverflow.com/questions/8097354/how-do-i-capture-the-output-into-a-variable-from-an-external-process-in-powershe

[watson]

@voxpelli "*.js lib/*.js" is just a string and isn't interpreted, so the glob isn't evaluated. Not sure how you would do that?

[voxpelli]

@watson Well, it does work on macOS and Linux and the intention of my script is mostly to share the glob between the two rather than to have it only be interpreted just once.

But I'm not thinking that you would do exactly that, but rather have something evolve so that JS_FILES would have a space separated list of all of the resolved files – then calling dependency-check . $JS_FILES (or whatever the compatible syntax for doing that in PowerShell is) would make it so that it would work right now, with no changes – right?

[watson]

I think reason it works for you is not that it expands the glob into JS_FILES, but because your shell expands them after $JS_FILES is evaluated in the context of the dependency-check command. Try JS_FILES="*.js lib/*.js" && echo $JS_FILES to see what it actually contains. In zsh, to expand the glob and assign the value to an environment variable you'd have to do something like this JS_FILES=(*.js lib/*.js).

But the whole issue unfortunately is that expanding globs are very shell depended and assigning them to environment variables is as well. So coming up with a native shell command that works the same across shells is close to impossible I would guess.

[voxpelli]

@watson This should work for you:

JS_FILES=$(glob 'lib/instrumentation/modules/*') && dependency-check . $JS_FILES --unused --no-dev

It runs glob-cli2 as a sub-expression, assigns the output to JS_FILES and then uses that space-delimited output to assign which files that dependency-check should check.

(An unrelated thing: Also add --no-default-entries if you want to test just those files and no files that dependency-check finds and checks by default)

[watson]

To the best of my Google abilities, there's no way to construct a command like JS_FILES=$(glob 'lib/instrumentation/modules/*') that work on both unix and windows shells

[vweevers]

Tools like tape, ava, standard also resolve globs internally. Any objections to doing the same here?

If not, I'll take a stab at implementing it.

[bcomnes]

Still seeing this issue on windows:

"dependency-check ./package.json --entry 'src/**/!(*.test).js' --unused --missing --no-dev --no-peer -i @oclif/plugin-not-found -i @oclif/config -i @oclif/plugin-help",

> dependency-check ./package.json --entry 'src/**/!(*.test).js' --unused --missing --no-dev --no-peer -i @oclif/plugin-not-found -i @oclif/config -i @oclif/pl
ugin-help

.js' was unexpected at this time.

D:\Users\bret\Docs\netlify-cli>  "D:\Users\bret\Docs\netlify-cli\node_modules\.bin\\node.exe"  "D:\Users\bret\Docs\netlify-cli\node_modules\.bin\\..\dependenc
y-check\cli.js" ./package.json --entry 'src/**/!(*.test).js' --unused --missing --no-dev --no-peer -i @oclif/plugin-not-found -i @oclif/config -i @oclif/plugi
n-help
npm ERR! code ELIFECYCLE
npm ERR! errno 255
npm ERR! [email protected] test:deps: `dependency-check ./package.json --entry 'src/**/!(*.test).js' --unused --missing --no-dev --no-peer -i @oclif/p
lugin-not-found -i @oclif/config -i @oclif/plugin-help`
npm ERR! Exit status 255

Any ideas?