src: add config file support

[marco-ippolito]

Refs: #53787

Introducing node.json

node --experimental-config-file=noderc.json file.js

Why a config file?

With the introduction of test runner, sea, and other feature that require a lot of flags, a json config flag would improve by a lot the developer experience and increase adoption.
Example:

node --experimental-config-file=noderc.json --test --experimental-test-coverage

{
 "test-coverage-lines": 80,
 "test-coverage-branches": 60
}

Current solutions

1. CLI Flags
2. NODE_OPTIONS
3. --env-file with NODE_OPTIONS

All of these solutions dont have a good developer experience.

Why JSON?

Simply because we already support json parsing (simdjson) and its kinda the default in the js ecosystem.

Do we have to support all the configurations possible?

NO

Not all flag need to be supported by the config file, and not all flags CAN be supported.
The config file will only support flags that can be passed through NODE_OPTIONS:
https://nodejs.org/api/cli.html#node_optionsoptions

How does it work?

This is an early development, it work similarly to the --env-file flag. It will parse the provided file with --experimental-config-file and add to NODE_OPTIONS the recognized flags. Unrecognized flags will be ignored.

Versioning

json_schema!
When we release a new version, we also release a json schema file.

{
  "$schema": "https://nodejs.org/dist/v20.18.3/docs/node_config_json_schema.json",
}

The users can has autocompletion and validation of their config file specific to the version they are using.
Since the config file mirrors the features that the current version has, it would be impossible to provide semver major backwards compatibility (immagine a feature gets removed we also have to remove it from the config file).

Examples in the ecosystem

• deno.json
• bun toml

Security

Just like .env files, Node.js entrusts the user to provide a secure and valid configuration.

@nodejs/tsc for visibility since its a big feature

[nodejs-github-bot]

Review requested:

• @nodejs/gyp
• @nodejs/startup

[anonrig]

I took a somewhat similar approach last year. Happy to talk in private if you need any help. Ref: anonrig#4

[AugustinMauroy]

Should this configuration file include experimental options?
I don't think so, it should just allow config to be used from feature mark to stable.

[AugustinMauroy]

And if this configuration file could support the version of node. It could issue a warning if the wrong version is used for the project.
But that would require node to be able to resolve semver ranges.
And I think that would really add a degree of complexity to the file.

[marco-ippolito]

I don't see the downside of supporting experimental features. Users can validate with the json schema we provide for the version they are using (you can read in the docs how)

[AugustinMauroy]

I think that enabling feature experimental support will increase their adoption, and so in the event of a breaking change, it will impact more users.

[codecov]

Codecov Report

Attention: Patch coverage is 72.48062% with 71 lines in your changes missing coverage. Please review.

Project coverage is 90.35%. Comparing base (3b0fce1) to head (a7ee706).
Report is 10 commits behind head on main.

Files with missing lines	Patch %	Lines
src/node_config_file.cc	63.02%	33 Missing and 11 partials ⚠️
src/node_options.cc	62.50%	14 Missing and 10 partials ⚠️
src/node.cc	83.33%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #57016      +/-   ##
==========================================
- Coverage   90.38%   90.35%   -0.03%     
==========================================
  Files         628      629       +1     
  Lines      184007   184265     +258     
  Branches    35952    36008      +56     
==========================================
+ Hits       166314   166499     +185     
- Misses      10869    10907      +38     
- Partials     6824     6859      +35     

Files with missing lines	Coverage Δ
lib/internal/options.js	100.00% <100.00%> (ø)
lib/internal/process/pre_execution.js	93.31% <100.00%> (+1.65%)	⬆️
src/node_errors.h	85.00% <ø> (ø)
src/node_options.h	98.32% <ø> (ø)
src/node.cc	73.96% <83.33%> (+0.38%)	⬆️
src/node_options.cc	85.32% <62.50%> (-2.05%)	⬇️
src/node_config_file.cc	63.02% <63.02%> (ø)

... and 21 files with indirect coverage changes

[jasnell]

Historically I've been fairly anti-config file but I actually think I've come around to the idea. I do think, however, that we need to be careful about making things too confusing for users with some options being passed as command line flags, others being passed as environment variables, and now others possibly being passed by config file. Ideally these mechanisms would integrate with each other such that we can add a configuration option once and it just works (much like we can do today with adding a command line flag and indicate whether it can be used in NODE_OPTIONS or not). It would even be ideal for us to either be able to add an option to the JSON schema and automatically have the code generated in the runtime to support the option or add the option programmatically and provide a way for node.js to generate the json schema dynamically from that. Either way, I've come around to supporting moving in this direction (even if while kicking and screaming).

[jasnell]

@lemire ... I'm curious, does simdjson support jsonc? (JSON with comments). Would it be difficult to add support for jsonc if not? If we're going to use json as a configuration format, then supporting comments is ideal. (note that both deno and bun support comments in their respective formats)

[marco-ippolito]

I had to rebase after #57142 😭

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/65344/

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/65347/

[nodejs-github-bot]

Landed in 5ab7c4c...772c609

[bengl]

Would it be possible in a future commit to allow for (but maybe not require) options to be namespaced in a "node_options" property?

e.g.

{
  "node_options": {
    "test-coverage-lines": 80,
    "test-coverage-branches": 60
  }
}

This would allow for re-using existing JSON files currently used for other kinds of config.

[marco-ippolito]

Unless we decide to support other use cases in addition to flags, imho it does not make sense to namespace it.

[bengl]

If it's namespaced, I can just use package.json and not need yet another file. I understand that some folks don't like packing extra info in there, but we've already done that with type.

EDIT: After I posted, I realized that as-is I can just use top-level properties in a package.json. Properties aren't likely to collide with other package.json properties, but namespacing them would ensure no collisions.

[targos]

Moving to nodeOptions (camelCase) SGTM, just to make room for future options that can be more ergonomic than a 1-1 mapping to CLI flags.

[BridgeAR]

package.json is already overboarded in my opinion and it is a cleare separation to have Node.js configurations in a file that is actually also meant to be local for each user. I could have local settings that I don't want to share with others. So separating the files is good in my opinion.

[bengl]

@BridgeAR Nothing stops anyone from using separate files if they want to. This PR requires that the file be specified, and namespacing makes it easier me to use the same file for more than one purpose, to avoid having tons of config files all over the root of my project. Having separate files for the reasons you indicated also totally makes sense, but I think either approach should be possible because not all use cases are predictable.

Also, as @targos mentioned, namespacing adds a layer of future-proofing. We don't know what folks will generally want in the future (both as developers of the project and as users), and namespacing makes non-breaking changes more possible.

[BridgeAR]

Should we consider removing the need for the flag? I would personally prefer a fixed file name that is just checked for in the root folder than having it dynamically configured by users. That way more people would likely pick it up seeing the file better and it requires the user to think about it less / it's easier to use.
That way there is also no discussion around namespacing it. It would just be that one file.

[marco-ippolito]

Let's move the discussion to #57170

[GeoffreyBooth]

I didn't notice this PR being opened, much less landing. If there had been a comment posted on #53787 about this, everyone who was following that issue would've been aware of this PR while there was still time to review it. Some of these post-landing concerns such as about namespacing or default filenames could've been addressed before landing.

That said, I don't have any objections to this PR as is, so I'm not asking for it to be reverted or anything like that. Perhaps we should mark it as dont-land-on-v23.x until we can sort through some of the issues that people have brought up above?

[styfle]

Can we remove the duplicate json?

node_config_schema.json

Or better yet, use hyphens

node-config-schema.json

[marco-ippolito]

Good idea