JRFC 19 - Let's Stop Installing Packages

[jbenet]

Today's paradigm includes installing software. It's really silly, having to go find a particular package, and then download it manually. Our package managers should just make the code available. If it can be found in the registry, it should be importable in the code.

How?

Easy, mount the registry:

/npm/<module-name>/<version>

Or, in my world:

/ipns/npmjs.org/<module-name>@<version>

Concerns

What about Security? is this safe!?

Security is not about installing software X at time Y, but about checking integrity (hash the code) and authenticity (sign the code). This could be done on import, every single time you run the code, which would be much safer than just hoping all your files are the same as when you last looked at them. You did look at all the modules you imported, right? You are sure that foobar module you used doesn't actually open a back door, right?

But what about production? Will I have to randomly download modules?

Not at all, things will be cached locally, and just make sure things stay local, why not pin them?

> mounted-npm pin module-i-care-about

Basically, make your "mounted registry" save things locally that you're going to use regularly. (IPFS will do this for you).

But, maybe I want to version lock?

And you should! Lock your local files to exactly the modules you want:

> echo [email protected] >> modules-i-want
> echo [email protected] >> modules-i-want
> cat modules-i-want | mounted-npm lock
> mounted-npm ls
[email protected]
[email protected]

[jbenet]

@groundwater you'll want this in NodeOS, i think :) -- give me two weeks and IPFS may be stable enough for it.

[groundwater]

This is kind of like a lazy install of modules.

I'm also interested in what things we can dream up when file-systems and mounts are easy and safe to hack on. Kinda like FUSE, but without the constant segfaults 😄

[jbenet]

Kinda like FUSE, but without the constant segfaults

FUSE gives you segfaults!? I've yet to see this-- perhaps I'll run into it shortly. Is this osx or linux?

[groundwater]

It gives me segfaults because I'm causing them. Basically I'm a shitty FUSE developer =]

[mlovci]

http://xkcd.com/1367/

[jbenet]

@mlovci precisely. The divide between web and fs is really stupid and it's time to end it. We don't do http requests on ever file open, and with good reason. Need a better protocol :)

[jbenet]

@groundwater try fuse4js? shouldn't be able to segfault :)

[groundwater]

Retrieving and caching content is pretty straight forward. How would you deal with directory listing?

[jbenet]

@groundwater I wouldn't. Don't list root (-r+x on dir). Do list under keywords/, author/<username>/, etc.

[groundwater]

Would you use FUSE?

[jbenet]

More like, i will use FUSE ;)

[groundwater]

Too bad we can't just patch the fs module.

[jbenet]

For @mikolalysenko:

Below, it could easily be foo@version or foo/version. I'm opting for foo@version because i like it more. You can also keep the node_modules/ but i took it out below. Con

/ipns/npmjs.org/package/[email protected]
/ipns/npmjs.org/package/[email protected]
/ipns/npmjs.org/package/[email protected] 
/ipns/npmjs.org/package/[email protected]/[email protected]  (same object as .../[email protected] above)
/ipns/npmjs.org/package/[email protected] 
/ipns/npmjs.org/package/[email protected]/[email protected] (same as .../[email protected])
/ipns/npmjs.org/package/[email protected]/[email protected] (same as .../[email protected])
/ipns/npmjs.org/package/[email protected]/[email protected] (same as .../[email protected])

[jbenet]

cc @mafintosh and @maxogden

[mafintosh]

@jbenet wouldn't I need to list the /ipns/npmjs.org/package (which can be HUGE) to match a semver using the above approach?

[jbenet]

@mafintosh where match here means resolve something like >=1.2.3 ? Ah, then maybe we do want name/version in top level:

/ipns/npmjs.org/package/foo/1.0.0
/ipns/npmjs.org/package/foo/1.0.1
/ipns/npmjs.org/package/bar/1.0.0/foo@>=1.0.0  # only needs to list /ipns/npmjs.org/package/foo/
/ipns/npmjs.org/package/baz/1.0.0/foo@^1.0.1

[mafintosh]

Yes this makes it a lot easier :) I would probably also not include sub dependencies. I don't really see the benefit of having them.

[jbenet]

@mafintosh we get subdependencies for free.

/ipns/npmjs.org/package/foo/1.0.0
/ipns/npmjs.org/package/bar/1.0.0                         
/ipns/npmjs.org/package/bar/1.0.0/foo@>=1.0.0
/ipns/npmjs.org/package/bork/1.0.0/bar@>=1.0.0  
# \o this resolves to bar/1.0.0 above, which already has foo@>=1.0.0

content-addressed deduplication :) !

[mafintosh]

Nice!!

[jonathanmarvens]

Random stranger stopping by to say that these are awesome thoughts :) .

@jbenet

- Jonathan

[silky]

check the nix package managed and nixos - http://nixos.org/ - https://nixos.org/nix/

[BillDStrong]

Why would this only be at that layer?

If you want to go that route, create a Linux distro that mounts IPFS early enough in the chain, then symlink all the normal filenames to IPNS addresses that always have the latest released version. Instant always up to date system.

[burdges]

You need reproducible builds to do this securely, along with Guix or NixOS tool for keeping package options straight.