[WIP] WASM demo #1746

JasperDeSutter · 2019-09-01T22:57:03Z

This is a stab at #1007, having a monaco editor with rust-analyzer running in WASM.

features:

Issues when running ra_ide_api in wasm:

std::time::Instant does not work -> replaced by shim for feature "wasm"
rayon does not work -> replaced by standard iterators for feature "wasm"

as for the implementation:

Started from a standard wasm-bindgen template, as explained here: https://rustwasm.github.io/docs/book/game-of-life/hello-world.html

Currently the code is executed on the main thread (not a webworker), this might be added later when things become too slow. Also, when trying this before, there was a webpack issue which generated a wrong bundle for webworkers.

value passing JS -> WASM happens through arguments for ease of use, and WASM -> JS uses serde_wasm_bindgen. This was the most ergonomic way to work with structured data, but might leave some performance on the table.

logs are filtered to Level::Warn, because the abundance of Info logging really slows down everything a lot though the console_log logger.

It's a pain to try to figure out monaco's API's and how they differ from the vscode/LSP ones. The bad documentation makes me resort to plowing through the npm package code...

Still very much work-in-progress here, but I'm looking for feedback already to be sure.

matklad · 2019-09-02T10:07:45Z

crates/ra_ide_api/src/change.rs

@@ -134,7 +133,7 @@ impl LibraryData {
        root_id: SourceRootId,
        files: Vec<(FileId, RelativePathBuf, Arc<String>)>,
    ) -> LibraryData {
-        let symbol_index = SymbolIndex::for_files(files.par_iter().map(|(file_id, _, text)| {
+        let symbol_index = SymbolIndex::for_files(files.iter().map(|(file_id, _, text)| {


I feel like we should just #[cfg(feature = "wasm")] this: rayon is pretty important for perf

you mean target_arch="wasm32"?

Hm, good question....

I feel like it's better to use Cargo feature, but that's just a gut feeling, so I am fine either way!

crates/ra_ide_api/src/change.rs

matklad · 2019-09-02T10:10:38Z

crates/wasm_demo/src/lib.rs

+    }
+
+    pub fn update(&mut self, code: String) -> JsValue {
+        // TODO: how to update analyisis source?


You'll need AnalysisHost and AnalysisChange for this, but re-creating on every change is surely fine for the first stab!

matklad · 2019-09-02T10:23:21Z

crates/wasm_demo/README.md

+
+cd www
+npm i
+npx webpack-dev-server --host 0.0.0.0


Ideally, we should build a static page and host it on GH pages, like here: https://github.com/matklad/tom/tree/gh-pages

matklad · 2019-09-02T10:25:07Z

I've looked at it, and this looks awesome!

Let's do two things before merging though:

@JasperDeSutter add #[cfg]s for wasm specific things
@matklad create basic infra for gh-pages website

matklad · 2019-09-02T12:13:11Z

#1747 makes the scaffold for gh-pages

bjorn3 · 2019-09-02T19:21:32Z

website/src/wasm-demo/www/example-code.js

+    println!("linked list has length: {}", list.len());
+    println!("{}", list.stringify());
+}
+`


Missing ; and trailing newline.

JasperDeSutter · 2019-09-02T19:22:18Z

wasm-pack will be needed to build the wasm-demo (https://rustwasm.github.io/wasm-pack/installer/), where should this be added? As is node/npm, but I guess this is already there for vscode tests.

matklad · 2019-09-02T19:58:01Z

Yeah, we already require node, so we can assume it exists. Wasm pack should be installed by website gen using cargo install

bjorn3 · 2019-09-02T20:23:16Z

website/src/wasm-demo/www/index.js

+    let allTokens = []
+
+    function update() {
+        console.warn("update")


Could you please end statements with ;? While javascript allows it, only npm doesnt use them.

Most of the JS code is still formatted/structured badly, heck it even has warn's for logging and commented-out code.
The whole file needs some serious cleaning, but I was planning to do this after implementing most of the demo-able functionality.

killercup · 2019-09-05T07:20:52Z

website/website-gen/src/main.rs

+    // install wasm-pack if not available
+    let res = Command::new("wasm-pack").arg("--version").status();
+    if res.is_err() {
+        let status = cargo().args(&["install", "wasm-pack"]).status()?;


I would suggest downloading https://github.com/rustwasm/wasm-pack/releases/download/v0.8.1/wasm-pack-v0.8.1-x86_64-unknown-linux-musl.tar.gz instead and saving a lot of time when it is not cached. You can even easily check the version and there are binaries for other platforms too. This way you can also install it to a specific location and execute it from there so you know the version matches what you expect.

The flip side is that we'd have to implement our own caching logic, to avoid download on every local build. I think it makes sense to defer these concerns to cargo install and travis own caching.

matklad · 2019-09-05T09:10:36Z

Ok, I think this looks good enough to lend now, but the branch needs rebasing

flodiebold · 2019-09-05T11:10:48Z

website/src/wasm-demo/www/index.js

@@ -0,0 +1,202 @@
+// @ts-check


since we're running this through webpack anyway, why not use TS? (that shouldn't block this PR though 🙂 )

Yeah, that’s actually a really great idea! I’d love to get rid of webpack if possible though, but I guess Monaco needs it?

Yeah, TypeScript makes more sense indeed. I just didn't want to add the extra dependencies to ts-loader + extra config required to make TS work for now.
As for webpack, monaco does require webpack-like semantics, such as importing CSS files into JavaScript. This can probably be done with another bundler instead, but probably won't work without any tool (or just tsc).

matklad · 2019-09-06T18:27:14Z

@JasperDeSutter looks like this still has some conflicts. Perhaps it would be easier to squash down the history here to a single commit and rebase that?

JasperDeSutter · 2019-09-07T08:06:04Z

@matklad done, everything should be up-to-date now

matklad · 2019-09-09T08:47:19Z

Tested it locally, and it works! However, given the number of deps here and the size of the .wasm file, I feel it makes sense to move the whole website do the separate repository. I'll try to look into this today

CryZe · 2019-09-09T08:49:51Z

crates/ra_ide_api/Cargo.toml

@@ -4,6 +4,9 @@ name = "ra_ide_api"
 version = "0.1.0"
 authors = ["rust-analyzer developers"]

+[features]
+wasm = []


Is there a reason this is a feature? You should be able to just check for wasm via a normal cfg based on the target?!

#1746 (comment)

Not sure which is better, but, with a feature, we can check that wasm build actually builds without needing to compile to wasm

Ah, looks like you already had a discussion about this further up.

matklad · 2019-09-09T16:28:27Z

I'll try to look into this today

Nope, I won't get to this today sadly :(

1888: allow compiling ra_ide_api on wasm r=matklad a=matklad bors r+ this is from #1746 Co-authored-by: Aleksey Kladov <[email protected]>

@JasperDeSutter

all credit goes to @JasperDeSutter rust-lang/rust-analyzer#1746

@JasperDeSutter

all credit goes to @JasperDeSutter rust-lang/rust-analyzer#1746

@JasperDeSutter

all credit goes to @JasperDeSutter rust-lang/rust-analyzer#1746

@JasperDeSutter

all credit goes to @JasperDeSutter rust-lang/rust-analyzer#1746

@JasperDeSutter

all credit goes to @JasperDeSutter rust-lang/rust-analyzer#1746

matklad · 2019-09-20T20:17:38Z

finally got to this!

https://rust-analyzer.github.io/wasm-demo.html

I've moved it to the separate repo, to avoid adding mulimegabyte blobs to the git history:

https://github.com/rust-analyzer/rust-analyzer.github.io

The setup is pretty horrible and could be improved

matklad · 2019-09-20T20:17:56Z

Thank you so much for bringing this all together @JasperDeSutter !

lnicola · 2019-09-20T20:24:55Z

That's pretty cool, but does it work? I can't get completion to yield anything useful, not even for the List type in the demo:

Also, wheel scroll doesn't work in Firefox, but that's probably an editor issue.

matklad · 2019-09-20T20:47:49Z

I think completion is one of the many things that is not implemented. It would be cool to get completion, go to definition and file outline working though!

…

On Fri, 20 Sep 2019 at 23:24, Laurențiu Nicola ***@***.***> wrote: That's pretty cool, but does it work? I can't get completion to yield anything useful, not even for the List type in the demo: [image: image] <https://user-images.githubusercontent.com/308347/65356695-c59b2a00-dbfd-11e9-93ff-fd93fdcc14ab.png> [image: image] <https://user-images.githubusercontent.com/308347/65356709-d0ee5580-dbfd-11e9-9e32-40224b7d84a6.png> [image: image] <https://user-images.githubusercontent.com/308347/65356725-d6e43680-dbfd-11e9-9014-fbc8ce7006b2.png> — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#1746>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AANB3M27ZHLROZWP3YGXGMLQKUWRRANCNFSM4ISYHU6Q> .

@JasperDeSutter

all credit goes to @JasperDeSutter rust-lang/rust-analyzer#1746

matklad reviewed Sep 2, 2019

View reviewed changes

crates/ra_ide_api/src/change.rs Show resolved Hide resolved

matklad reviewed Sep 2, 2019

View reviewed changes

bjorn3 reviewed Sep 2, 2019

View reviewed changes

killercup reviewed Sep 5, 2019

View reviewed changes

flodiebold reviewed Sep 5, 2019

View reviewed changes

add initial WASM demo

2abc34b

JasperDeSutter force-pushed the wasm-demo branch from 4661321 to 2abc34b Compare September 7, 2019 08:03

CryZe reviewed Sep 9, 2019

View reviewed changes

matklad mentioned this pull request Sep 20, 2019

allow compiling ra_ide_api on wasm #1888

Merged

bors bot added a commit that referenced this pull request Sep 20, 2019

Merge #1888

07bbe57

1888: allow compiling ra_ide_api on wasm r=matklad a=matklad bors r+ this is from #1746 Co-authored-by: Aleksey Kladov <[email protected]>

matklad added a commit to rust-analyzer/rust-analyzer.github.io that referenced this pull request Sep 20, 2019

add wasm demo

b5b5fa2

all credit goes to @JasperDeSutter rust-lang/rust-analyzer#1746

matklad added a commit to rust-analyzer/rust-analyzer.github.io that referenced this pull request Sep 20, 2019

add wasm demo

3faf01e

all credit goes to @JasperDeSutter rust-lang/rust-analyzer#1746

matklad added a commit to rust-analyzer/rust-analyzer.github.io that referenced this pull request Sep 20, 2019

add wasm demo

d5ab238

all credit goes to @JasperDeSutter rust-lang/rust-analyzer#1746

matklad added a commit to rust-analyzer/rust-analyzer.github.io that referenced this pull request Sep 20, 2019

add wasm demo

e3c3220

all credit goes to @JasperDeSutter rust-lang/rust-analyzer#1746

matklad added a commit to rust-analyzer/rust-analyzer.github.io that referenced this pull request Sep 20, 2019

add wasm demo

d192c52

all credit goes to @JasperDeSutter rust-lang/rust-analyzer#1746

matklad closed this Sep 20, 2019

matklad added a commit to rust-analyzer/rust-analyzer-wasm that referenced this pull request Dec 23, 2019

add wasm demo

7175b14

all credit goes to @JasperDeSutter rust-lang/rust-analyzer#1746

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] WASM demo #1746

[WIP] WASM demo #1746

JasperDeSutter commented Sep 1, 2019 •

edited

Loading

matklad Sep 2, 2019

bjorn3 Sep 2, 2019

matklad Sep 2, 2019

matklad Sep 2, 2019

matklad Sep 2, 2019

matklad commented Sep 2, 2019

matklad commented Sep 2, 2019

bjorn3 Sep 2, 2019

JasperDeSutter commented Sep 2, 2019

matklad commented Sep 2, 2019

bjorn3 Sep 2, 2019

JasperDeSutter Sep 2, 2019

killercup Sep 5, 2019 •

edited

Loading

matklad Sep 5, 2019

matklad commented Sep 5, 2019

flodiebold Sep 5, 2019

matklad Sep 5, 2019

JasperDeSutter Sep 5, 2019

matklad commented Sep 6, 2019

JasperDeSutter commented Sep 7, 2019

matklad commented Sep 9, 2019

CryZe Sep 9, 2019

matklad Sep 9, 2019

CryZe Sep 9, 2019

matklad commented Sep 9, 2019

matklad commented Sep 20, 2019

matklad commented Sep 20, 2019

lnicola commented Sep 20, 2019 •

edited

Loading

matklad commented Sep 20, 2019 via email

[WIP] WASM demo #1746

[WIP] WASM demo #1746

Conversation

JasperDeSutter commented Sep 1, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

matklad commented Sep 2, 2019

matklad commented Sep 2, 2019

Choose a reason for hiding this comment

JasperDeSutter commented Sep 2, 2019

matklad commented Sep 2, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

killercup Sep 5, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

matklad commented Sep 5, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

matklad commented Sep 6, 2019

JasperDeSutter commented Sep 7, 2019

matklad commented Sep 9, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

matklad commented Sep 9, 2019

matklad commented Sep 20, 2019

matklad commented Sep 20, 2019

lnicola commented Sep 20, 2019 • edited Loading

matklad commented Sep 20, 2019 via email

JasperDeSutter commented Sep 1, 2019 •

edited

Loading

killercup Sep 5, 2019 •

edited

Loading

lnicola commented Sep 20, 2019 •

edited

Loading