Skip to content

Commit

Permalink
Discuss async/parallel processing of written/transformed chunks
Browse files Browse the repository at this point in the history
See WICG/serial#165. This ends up adding two pieces of advice:

* One about how producers should wait on the promise returned by writer.write(), and how underlying sinks and transformers can use their promise return values to control that.

* One about how specification authors should avoid reading from chunks in parallel.
  • Loading branch information
domenic authored Sep 6, 2022
1 parent eb1769c commit cdffad6
Showing 1 changed file with 37 additions and 3 deletions.
40 changes: 37 additions & 3 deletions index.bs
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,6 @@ spec:infra; type:dfn; text:list
spec:html; type:dfn; text:entangle
spec:html; type:dfn; text:message port post message steps
spec:html; type:dfn; text:port message queue
# TODO: remove these once whatwg/html#7414 is merged
spec:websockets; type:interface; text:WebSocket
spec:websockets; type:attribute; text:bufferedAmount; for:WebSocket
</pre>

<pre class="anchors">
Expand All @@ -34,6 +31,7 @@ urlPrefix: https://tc39.es/ecma262/; spec: ECMASCRIPT
text: ArrayBuffer; url: #sec-arraybuffer-objects
text: DataView; url: #sec-dataview-objects
text: Number; url: #sec-ecmascript-language-types-number-type
text: SharedArrayBuffer; url: #sec-sharedarraybuffer-objects
text: Uint8Array; url: #sec-typedarray-objects
text: %Object.prototype%; url: #sec-properties-of-the-object-prototype-object
type: dfn
Expand Down Expand Up @@ -4032,6 +4030,12 @@ callback UnderlyingSinkAbortCallback = Promise<undefined> (optional any reason);
its previous value, only increasing to signal the desire for more chunks once the write
succeeds.

<p>Finally, the promise potentially returned by this function is used to ensure that <a
href="#write-mutable-chunks">well-behaved</a> [=producers=] do not attempt to mutate the
[=chunk=] before it has been fully processed. (This is not guaranteed by any specification
machinery, but instead is an informal contract between [=producers=] and the [=underlying
sink=].)

<dt><dfn dict-member for="UnderlyingSink" lt="close">close()</dfn></dt>
<dd>
<p>A function that is called after the [=producer=] signals, via
Expand Down Expand Up @@ -4333,6 +4337,11 @@ following table:
<p>Note that what "success" means is up to the [=underlying sink=]; it might indicate simply that
the [=chunk=] has been accepted, and not necessarily that it is safely saved to its ultimate
destination.

<p id="write-mutable-chunks">If <var ignore>chunk</var> is mutable, [=producers=] are advised to
avoid mutating it after passing it to {{WritableStreamDefaultWriter/write()}}, until after the
promise returned by {{WritableStreamDefaultWriter/write()}} settles. This ensures that the
[=underlying sink=] receives and processes the same value that was passed in.
</dl>

<div algorithm>
Expand Down Expand Up @@ -5463,6 +5472,11 @@ callback TransformerTransformCallback = Promise<undefined> (any chunk, Transform
success or failure of the transformation. A rejected promise will error both the readable and
writable sides of the transform stream.

<p>The promise potentially returned by this function is used to ensure that <a
href="#write-mutable-chunks">well-behaved</a> [=producers=] do not attempt to mutate the [=chunk=]
before it has been fully transformed. (This is not guaranteed by any specification machinery, but
instead is an informal contract between [=producers=] and the [=transformer=].)

<p>If no {{Transformer/transform|transform()}} method is supplied, the identity transform is
used, which enqueues chunks unchanged from the writable side to the readable side.

Expand Down Expand Up @@ -6925,6 +6939,15 @@ for="ReadableStream">locked</dfn> if ! [$IsReadableStreamLocked$](|stream|) retu
|writeAlgorithm|, |closeAlgorithmWrapper|, |abortAlgorithmWrapper|, |highWaterMark|,
|sizeAlgorithm|).

Other specifications should be careful when constructing their
<i>[=WritableStream/set up/writeAlgorithm=]</i> to avoid [=in parallel=] reads from the given
[=chunk=], as such reads can violate the run-to-completion semantics of JavaScript. To avoid this,
they can make a synchronous copy or transfer of the given value, using operations such as
[$StructuredSerializeWithTransfer$], [=get a copy of the bytes held by the buffer source=], or
<a for="ArrayBuffer" lt="transfer">transferring an `ArrayBuffer`</a>. An exception is when the
[=chunk=] is a {{SharedArrayBuffer}}, for which it is understood that parallel mutations are a fact
of life.

<div class="example" id="example-set-up-ws">
Creating a {{WritableStream}} from other specifications is thus a two-step process, like so:

Expand All @@ -6936,6 +6959,8 @@ for="ReadableStream">locked</dfn> if ! [$IsReadableStreamLocked$](|stream|) retu
directly on the [=this=] value inside their constructor steps.</p>
</div>

<hr>

The following definitions must only be used on {{WritableStream}} instances initialized via the
above [=WritableStream/set up=] algorithm:

Expand Down Expand Up @@ -7021,6 +7046,15 @@ reason.
1. Perform ! [$SetUpTransformStreamDefaultController$](|stream|, |controller|,
|transformAlgorithmWrapper|, |flushAlgorithmWrapper|).

Other specifications should be careful when constructing their
<i>[=TransformStream/set up/transformAlgorithm=]</i> to avoid [=in parallel=] reads from the given
[=chunk=], as such reads can violate the run-to-completion semantics of JavaScript. To avoid this,
they can make a synchronous copy or transfer of the given value, using operations such as
[$StructuredSerializeWithTransfer$], [=get a copy of the bytes held by the buffer source=], or
<a for="ArrayBuffer" lt="transfer">transferring an `ArrayBuffer`</a>. An exception is when the
[=chunk=] is a {{SharedArrayBuffer}}, for which it is understood that parallel mutations are a fact
of life.

<div class="example" id="example-set-up-ts">
Creating a {{TransformStream}} from other specifications is thus a two-step process, like so:

Expand Down

0 comments on commit cdffad6

Please sign in to comment.