diff --git a/files/en-us/web/api/readablebytestreamcontroller/byobrequest/index.md b/files/en-us/web/api/readablebytestreamcontroller/byobrequest/index.md index 19faf0971e222db..9f8f18eb9076dda 100644 --- a/files/en-us/web/api/readablebytestreamcontroller/byobrequest/index.md +++ b/files/en-us/web/api/readablebytestreamcontroller/byobrequest/index.md @@ -4,7 +4,6 @@ slug: Web/API/ReadableByteStreamController/byobRequest page-type: web-api-instance-property tags: - API - - Experimental - Property - ReadableByteStreamController - Reference @@ -12,19 +11,36 @@ tags: - byobRequest browser-compat: api.ReadableByteStreamController.byobRequest --- -{{SeeCompatTable}}{{APIRef("Streams")}} +{{APIRef("Streams")}} -The **`byobRequest`** read-only property of the -{{domxref("ReadableByteStreamController")}} interface returns the current BYOB pull -request, or `undefined` if there are no pending requests. +The **`byobRequest`** read-only property of the {{domxref("ReadableByteStreamController")}} interface returns the current BYOB request, or `null` if there are no pending requests. + +An underlying byte source should check this property, and use it to write data to the stream if it exists (rather than using {{domxref("ReadableByteStreamController.enqueue()")}}). +This will result in an efficient zero-byte transfer of the data to the consumer. ## Value -A {{domxref("ReadableStreamBYOBRequest")}} object instance, or `undefined`. +A {{domxref("ReadableStreamBYOBRequest")}} object instance, or `null`. ## Examples -TBD. +The example in [Using readable byte streams > Creating a readable socket push byte stream](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams#creating_a_readable_socket_push_byte_stream) shows how you use a `byobRequest` to transfer data (if it exists), or otherwise copy the data to the stream's internal queues using {{domxref("ReadableByteStreamController.enqueue()")}}. + +The relevant code is reproduced below. +If the `byobRequest` exists, data is read into {{domxref("ReadableStreamBYOBRequest.view","controller.byobRequest.view")}}, and then {{domxref("ReadableStreamBYOBRequest.respond()")}} is called to signal the amount of data that is ready to transfer. + +```js +if (controller.byobRequest) { + const v = controller.byobRequest.view; + bytesRead = socket.readInto(v.buffer, v.byteOffset, v.byteLength); + if (bytesRead === 0) { + controller.close(); + } + controller.byobRequest.respond(bytesRead); +} else { + // Write to data using enqueue(). +} +``` ## Specifications @@ -33,3 +49,7 @@ TBD. ## Browser compatibility {{Compat}} + +## See also + +- [Using readable byte streams](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams) diff --git a/files/en-us/web/api/readablebytestreamcontroller/close/index.md b/files/en-us/web/api/readablebytestreamcontroller/close/index.md index 937e748f344975a..2a4cc50f5902d8c 100644 --- a/files/en-us/web/api/readablebytestreamcontroller/close/index.md +++ b/files/en-us/web/api/readablebytestreamcontroller/close/index.md @@ -4,7 +4,6 @@ slug: Web/API/ReadableByteStreamController/close page-type: web-api-instance-method tags: - API - - Experimental - Method - ReadableByteStreamController - Reference @@ -12,13 +11,14 @@ tags: - close browser-compat: api.ReadableByteStreamController.close --- -{{SeeCompatTable}}{{APIRef("Streams")}} +{{APIRef("Streams")}} -The **`close()`** method of the -{{domxref("ReadableByteStreamController")}} interface closes the associated stream. +The **`close()`** method of the {{domxref("ReadableByteStreamController")}} interface closes the associated stream. -> **Note:** Readers will still be able to read any previously-enqueued -> chunks from the stream, but once those are read, the stream will become closed. +This might be called by the underlying source when its data source has been exhausted/completed. + +> **Note:** Readers will still be able to read any previously-enqueued chunks from the stream, but once those are read, the stream will become closed. +> However if there is an outstanding and partially written {{domxref("ReadableByteStreamController.byobRequest","byobRequest")}} when `close()` is called, the stream will be errored. ## Syntax @@ -37,12 +37,24 @@ None ({{jsxref("undefined")}}). ### Exceptions - {{jsxref("TypeError")}} - - : Thrown if the source object is not a `ReadableByteStreamController`, or the stream - is not readable for some other reason. + - : Thrown if the source object is not a `ReadableByteStreamController`, it is already closed, or the stream is not readable for some other reason. ## Examples -TBD. +The example in [Using readable byte streams > Creating a readable socket push byte stream](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams#creating_a_readable_socket_push_byte_stream) how we might close the stream when there is no more data. + +The relevant code is reproduced below. +This relies on the hypothetical `readInto()` method returning 0 bytes only when there is no more data. + +```js +bytesRead = socket.readInto(v.buffer, v.byteOffset, v.byteLength); +if (bytesRead === 0) { + controller.close(); +} +``` + +After calling close, the stream will be closed, and any consumers signalled. +For example if using a {{domxref("ReadableStreamBYOBReader")}} any {{domxref("ReadableStreamBYOBReader.read()","read()")}} requests would resolve with `done: true` and the promise from {{domxref("ReadableStreamBYOBReader.closed")}} would also be resolved. ## Specifications @@ -51,3 +63,7 @@ TBD. ## Browser compatibility {{Compat}} + +## See also + +- [Using readable byte streams](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams) diff --git a/files/en-us/web/api/readablebytestreamcontroller/desiredsize/index.md b/files/en-us/web/api/readablebytestreamcontroller/desiredsize/index.md index ef0e774c2ad9b39..b591539d8fd603c 100644 --- a/files/en-us/web/api/readablebytestreamcontroller/desiredsize/index.md +++ b/files/en-us/web/api/readablebytestreamcontroller/desiredsize/index.md @@ -4,7 +4,6 @@ slug: Web/API/ReadableByteStreamController/desiredSize page-type: web-api-instance-property tags: - API - - Experimental - Property - ReadableByteStreamController - Reference @@ -12,19 +11,26 @@ tags: - desiredSize browser-compat: api.ReadableByteStreamController.desiredSize --- -{{SeeCompatTable}}{{APIRef("Streams")}} +{{APIRef("Streams")}} -The **`desiredSize`** read-only property of the -{{domxref("ReadableByteStreamController")}} interface returns the desired size required -to fill the stream's internal queue. +The **`desiredSize`** read-only property of the {{domxref("ReadableByteStreamController")}} interface returns the number of bytes required to fill the stream's internal queue to its "desired size". + +The value is used by the stream to indicate a preferred flow rate to the underlying source. +Sources that support throttling or pausing their inflow of data (not all do!) should control the inflow such that `desiredSize` of the stream buffer is kept positive and as close to zero as possible. + +The `desiredSize` is used to apply [backpressure](/en-US/docs/Web/API/Streams_API/Concepts#backpressure) from downstream consumers. ## Value An integer. Note that this can be negative if the queue is over-full. +The value will be `null` if the stream has errored and `0` if it is closed. + ## Examples -TBD. +The [A readable stream with an underlying push source and backpressure support](https://streams.spec.whatwg.org/#example-rs-push-backpressure) example in the spec provides a good example of using `desiredSize` to manually detect when the stream is full and apply backpressure. + +While the example uses a default source, the concepts are exactly the same as for readable byte sources. ## Specifications diff --git a/files/en-us/web/api/readablebytestreamcontroller/enqueue/index.md b/files/en-us/web/api/readablebytestreamcontroller/enqueue/index.md index a34cb9252dd07f6..c38b3a7a731093e 100644 --- a/files/en-us/web/api/readablebytestreamcontroller/enqueue/index.md +++ b/files/en-us/web/api/readablebytestreamcontroller/enqueue/index.md @@ -4,7 +4,6 @@ slug: Web/API/ReadableByteStreamController/enqueue page-type: web-api-instance-method tags: - API - - Experimental - Method - ReadableByteStreamController - Reference @@ -12,11 +11,11 @@ tags: - enqueue browser-compat: api.ReadableByteStreamController.enqueue --- -{{SeeCompatTable}}{{APIRef("Streams")}} +{{APIRef("Streams")}} -The **`enqueue()`** method of the -{{domxref("ReadableByteStreamController")}} interface enqueues a given chunk in the -associated stream. +The **`enqueue()`** method of the {{domxref("ReadableByteStreamController")}} interface enqueues a given chunk on the associated readable byte stream (the chunk is copied into the stream's internal queues). + +This should only be used to transfer data to the queue when {{domxref("ReadableByteStreamController.byobRequest","byobRequest")}} is `null`. ## Syntax @@ -36,13 +35,25 @@ None ({{jsxref("undefined")}}). ### Exceptions - {{jsxref("TypeError")}} - - : Thrown if the source object is not a `ReadableByteStreamController`, or the stream - cannot be read for some other reason, or the chunk is not an object, or the chunk's - internal array buffer is non-existent or detached. + - : Thrown if the source object is not a `ReadableByteStreamController`, or the stream cannot be read for some other reason, or the chunk is not an object, or the chunk's internal array buffer is non-existent, zero-length, or detached. + It is also thrown if the stream has been closed. ## Examples -TBD. +The example in [Using readable byte streams > Creating a readable socket push byte stream](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams#creating_a_readable_socket_push_byte_stream) shows how you can use `enqueue()` to copy data to the stream if there is no outstanding {{domxref("ReadableByteStreamController.byobRequest","byobRequest")}}. +If there is a `byobRequest` then it should be used! + +The code below shows data being read into an `ArrayBuffer` using a "hypothetical" `socket.readInto()` method and then enqueued (but only if data was actually copied): + +```js +const buffer = new ArrayBuffer(DEFAULT_CHUNK_SIZE); +bytesRead = socket.readInto(buffer, 0, DEFAULT_CHUNK_SIZE); +if (bytesRead === 0) { + controller.close(); +} else { + controller.enqueue(new Uint8Array(buffer, 0, bytesRead)); +} +``` ## Specifications @@ -51,3 +62,7 @@ TBD. ## Browser compatibility {{Compat}} + +## See also + +- [Using readable byte streams](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams) diff --git a/files/en-us/web/api/readablebytestreamcontroller/error/index.md b/files/en-us/web/api/readablebytestreamcontroller/error/index.md index 7098218fb3b5578..c08ea3fd64a17e9 100644 --- a/files/en-us/web/api/readablebytestreamcontroller/error/index.md +++ b/files/en-us/web/api/readablebytestreamcontroller/error/index.md @@ -5,18 +5,18 @@ page-type: web-api-instance-method tags: - API - Error - - Experimental - Method - ReadableByteStreamController - Reference - Streams browser-compat: api.ReadableByteStreamController.error --- -{{SeeCompatTable}}{{APIRef("Streams")}} +{{APIRef("Streams")}} -The **`error()`** method of the -{{domxref("ReadableByteStreamController")}} interface causes any future interactions -with the associated stream to error. +The **`error()`** method of the {{domxref("ReadableByteStreamController")}} interface causes any future interactions with the associated stream to error with the specified reason. + +This is commonly called by an underlying source to surface an error from the interface where it gets its data (such as a file-read or socket error). +It can also be called from elsewhere to trigger a stream error, for example if another part of the system that the stream relies on fails. ## Syntax @@ -36,12 +36,28 @@ None ({{jsxref("undefined")}}). ### Exceptions - {{jsxref("TypeError")}} - - : Thrown if the source object is not a `ReadableByteStreamController`, or the stream - is not readable for some other reason. + - : Thrown if the source object is not a `ReadableByteStreamController`, or the stream is not readable for some other reason. ## Examples -TBD. +The example in [Using readable byte streams > Creating a readable socket push byte stream](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams#creating_a_readable_socket_push_byte_stream) shows how you might use `error()` to manually trigger a stream error if another part of the system it relies on fails. + +Specifically, the underlying source `start()` method calls `readRepeatedly()` to perform all setup operations and to make a request for data. +This returns a promise. +If there are any errors thrown when reading the data they will be caught by the chained `catch()` function. +In `catch()` we then call `error()` on the controller, passing the reason from the underlying source. + +```js +start(controller) { + readRepeatedly().catch((e) => controller.error(e)); +} + +function readRepeatedly() { + return socket.select2().then(() => { + // ... + } +} +``` ## Specifications @@ -50,3 +66,7 @@ TBD. ## Browser compatibility {{Compat}} + +## See also + +- [Using readable byte streams](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams) diff --git a/files/en-us/web/api/readablebytestreamcontroller/index.md b/files/en-us/web/api/readablebytestreamcontroller/index.md index 6a224b834653683..5c9df1ed70a0a99 100644 --- a/files/en-us/web/api/readablebytestreamcontroller/index.md +++ b/files/en-us/web/api/readablebytestreamcontroller/index.md @@ -4,7 +4,6 @@ slug: Web/API/ReadableByteStreamController page-type: web-api-interface tags: - API - - Experimental - Fetch - Interface - ReadableByteStreamController @@ -12,18 +11,37 @@ tags: - Streams browser-compat: api.ReadableByteStreamController --- -{{APIRef("Streams")}}{{SeeCompatTable}} +{{APIRef("Streams")}} -The **`ReadableByteStreamController`** interface of the [Streams API](/en-US/docs/Web/API/Streams_API) represents a controller allowing control of a {{domxref("ReadableStream")}}'s state and internal queue. Byte stream controllers are for byte streams. +The **`ReadableByteStreamController`** interface of the [Streams API](/en-US/docs/Web/API/Streams_API) represents a controller for a [readable byte stream](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams). +It allows control of the state and internal queue of a {{domxref("ReadableStream")}} with an underlying byte source, and enables efficient zero-copy transfer of data from the underlying source to a consumer when the stream's internal queue is empty. + +An instance of this controller type is created if an `underlyingSource` object with the property `type="bytes"` is passed as an argument to the [`ReadableStream()` constructor](/en-US/docs/Web/API/ReadableStream/ReadableStream#type). +The `underlyingSource` object may also define [`start()`](/en-US/docs/Web/API/ReadableStream/ReadableStream#start) and [`pull()`](/en-US/docs/Web/API/ReadableStream/ReadableStream#pull) callback functions. +These are called with the controller as a parameter, in order to setup the underlying source, and request data when needed. + +The underlying source uses the controller to supply data to the stream via its [`byobRequest`](#readablebytestreamcontroller.byobrequest) property or [`enqueue()`](#readablebytestreamcontroller.enqueue) method. +[`byobRequest`](#readablebytestreamcontroller.byobrequest) is a {{domxref("ReadableStreamBYOBRequest")}} object that represents a pending request from a consumer to make a zero-copy transfer of data direct to a consumer. +`byobRequest` must be used to copy data if it exists (do not use `enqueue()` in this case)! +If the underlying source needs to pass data to the stream and `byobRequest` is `null` then the source can call [`enqueue()`](#readablebytestreamcontroller.enqueue) to add the data to the stream's internal queues. + +Note that the [`byobRequest`](#readablebytestreamcontroller.byobrequest) is only created in "BYOB mode" when there is a request from a reader and the stream's internal queue is empty. +"BYOB mode" is enabled when using a {{domxref("ReadableStreamBYOBReader")}} (typically constructed by calling {{domxref("ReadableStream.getReader()")}} with the argument `{ mode: 'byob' }`). +It is also enabled when using a default reader and [`autoAllocateChunkSize`](/en-US/docs/Web/API/ReadableStream/ReadableStream#autoallocatechunksize) is specified in the [`ReadableController()` constructor](/en-US/docs/Web/API/ReadableStream/ReadableStream#autoallocatechunksize). + +An underlying byte source can also use the controller to [`close()`](#readablebytestreamcontroller.close) the stream when all the data has been sent and report errors from the underlying source using [`error()`](#readablebytestreamcontroller.error). +The controller's [`desiredSize`](#readablebytestreamcontroller.desiredsize) property is used to apply "backpressure", informing the underlying source of the size of the internal queue (small values indicate that the queue is filling up, hinting to the underlying source that it is be desirable to pause or throttle the inflow). + +Note that even though the controller is primarily used by the underlying byte source, there is no reason it cannot be stored used by other parts of the system to signal the stream. ## Constructor -None. `ReadableByteStreamController` instances are created automatically during `ReadableStream` construction. +None. `ReadableByteStreamController` instances are automatically created if an `underlyingSource` with the property `type="bytes"` is passed to the [`ReadableStream()` constructor](/en-US/docs/Web/API/ReadableStream/ReadableStream#type). ## Properties - {{domxref("ReadableByteStreamController.byobRequest")}} {{readonlyInline}} - - : Returns the current BYOB pull request. + - : Returns the current BYOB pull request, or `null` if there no outstanding request. - {{domxref("ReadableByteStreamController.desiredSize")}} {{readonlyInline}} - : Returns the desired size required to fill the stream's internal queue. @@ -38,7 +56,9 @@ None. `ReadableByteStreamController` instances are created automatically during ## Examples -TBD. +The controller is used by an underlying source to transfer or enqueue data, to signal that the stream has no more data (has closed) or has errored. It is also used to signal the underlying source from "upstream" of the desired data rate, using {{domxref("ReadableByteStreamController.desiredSize","desiredSize")}}. + +The example in [Using readable byte streams](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams), in particular [Creating a readable socket push byte stream](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams#creating_a_readable_socket_push_byte_stream), show most of these cases. ## Specifications @@ -47,3 +67,7 @@ TBD. ## Browser compatibility {{Compat}} + +## See also + +- [Using readable byte streams](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams) diff --git a/files/en-us/web/api/readablestream/getreader/index.md b/files/en-us/web/api/readablestream/getreader/index.md index 41e44badc74599b..b2ce2010090d895 100644 --- a/files/en-us/web/api/readablestream/getreader/index.md +++ b/files/en-us/web/api/readablestream/getreader/index.md @@ -20,18 +20,22 @@ While the stream is locked, no other reader can be acquired until this one is re ```js getReader() -getReader(mode) +getReader(options) ``` ### Parameters -- `mode` {{optional_inline}} +- `options` {{optional_inline}} - - : An object containing a property `mode`, specifying the type of reader to create. - Values can be: + - : An object containing the following properties: - - `"byob"`, which results in a {{domxref("ReadableStreamBYOBReader")}} being created that can read readable byte streams (i.e. can handle "bring your own buffer" reading). - - `undefined` (or not specified at all — this is the default), which results in a {{domxref("ReadableStreamDefaultReader")}} being created that can read individual chunks from a stream. + - `mode` {{optional_inline}} + + - : An property that specifies the type of reader to create. + Values can be: + + - `"byob"`, which results in a {{domxref("ReadableStreamBYOBReader")}} being created that can read readable byte streams (streams that support zero-copy transfer from an underlying byte source to the reader when internal stream buffers are empty). + - `undefined` (or not specified at all — this is the default), which results in a {{domxref("ReadableStreamDefaultReader")}} being created that can read individual chunks from a stream. ### Return value @@ -42,8 +46,8 @@ A {{domxref("ReadableStreamDefaultReader")}} or {{domxref("ReadableStreamBYOBRea - {{jsxref("RangeError")}} - : Thrown if the provided mode value is not `"byob"` or `undefined`. - {{jsxref("TypeError")}} - - : Thrown if the stream you are trying to create a reader for is not a - {{domxref("ReadableStream")}}. + - : Thrown if the stream you are trying to create a reader for is already locked, or not a {{domxref("ReadableStream")}}. + This is also thrown if a BYOB reader is requested and the stream controller is not a {{domxref("ReadableByteStreamController")}} (the stream was not [constructed](/en-US/docs/Web/API/ReadableStream/ReadableStream) as an underlying source with [`type="bytes"`](/en-US/docs/Web/API/ReadableStream/ReadableStream#type)). ## Examples diff --git a/files/en-us/web/api/readablestream/readablestream/index.md b/files/en-us/web/api/readablestream/readablestream/index.md index 2ad797a4d8eb2ef..8098072363d3991 100644 --- a/files/en-us/web/api/readablestream/readablestream/index.md +++ b/files/en-us/web/api/readablestream/readablestream/index.md @@ -63,12 +63,11 @@ new ReadableStream(underlyingSource, queuingStrategy) (bring your own buffer)/byte stream. If it is not included, the passed controller will be a {{domxref("ReadableStreamDefaultController")}}. - `autoAllocateChunkSize` {{optional_inline}} - - : For byte streams, the developer can set the `autoAllocateChunkSize` - with a positive integer value to turn on the stream's auto-allocation feature. - With this turned on, the stream implementation will automatically allocate an - {{jsxref("ArrayBuffer")}} with a size of the given integer, and the consumer can - also use a default reader. + - : For byte streams, the developer can set the `autoAllocateChunkSize` with a positive integer value to turn on the stream's auto-allocation feature. + With this is set, the stream implementation will automatically allocate a view buffer of the specified size in {{domxref("ReadableByteStreamController.byobRequest")}} when required. + This must be set to enable zero-copy transfers to be used with a default {{domxref("ReadableStreamDefaultReader")}}. + If not set, a default reader will still stream data, but {{domxref("ReadableByteStreamController.byobRequest")}} will always be `null` and transfers to the consumer must be via the stream's internal queues. - `queuingStrategy` {{optional_inline}} - : An object that optionally defines a queuing strategy for the stream. This takes two diff --git a/files/en-us/web/api/readablestreambyobreader/cancel/index.md b/files/en-us/web/api/readablestreambyobreader/cancel/index.md index 5285a722bc23b5e..2d4d7db68c6daa2 100644 --- a/files/en-us/web/api/readablestreambyobreader/cancel/index.md +++ b/files/en-us/web/api/readablestreambyobreader/cancel/index.md @@ -4,7 +4,6 @@ slug: Web/API/ReadableStreamBYOBReader/cancel page-type: web-api-instance-method tags: - API - - Experimental - Method - ReadableStreamBYOBReader - Reference @@ -12,14 +11,12 @@ tags: - cancel browser-compat: api.ReadableStreamBYOBReader.cancel --- -{{SeeCompatTable}}{{APIRef("Streams")}} +{{APIRef("Streams")}} -The **`cancel()`** method of the -{{domxref("ReadableStreamBYOBReader")}} interface returns a {{jsxref("Promise")}} that resolves when the stream is canceled. Calling this method signals a loss of interest in the stream by a consumer. +The **`cancel()`** method of the {{domxref("ReadableStreamBYOBReader")}} interface returns a {{jsxref("Promise")}} that resolves when the stream is canceled. +Calling this method signals a loss of interest in the stream by a consumer. -> **Note:** If the reader is active, the `cancel()` method -> behaves the same as that for the associated stream -> ({{domxref("ReadableStream.cancel()")}}). +> **Note:** If the reader is active, the `cancel()` method behaves the same as that for the associated stream ({{domxref("ReadableStream.cancel()")}}). ## Syntax @@ -35,18 +32,25 @@ cancel(reason) ### Return value -A {{jsxref("Promise")}}, which fulfills with the value given in the `reason` -parameter. +A {{jsxref("Promise")}}, which fulfills with the value given in the `reason` parameter. ### Exceptions - {{jsxref("TypeError")}} - - : The source object is not a `ReadableStreamBYOBReader`, or the stream has - no owner. + - : The source object is not a `ReadableStreamBYOBReader`, or the stream has no owner. ## Examples -TBD. +This example code calls the `cancel()` method when a button is pressed, passing the string "user choice" as a reason. +The promise resolves when cancellation completes. + +```js +button.addEventListener('click', () => + { reader.cancel("user choice").then( () => { console.log(`cancel complete`) }) } +); +``` + +Note that this code can be seen running in the [Using readable byte streams](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams#result) example code (press the **Cancel stream** button). ## Specifications @@ -55,3 +59,7 @@ TBD. ## Browser compatibility {{Compat}} + +## See also + +- [Using readable byte stream](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams) diff --git a/files/en-us/web/api/readablestreambyobreader/closed/index.md b/files/en-us/web/api/readablestreambyobreader/closed/index.md index c66cac8dd97d259..6e4e900bf4a8840 100644 --- a/files/en-us/web/api/readablestreambyobreader/closed/index.md +++ b/files/en-us/web/api/readablestreambyobreader/closed/index.md @@ -4,7 +4,6 @@ slug: Web/API/ReadableStreamBYOBReader/closed page-type: web-api-instance-property tags: - API - - Experimental - Property - ReadableStreamBYOBReader - Reference @@ -12,13 +11,11 @@ tags: - closed browser-compat: api.ReadableStreamBYOBReader.closed --- -{{SeeCompatTable}}{{APIRef("Streams")}} +{{APIRef("Streams")}} -The **`closed`** read-only property -of the {{domxref("ReadableStreamBYOBReader")}} interface returns a -{{jsxref("Promise")}} that fulfills when the stream closes, or rejects if the -stream throws an error or the reader's lock is released. This property enables you -to write code that responds to an end to the streaming process. +The **`closed`** read-only property of the {{domxref("ReadableStreamBYOBReader")}} interface returns a {{jsxref("Promise")}} that fulfills when the stream closes, or rejects if the stream throws an error or the reader's lock is released. + +This property enables you to write code that responds to an end to the streaming process. ## Value @@ -26,7 +23,14 @@ A {{jsxref("Promise")}}. ## Examples -TBD. +The code below shows the pattern for handling the closed/error state of a BYOBReader. + +```js +const reader = stream.getReader({mode: "byob"}); +reader.closed + .then( () => { /* Resolved - code to handle stream closing */ } ) + .catch( () => { /* Rejected - code to handle error */ } ); +``` ## Specifications diff --git a/files/en-us/web/api/readablestreambyobreader/index.md b/files/en-us/web/api/readablestreambyobreader/index.md index 5eebd1ef5cb148b..b88f702c0a616d5 100644 --- a/files/en-us/web/api/readablestreambyobreader/index.md +++ b/files/en-us/web/api/readablestreambyobreader/index.md @@ -4,7 +4,6 @@ slug: Web/API/ReadableStreamBYOBReader page-type: web-api-interface tags: - API - - Experimental - Fetch - Interface - ReadableStreamBYOBReader @@ -12,9 +11,19 @@ tags: - Streams browser-compat: api.ReadableStreamBYOBReader --- -{{APIRef("Streams")}}{{SeeCompatTable}} +{{APIRef("Streams")}} -The `ReadableStreamBYOBReader` interface of the [Streams API](/en-US/docs/Web/API/Streams_API) represents a BYOB ("bring your own buffer") reader that can be used to read stream data supplied by the developer (e.g. a custom {{domxref("ReadableStream.ReadableStream","ReadableStream()")}} constructor). +The `ReadableStreamBYOBReader` interface of the [Streams API](/en-US/docs/Web/API/Streams_API) defines a reader for a {{domxref("ReadableStream")}} that supports zero-copy reading from an underlying byte source. +It is used for efficient copying from underlying sources where the data is delivered as an "anonymous" sequence of bytes, such as files. + +An instance of this reader type should usually be obtained by calling {{domxref("ReadableStream.getReader()")}} on the stream, specifying `mode: "byob"` in the options parameter. +The readable stream must have an _underlying byte source_. In other words, it must have been [constructed](/en-US/docs/Web/API/ReadableStream/ReadableStream) specifying an underlying source with [`type: "bytes"`](/en-US/docs/Web/API/ReadableStream/ReadableStream#type)). + +Using this kind of reader, a [`read()`](#readablestreambyobreader.read) request when the readable stream's internal queues are empty will result in a zero copy transfer from the underlying source (bypassing the stream's internal queues). +If the internal queues are not empty, a `read()` will satisfy the request from the buffered data. + +Note that the methods and properties are similar to those for the default reader ({{domxref("ReadableStreamDefaultReader")}}). +The `read()` method differs in that it provide a view into which data should be written. ## Constructor @@ -31,13 +40,82 @@ The `ReadableStreamBYOBReader` interface of the [Streams API](/en-US/docs/Web/AP - {{domxref("ReadableStreamBYOBReader.cancel()")}} - : Returns a {{jsxref("Promise")}} that resolves when the stream is canceled. Calling this method signals a loss of interest in the stream by a consumer. The supplied `reason` argument will be given to the underlying source, which may or may not use it. - {{domxref("ReadableStreamBYOBReader.read()")}} - - : Returns a {{jsxref("Promise")}} that resolves with an object indicating the state of the stream: either the next chunk in the stream or an indication that the stream is closed. + - : Passes a view into which data must be written, and returns a {{jsxref("Promise")}} that resolves with the next chunk in the stream or rejects with an indication that the stream is closed or has errored. - {{domxref("ReadableStreamBYOBReader.releaseLock()")}} - : Releases the reader's lock on the stream. ## Examples -TBD. +The example below is taken from the live examples in [Using readable byte streams](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams#examples). + +First create the reader using {{domxref("ReadableStream.getReader()")}} on the stream, specifying `mode: "byob"` in the options parameter. +As this is a "Bring Your Own Buffer" reader, we also need to create an `ArrayBuffer` to read into. + +```js +const reader = stream.getReader({mode: "byob"}); +let buffer = new ArrayBuffer(4000); +``` + +A function that uses the reader is shown below. +This calls the `read()` method recursively to read data into the buffer. +The method takes a [`Uint8Array`](/en-US/docs/Web/JavaScript/Reference/Global_Objects/Uint8Array) [typed array](/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray) which is a view over the part of the original array buffer that has not yet been written. +The parameters of the view are calculated from the data that was received in previous calls, which define an offset into the original array buffer. + +```js +readStream(reader); + +function readStream(reader) { + let bytesReceived = 0; + let offset = 0; + + while (offset < buffer.byteLength) { + // read() returns a promise that resolves when a value has been received + reader.read( new Uint8Array(buffer, offset, buffer.byteLength - offset) ).then(function processBytes({ done, value }) { + // Result objects contain two properties: + // done - true if the stream has already given all its data. + // value - some data. Always undefined when done is true. + + if (done) { + // There is no more data in the stream + return; + } + + buffer = value.buffer; + offset += value.byteLength; + bytesReceived += value.byteLength; + + // Read some more, and call this function again + return reader.read( new Uint8Array(buffer, offset, buffer.byteLength - offset) ).then(processBytes); + }); + } +} +``` + +When there is no more data in the stream, the `read()` method resolves with an object with the property `done` set to `true`, and the function returns. + +The {{domxref("ReadableStreamBYOBReader.closed")}} property returns a promise that can be used to monitor for the stream being closed or errored, or the reader lock being released. + +```js +reader.closed + .then( () => { /* Resolved - code to handle stream closing */ } ) + .catch( () => { /* Rejected - code to handle error */ } ); +``` + +To cancel the stream call {{domxref("ReadableStreamBYOBReader.cancel()")}}, optionally specifying a _reason_. +This returns a promise that will resolve when the stream has been cancelled. +When the stream is cancelled the controller will in turn call `cancel()` on the underlying source, passing in the optional reason. + +The example code in [Using readable byte streams](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams#examples) calls the cancel method when a button is pressed, as shown: + +```js +button.addEventListener('click', () => { reader.cancel("user choice").then( () => { console.log(`cancel complete`) }) } ); +``` + +The consumer can also call `releaseLock()` to release the reader's hold on the stream, but only when no read is pending: + +```js +reader.releaseLock(); +``` ## Specifications @@ -46,3 +124,7 @@ TBD. ## Browser compatibility {{Compat}} + +## See also + +- [Using readable byte stream](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams) diff --git a/files/en-us/web/api/readablestreambyobreader/read/index.md b/files/en-us/web/api/readablestreambyobreader/read/index.md index e2d0c239da4b53a..3e1d7a7e42352e0 100644 --- a/files/en-us/web/api/readablestreambyobreader/read/index.md +++ b/files/en-us/web/api/readablestreambyobreader/read/index.md @@ -4,7 +4,6 @@ slug: Web/API/ReadableStreamBYOBReader/read page-type: web-api-instance-method tags: - API - - Experimental - Method - ReadableStreamBYOBReader - Reference @@ -12,10 +11,24 @@ tags: - read browser-compat: api.ReadableStreamBYOBReader.read --- -{{SeeCompatTable}}{{APIRef("Streams")}} +{{APIRef("Streams")}} -The **`read()`** method of the -{{domxref("ReadableStreamBYOBReader")}} interface returns a {{jsxref("Promise")}} that resolves with an object representing the next chunk in the stream's queue. +The **`read()`** method of the {{domxref("ReadableStreamBYOBReader")}} interface is used to read data into a view on a user-supplied buffer from an associated [readable byte stream](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams). +A request for data will be statisfied from the stream's internal queues if there is any data present. +If the stream queues are empty, the request may be supplied as a zero-copy transfer from the underlying byte source. + +The method takes as an argument a view on a buffer that supplied data is to be read into, and returns a {{jsxref("Promise")}}. +The promise resolves with an object that has properties `value` and `done` when data comes available, or if the stream is cancelled. +If the stream is errored, the promise will be rejected with the relevant error object. + +If a chunk of data is supplied, the `value` property will contain a new view. +This will be a view over the same buffer/backing memory (and of the same type) as the original `view` passed to the `read()` method, now populated with the new chunk of data. +Note that once the promise resolves, the original `view` passed to the method will be detached and no longer usable. +The promise will resolve with a `value: undefined` if the stream has been cancelled. +In this case the backing memory region of `view` is discarded and not returned to the caller (all previously read data in the view's buffer is lost). + +The `done` property indicates whether or not more data is expected. +The value is set `true` if the stream is closed or cancelled, and `false` otherwise. ## Syntax @@ -26,28 +39,95 @@ read(view) ### Parameters - `view` - - : The view to be read into. + - : The view that data is to be read into. ### Return value -A {{jsxref("Promise")}}, which fulfills/rejects with a result depending on the state of -the stream. The following are possible: +A {{jsxref("Promise")}}, which resolves/rejects with a result depending on the state of the stream. + +The following are possible: + +- If a chunk is available and the stream is still active, the promise resolves with an object of the form: + + ```js + { value: theChunk, done: false } + ``` + + `theChunk` is a view containing the new data. + This is a view of the same type and over the same backing memory as the `view` passed to the `read()` method. + The original `view` will be detached and no longer usable. + +- If the stream is closed, the promise resolves with an object of the form (where `theChunk` has the same properties as above): + + ```js + { value: theChunk, done: true } + ``` + +- If the stream is cancelled, the promise fulfills with an object of the form: + + ```js + { value: undefined, done: true } + ``` + + In this case the backing memory is discarded. -- If a chunk is available, the promise fulfills with an object of the form - `{ value: theChunk, done: false }`. -- If the stream is closed, the promise fulfills with an object of the - form `{ value: undefined, done: true }`. - If the stream throws an error, the promise rejects with the relevant error. + ### Exceptions - {{jsxref("TypeError")}} - - : The source object is not a `ReadableStreamBYOBReader`, the stream has no - owner, the view is not an object or has become detached, or the view's length is 0. + - : The source object is not a `ReadableStreamBYOBReader`, the stream has no owner, the view is not an object or has become detached, or the view's length is 0. ## Examples -TBD. +The example code here is taken from the live examples in [Using readable byte streams](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams#examples). + +First we create the reader using {{domxref("ReadableStream.getReader()")}} on the stream, specifying `mode: "byob"` in the options parameter. +We also need create an `ArrayBuffer`, which is the "backing memory" of the views that we will write into. + +```js +const reader = stream.getReader({mode: "byob"}); +let buffer = new ArrayBuffer(4000); +``` + +A function that uses the reader is shown below. +This calls the `read()` method recursively to read data into the buffer. +The method takes a [`Uint8Array`](/en-US/docs/Web/JavaScript/Reference/Global_Objects/Uint8Array) [typed array](/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray) which is a view over the part of the original array buffer that has not yet been written. +The parameters of the view are calculated from the data that was received in previous calls, which define an offset into the original array buffer. + +```js +readStream(reader); + +function readStream(reader) { + let bytesReceived = 0; + let offset = 0; + + while (offset < buffer.byteLength) { + // read() returns a promise that resolves when a value has been received + reader.read( new Uint8Array(buffer, offset, buffer.byteLength - offset) ).then(function processBytes({ done, value }) { + // Result objects contain two properties: + // done - true if the stream has already given all its data. + // value - some data. Always undefined when done is true. + + if (done) { + // There is no more data in the stream + return; + } + + buffer = value.buffer; + offset += value.byteLength; + bytesReceived += value.byteLength; + + // Read some more, and call this function again + // Note that here we create a new view over the original buffer. + return reader.read( new Uint8Array(buffer, offset, buffer.byteLength - offset) ).then(processBytes); + }); + } +} +``` + +When there is no more data in the stream, the `read()` method resolves with an object with the property `done` set to `true`, and the function returns. ## Specifications @@ -56,3 +136,7 @@ TBD. ## Browser compatibility {{Compat}} + +## See also + +- [Using readable byte stream](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams) diff --git a/files/en-us/web/api/readablestreambyobreader/readablestreambyobreader/index.md b/files/en-us/web/api/readablestreambyobreader/readablestreambyobreader/index.md index abe39f5bd21865b..a96d2ebeeb6723b 100644 --- a/files/en-us/web/api/readablestreambyobreader/readablestreambyobreader/index.md +++ b/files/en-us/web/api/readablestreambyobreader/readablestreambyobreader/index.md @@ -5,19 +5,17 @@ page-type: web-api-constructor tags: - API - Constructor - - Experimental - ReadableStreamBYOBReader - Reference - Streams browser-compat: api.ReadableStreamBYOBReader.ReadableStreamBYOBReader --- -{{SeeCompatTable}}{{APIRef("Streams")}} +{{APIRef("Streams")}} -The **`ReadableStreamBYOBReader()`** -constructor creates and returns a `ReadableStreamBYOBReader` object instance. +The **`ReadableStreamBYOBReader()`** constructor creates and returns a `ReadableStreamBYOBReader` object instance. -> **Note:** You generally wouldn't use this constructor manually; instead, -> you'd use the {{domxref("ReadableStream.getReader()")}} method. +> **Note:** You generally wouldn't use this constructor manually; +> instead, you'd use the {{domxref("ReadableStream.getReader()")}} method with the argument `"byob"`. ## Syntax @@ -37,13 +35,16 @@ An instance of the {{domxref("ReadableStreamBYOBReader")}} object. ### Exceptions - {{jsxref("TypeError")}} - - : Thrown if the supplied `stream` parameter is not a {{domxref("ReadableStream")}}, - or it is already locked for reading by another reader, or its stream controller is not - a {{domxref("ReadableByteStreamController")}}. + - : Thrown if the supplied `stream` parameter is not a {{domxref("ReadableStream")}}, or it is already locked for reading by another reader, or its stream controller is not a {{domxref("ReadableByteStreamController")}}. ## Examples -TBD +The constructor is rarely called directly. +Instead call {{domxref("ReadableStream.getReader()")}} as shown: + +```js +const reader = stream.getReader({mode: "byob"}); +``` ## Specifications diff --git a/files/en-us/web/api/readablestreambyobreader/releaselock/index.md b/files/en-us/web/api/readablestreambyobreader/releaselock/index.md index 524967009f06c63..0d726b192aeb173 100644 --- a/files/en-us/web/api/readablestreambyobreader/releaselock/index.md +++ b/files/en-us/web/api/readablestreambyobreader/releaselock/index.md @@ -4,7 +4,6 @@ slug: Web/API/ReadableStreamBYOBReader/releaseLock page-type: web-api-instance-method tags: - API - - Experimental - Method - ReadableStreamBYOBReader - Reference @@ -12,18 +11,15 @@ tags: - releaseLock browser-compat: api.ReadableStreamBYOBReader.releaseLock --- -{{SeeCompatTable}}{{APIRef("Streams")}} +{{APIRef("Streams")}} -The **`releaseLock()`** method of the -{{domxref("ReadableStreamBYOBReader")}} interface releases the reader's lock on the -stream. After the lock is released, the reader is no longer active. +The **`releaseLock()`** method of the {{domxref("ReadableStreamBYOBReader")}} interface releases the reader's lock on the stream. +After the lock is released, the reader is no longer active. -If the associated stream is errored when the lock is released, the reader will appear -errored in that same way subsequently; otherwise, the reader will appear closed. +The reader will appear errored if the associated stream is errored when the lock is released; otherwise, the reader will appear closed. -A reader's lock cannot be released while it still has a pending read request, i.e., if -a promise returned by the reader's {{domxref("ReadableStreamBYOBReader.read()")}} method -has not finished. This will result in a `TypeError` being thrown. +If the reader's lock is released while it still has pending read requests then the promises returned by the reader's {{domxref("ReadableStreamBYOBReader.read()")}} method are immediately rejected with a `TypeError`. +Unread chunks remain in the stream's internal queue and can be read later by acquiring a new reader. ## Syntax @@ -42,12 +38,17 @@ None ({{jsxref("undefined")}}). ### Exceptions - {{jsxref("TypeError")}} - - : Thrown if the source object is not a `ReadableStreamBYOBReader`, or a read request - is pending. + - : Thrown if the source object is not a `ReadableStreamBYOBReader`, or a read request is pending. ## Examples -TBD. +A trivial examples is shown below. +A lock is created as soon as the reader is created on the stream. + +```js +const reader = stream.getReader({mode: "byob"}); +reader.releaseLock(); +``` ## Specifications @@ -56,3 +57,7 @@ TBD. ## Browser compatibility {{Compat}} + +## See also + +- [Using readable byte stream](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams) diff --git a/files/en-us/web/api/readablestreambyobrequest/index.md b/files/en-us/web/api/readablestreambyobrequest/index.md index bce39a2175908a5..4464adabf4bc357 100644 --- a/files/en-us/web/api/readablestreambyobrequest/index.md +++ b/files/en-us/web/api/readablestreambyobrequest/index.md @@ -4,7 +4,6 @@ slug: Web/API/ReadableStreamBYOBRequest page-type: web-api-interface tags: - API - - Experimental - Fetch - Interface - ReadableStreamBYOBRequest @@ -12,11 +11,24 @@ tags: - Steams browser-compat: api.ReadableStreamBYOBRequest --- -{{APIRef("Streams")}}{{SeeCompatTable}} +{{APIRef("Streams")}} -The **`ReadableStreamBYOBRequest`** interface of the [Streams API](/en-US/docs/Web/API/Streams_API) represents a pull request into a {{domxref("ReadableByteStreamController")}} view. +The **`ReadableStreamBYOBRequest`** interface of the [Streams API](/en-US/docs/Web/API/Streams_API) represents a "pull request" for data from an underlying source that will made as a zero-copy transfer to a consumer (bypassing the stream's internal queues). -A view, as mentioned below, refers to a typed array representing the destination region to which the associated `ReadableByteStreamController` controller can write generated data. +`ReadableStreamBYOBRequest` objects are created in "BYOB mode" when a consumer makes a request for data and the stream's internal queue is _empty_. +(The stream will resolve the consumer's request directly if it already has buffered data). +An underlying byte source can access active BYOB requests through its controller's {{domxref("ReadableByteStreamController.byobRequest")}} property, which will be set to `null` if there is no outstanding request. + +An underlying source that supports "BYOB mode" should check for {{domxref("ReadableByteStreamController.byobRequest")}} and must use it for transferring data, if present. +If data arrives from the underlying source when {{domxref("ReadableByteStreamController.byobRequest")}} is `null`, it can be queued using {{domxref("ReadableByteStreamController.enqueue()")}}. +This might happen when an underlying push source receives new data when the stream's internal buffers are not empty. + +An underlying source uses the request by writing data to the BYOB request's [`view`](#readablestreambyobrequest.view) and then calling [`respond()`](#readablestreambyobrequest.respond), or by calling [`respondWithNewView()`](#readablestreambyobrequest.respondwithnewview) and passing a new view as an argument. +Note that the "new view" must actually be a view over the _same_ buffer as the original `view`, starting at the same offset. +This might be used to return a shorter buffer if the underlying source is unable to fill the entire original view. + +Note that a {{domxref("ReadableByteStreamController")}} is only created for underlying sources when `type="bytes"` is specified for the source in the [`ReadableStream()` constructor](/en-US/docs/Web/API/ReadableStream/ReadableStream#type). +"BYOB mode" is enabled when either [`autoAllocateChunkSize`](/en-US/docs/Web/API/ReadableStream/ReadableStream#autoallocatechunksize) is specified in the [`ReadableController()` constructor](/en-US/docs/Web/API/ReadableStream/ReadableStream#autoallocatechunksize) or when using a {{domxref("ReadableStreamBYOBReader")}} (typically constructed by calling {{domxref("ReadableStream.getReader()")}} with the argument `{ mode: 'byob' }`). ## Constructor @@ -26,17 +38,53 @@ None. `ReadableStreamBYOBRequest` instance is created automatically by `Readable - {{domxref("ReadableStreamBYOBRequest.view")}} {{readonlyInline}} - : Returns the current view. + This is a view on a buffer that will be transferred to the consumer when `ReadableStreamBYOBRequest.respond()` is called. ## Methods - {{domxref("ReadableStreamBYOBRequest.respond()")}} - - : xxx + - : Signals the associated readable byte stream that the specified number of bytes were written into the current [`view`](#readablestreambyobrequest.view), which then causes the pending request from the consumer to be resolved. + Note that after this method is called the `view` is transferred and no longer modifiable. - {{domxref("ReadableStreamBYOBRequest.respondWithNewView()")}} - - : xxx + - : Signals to the associated readable byte stream view passed as an argument should be transferred to the consumer of the readable byte stream. + This new view must use the same buffer as the original [`view`](#readablestreambyobrequest.view), start at the same offset, and be the same length or shorter. + Note that after this method is called the `view` is transferred and no longer modifiable. ## Examples -TBD. +The following code is taken from the live example in [Using readable byte streams > Creating a readable socket push byte stream](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams#creating_a_readable_socket_push_byte_stream). + +A push underlying byte source with data to transfer should first check that {{domxref("ReadableByteStreamController.byobRequest","controller.byobRequest")}} is non-`null`. Pul +A pull underlying byte source would only need this check if auto chunk allocation was not enabled and it was used with a default reader. + +```js +if (controller.byobRequest) { + /* code to transfer data */ +} +``` + +There are two ways to read data into a `ReadableStreamBYOBRequest` and then transfer it. +The first is to write the data into the {{domxref("ReadableStreamBYOBRequest.view")}} property and then call {{domxref("ReadableStreamBYOBRequest.respond()")}} to indicate the amount of data to be transferred. +After the operation the `byobRequest.view` is detached and the request should be discarded. + +The code below shows this case using a hypothetical `readInto()` method to copy data into the view: + +```js +const v = controller.byobRequest.view; +bytesRead = socket.readInto(v.buffer, v.byteOffset, v.byteLength); +controller.byobRequest.respond(bytesRead); +``` + +The other approach is to call {{domxref("ReadableStreamBYOBRequest.respondWithNewView()")}} passing your own view on the same underlying backing data. +Note that this just another way of specifying the range of the underlying buffer/memory backing that is actually transferred. +The `respondWithNewView` equivalent to the code above would be: + +```js +const v = controller.byobRequest.view; +bytesRead = socket.readInto(v.buffer, v.byteOffset, v.byteLength); +const newView = new Uint8Array(v.buffer, v.byteOffset, bytesRead); +controller.byobRequest.respondWithNewView(newView); +``` ## Specifications @@ -45,3 +93,7 @@ TBD. ## Browser compatibility {{Compat}} + +## See also + +- [Using readable byte stream](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams) diff --git a/files/en-us/web/api/readablestreambyobrequest/respond/index.md b/files/en-us/web/api/readablestreambyobrequest/respond/index.md index 02f7ec7cdf9f2a2..465b0265713f605 100644 --- a/files/en-us/web/api/readablestreambyobrequest/respond/index.md +++ b/files/en-us/web/api/readablestreambyobrequest/respond/index.md @@ -4,7 +4,6 @@ slug: Web/API/ReadableStreamBYOBRequest/respond page-type: web-api-instance-method tags: - API - - Experimental - Method - ReadableStreamBYOBRequest - Reference @@ -12,10 +11,11 @@ tags: - respond browser-compat: api.ReadableStreamBYOBRequest.respond --- -{{SeeCompatTable}}{{APIRef("Streams")}} +{{APIRef("Streams")}} -The **`error()`** method of the -{{domxref("ReadableStreamBYOBRequest")}} interface xxx +The **`respond()`** method of the {{domxref("ReadableStreamBYOBRequest")}} interface is used to signal to the associated [readable byte stream](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams) that the specified number of bytes were written into the {{domxref("ReadableStreamBYOBRequest.view")}}. + +After this method is called, the {{domxref("ReadableStreamBYOBRequest/view","view")}} will be transferred and no longer modifiable. ## Syntax @@ -26,7 +26,7 @@ respond(bytesWritten) ### Parameters - `bytesWritten` - - : xxx + - : The number of bytes written into {{domxref("ReadableStreamBYOBRequest.view")}}. ### Return value @@ -35,12 +35,24 @@ None ({{jsxref("undefined")}}). ### Exceptions - {{jsxref("TypeError")}} - - : Thrown if the source object is not a `ReadableStreamBYOBRequest`, or there is no - associated controller, or the associated internal array buffer is detached. + - : The request does not have an associated {{domxref("ReadableByteStreamController")}} or the view buffer is not detached/cannot be transferred into. ## Examples -TBD. +The code below is taken from the live examples in [Using readable byte stream](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams). + +The method is called by an underlying byte source as part of making a zero-copy transfer of data to fulfill a pending read request from a consumer. +The underlying source first writes data into the {{domxref("ReadableStreamBYOBRequest.view")}} and then calls this `respond()` method to indicate _how_ much data was copied into the buffer, and cause the data to be transferred to the reader. + +The code below shows this case using a hypothetical `readInto()` method to copy data into the view: + +```js +const v = controller.byobRequest.view; +bytesRead = socket.readInto(v.buffer, v.byteOffset, v.byteLength); +controller.byobRequest.respond(bytesRead); +``` + +After the operation the `byobRequest.view` is detached and the request should be discarded. ## Specifications @@ -49,3 +61,7 @@ TBD. ## Browser compatibility {{Compat}} + +## See also + +- [Using readable byte stream](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams) diff --git a/files/en-us/web/api/readablestreambyobrequest/respondwithnewview/index.md b/files/en-us/web/api/readablestreambyobrequest/respondwithnewview/index.md index 8b6702a1dc07561..7216d10ddc9561a 100644 --- a/files/en-us/web/api/readablestreambyobrequest/respondwithnewview/index.md +++ b/files/en-us/web/api/readablestreambyobrequest/respondwithnewview/index.md @@ -4,7 +4,6 @@ slug: Web/API/ReadableStreamBYOBRequest/respondWithNewView page-type: web-api-instance-method tags: - API - - Experimental - Method - ReadableStreamBYOBRequest - Reference @@ -12,10 +11,15 @@ tags: - respondWithNewView browser-compat: api.ReadableStreamBYOBRequest.respondWithNewView --- -{{SeeCompatTable}}{{APIRef("Streams")}} +{{APIRef("Streams")}} -The **`respondWithNewView()`** method of the -{{domxref("ReadableStreamBYOBRequest")}} interface xxx +The **`respondWithNewView()`** method of the {{domxref("ReadableStreamBYOBRequest")}} interface specifies a new view that the consumer of the associated readable byte stream should write to instead of {{domxref("ReadableStreamBYOBRequest.view")}}. + +The new view must be an {{domxref("ArrayBufferView")}} that provides a view onto the same backing memory region as {{domxref("ReadableStreamBYOBRequest.view")}}. +After this method is called, the view that was passed into the method will be transferred and no longer modifiable. + +The method is intended for use cases where an underlying byte source needs to transfer a `byobRequest.view` internally before finishing its response. +For example, the source may transfer the BYOB view to a separate worker thread, and wait for the worker to transfer it back once it has been filled. ## Syntax @@ -26,7 +30,10 @@ respondWithNewView(view) ### Parameters - `view` - - : xxx + - : A {{domxref("ArrayBufferView")}} that the consumer of the associated readable byte stream should write to instead of {{domxref("ReadableStreamBYOBRequest.view")}}. + + This must be a view onto the same backing memory region as {{domxref("ReadableStreamBYOBRequest.view")}} and occupy the same or less memory. + Specifically, it must be either the view's buffer or a transferred version, must have the same `byteOffset`, and a `byteLength` (number of bytes written) that is less than or equal to that of the view. ### Return value @@ -35,13 +42,23 @@ None ({{jsxref("undefined")}}). ### Exceptions - {{jsxref("TypeError")}} - - : Thrown if the source object is not a `ReadableStreamBYOBRequest`, or there is no - associated controller, or the associated internal array buffer is non-existent or - detached. + - : Thrown if the source object is not a `ReadableStreamBYOBRequest`, or there is no associated controller, or the associated internal array buffer is non-existent or detached. + It may also be thrown if the `view` is zero-length when there is an active reader, or non-zero when called on a closed stream. + +- {{jsxref("RangeError")}} + - : Thrown if the new `view` does not match the backing memory region of {{domxref("ReadableStreamBYOBRequest.view")}}. + For example, it is not the same buffer (or a transferred version), has a different `byteOffset`, or is larger than the memory available to the the backing view. ## Examples -TBD. +The view to be transferred must be of the same type as {{domxref("ReadableStreamBYOBRequest.view")}}, have the same underlying buffer and byte offset, and be the same or smaller byteLength. + +For example, we might define the view and respond as shown below: + +```js +const v = controller.byobRequest.view; +bytesRead = socket.readInto(v.buffer, v.byteOffset, v.byteLength); +byobRequest.respondWithNewView(byobRequest.view.subarray(v.byteOffset, bytesRead)); ## Specifications @@ -50,3 +67,7 @@ TBD. ## Browser compatibility {{Compat}} + +## See also + +- [Using readable byte stream](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams) diff --git a/files/en-us/web/api/readablestreambyobrequest/view/index.md b/files/en-us/web/api/readablestreambyobrequest/view/index.md index 5dae7c43d68e268..77315c7dc8083cb 100644 --- a/files/en-us/web/api/readablestreambyobrequest/view/index.md +++ b/files/en-us/web/api/readablestreambyobrequest/view/index.md @@ -4,7 +4,6 @@ slug: Web/API/ReadableStreamBYOBRequest/view page-type: web-api-instance-property tags: - API - - Experimental - Property - ReadableStreamBYOBRequest - Reference @@ -12,19 +11,19 @@ tags: - View browser-compat: api.ReadableStreamBYOBRequest.view --- -{{SeeCompatTable}}{{APIRef("Streams")}} +{{APIRef("Streams")}} -The **`view`** getter property of the -{{domxref("ReadableStreamBYOBRequest")}} interface returns the current view. +The **`view`** getter property of the {{domxref("ReadableStreamBYOBRequest")}} interface returns the current view. ## Value -A [typed array](/en-US/docs/Web/JavaScript/Typed_arrays) representing the -destination region to which the controller can write generated data. +A [typed array](/en-US/docs/Web/JavaScript/Typed_arrays) representing the destination region to which the controller can write generated data. + +`null` if the request has already been responded to, by calling {{domxref("ReadableStreamBYOBRequest.respond()")}} or {{domxref("ReadableStreamBYOBRequest.respondWithNewView()")}}. ## Examples -TBD. +See the examples in {{domxref("ReadableStreamBYOBRequest")}}. ## Specifications @@ -33,3 +32,7 @@ TBD. ## Browser compatibility {{Compat}} + +## See also + +- [Using readable byte stream](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams) diff --git a/files/en-us/web/api/readablestreamdefaultreader/index.md b/files/en-us/web/api/readablestreamdefaultreader/index.md index 6b4673b70de1927..0cd840839dde0ec 100644 --- a/files/en-us/web/api/readablestreamdefaultreader/index.md +++ b/files/en-us/web/api/readablestreamdefaultreader/index.md @@ -13,7 +13,13 @@ browser-compat: api.ReadableStreamDefaultReader --- {{APIRef("Streams")}} -The **`ReadableStreamDefaultReader`** interface of the [Streams API](/en-US/docs/Web/API/Streams_API) represents a default reader that can be used to read stream data supplied from a network (e.g. a fetch request). +The **`ReadableStreamDefaultReader`** interface of the [Streams API](/en-US/docs/Web/API/Streams_API) represents a default reader that can be used to read stream data supplied from a network (such as a fetch request). + +A `ReadableStreamDefaultReader` can be used to read from a {{domxref("ReadableStream")}} that has an underlying source of any type (unlike a {{domxref("ReadableStreamBYOBReader")}}, which can only be used with readable streams that have an _underlying byte source_). + +Note however that zero-copy transfer from an underlying source is only supported for underlying byte sources that autoallocate buffers. +In other words, the stream must have been [constructed](/en-US/docs/Web/API/ReadableStream/ReadableStream) specifying both [`type="bytes"`](/en-US/docs/Web/API/ReadableStream/ReadableStream#type) and [`autoAllocateChunkSize`](/en-US/docs/Web/API/ReadableStream/ReadableStream#autoallocatechunksize). +For any other underlying source, the stream will always satisfy read requests with data from internal queues. ## Constructor diff --git a/files/en-us/web/api/readablestreamdefaultreader/releaselock/index.md b/files/en-us/web/api/readablestreamdefaultreader/releaselock/index.md index 3bd44877c3c1591..019cedc8834a9a4 100644 --- a/files/en-us/web/api/readablestreamdefaultreader/releaselock/index.md +++ b/files/en-us/web/api/readablestreamdefaultreader/releaselock/index.md @@ -13,16 +13,12 @@ browser-compat: api.ReadableStreamDefaultReader.releaseLock --- {{APIRef("Streams")}} -The **`releaseLock()`** method of the -{{domxref("ReadableStreamDefaultReader")}} interface releases the reader's lock on the -stream. +The **`releaseLock()`** method of the {{domxref("ReadableStreamDefaultReader")}} interface releases the reader's lock on the stream. -If the associated stream is errored when the lock is released, the reader will appear -errored in that same way subsequently; otherwise, the reader will appear closed. +If the associated stream is errored when the lock is released, the reader will appear errored in that same way subsequently; otherwise, the reader will appear closed. -A reader's lock cannot be released while it still has a pending read request, i.e., if -a promise returned by the reader's {{domxref("ReadableStreamDefaultReader.read()")}} -method has not finished. This will result in a `TypeError` being thrown. +If the reader's lock is released while it still has pending read requests then the promises returned by the reader's {{domxref("ReadableStreamDefaultReader.read()")}} method are immediately rejected with a `TypeError`. +Unread chunks remain in the stream's internal queue and can be read later by acquiring a new reader. ## Syntax @@ -41,8 +37,7 @@ None ({{jsxref("undefined")}}). ### Exceptions - {{jsxref("TypeError")}} - - : Thrown if the source object is not a `ReadableStreamDefaultReader`, or a read - request is pending. + - : Thrown if the source object is not a `ReadableStreamDefaultReader`, or a read request is pending. ## Examples diff --git a/files/en-us/web/api/streams_api/concepts/index.md b/files/en-us/web/api/streams_api/concepts/index.md index 5955c831a913864..19480332a73a5a7 100644 --- a/files/en-us/web/api/streams_api/concepts/index.md +++ b/files/en-us/web/api/streams_api/concepts/index.md @@ -34,8 +34,6 @@ Only one reader can read a stream at a time; when a reader is created and starts Note that there are two different types of readable stream. As well as the conventional readable stream there is a type called a byte stream — this is an extended version of a conventional stream for reading underlying byte sources. Compared with the conventional readable stream, byte streams are allowed to be read by BYOB readers (BYOB, "bring your own buffer"). This kind of reader allows streams to be read straight into a buffer supplied by the developer, minimizing the copying required. Which underlying stream (and by extension, reader and controller) your code will use depends on how the stream was created in the first place (see the {{domxref("ReadableStream.ReadableStream","ReadableStream()")}} constructor page). -> **Warning:** Byte streams are not implemented anywhere as yet, and questions have been raised as to whether the spec details are in a finished enough state for them to be implemented. This may change over time. - You can make use of ready-made readable streams via mechanisms like a {{domxref("Response.body")}} from a fetch request, or roll your own streams using the {{domxref("ReadableStream.ReadableStream","ReadableStream()")}} constructor. ## Teeing @@ -60,28 +58,32 @@ An **internal queue** keeps track of the chunks that have been written to the st There is also a construct you'll use called a controller — each writer has an associated controller that allows you to control the stream (for example, to abort it if wished). -![](writable_streams.png) +![Writable streams data flow](writable_streams.png) You can make use of writable streams using the {{domxref("WritableStream.WritableStream","WritableStream()")}} constructor. These currently have very limited availability in browsers. ## Pipe chains -The Streams API makes it possible to pipe streams into one another (or at least it will do when browsers implement the relevant functionality) using a structure called a **pipe chain**. There are two methods available in the spec to facilitate this: +The Streams API makes it possible to pipe streams into one another using a structure called a **pipe chain**. +There are two methods that facilitate this: + +- {{domxref("ReadableStream.pipeThrough()")}} — pipes the stream through a **transform stream**, potentially transforming the data format along the way. + This might be used, for example, to encode or decode video frames, compress or decompress data, or otherwise convert data from one form to another. + + A transform stream consists of a pair of streams: a readable stream from which data is read and a writable stream into which it is written, along with appropriate mechanisms to ensure that new data is made available to read as soon as data is written. -- {{domxref("ReadableStream.pipeThrough()")}} — pipes the stream through a **transform stream**, which is a pair comprised of a writable stream that has data written to it, and a readable stream that then has the data read out of it — this acts as a kind of treadmill that constantly takes data in and transforms it to a new state. Effectively the same stream is written to, and then the same values are read. A simple example is a text decoder, where raw bytes are written, and then strings are read. You can find more useful ideas and examples in the spec — see [Transform streams](https://streams.spec.whatwg.org/#ts-model) for ideas, and [this web sockets example](https://streams.spec.whatwg.org/#example-both). + {{domxref("TransformStream")}} is a concrete implementation of a transform stream, but any object that has the same readable stream and writable stream properties can be passed to `pipeThrough()`. - {{domxref("ReadableStream.pipeTo()")}} — pipes to a writable stream that acts as the end point of the pipe chain. The start of the pipe chain is called the **original source**, and the end is called the **ultimate sink**. -![](pipechain.png) - -> **Note:** This functionality isn't fully thought through yet, or available in many browsers. At some point the spec writers hope to add something like a `TransformStream` class to make creating transform streams easier. +![pipe chain diagram](pipechain.png) ## Backpressure -An important concept in streams is **backpressure** — this is the process by which a single stream or a pipe chain regulates the speed of reading/writing. When a stream later in the chain is still busy and isn't yet ready to accept more chunks, it sends a signal backwards through the chain to tell earlier transform streams (or the original source) to slow down delivery as appropriate so that you don't end up with a bottleneck anywhere. +An important concept in streams is **backpressure** — this is the process by which a single stream or a pipe chain regulates the speed of reading/writing. When a stream later in the chain is still busy and isn't yet ready to accept more chunks, it sends a signal backwards through the chain to tell earlier transform streams (or the original source) to slow down delivery so that you don't end up with a bottleneck anywhere. -To use backpressure in a ReadableStream, we can ask the controller for the chunk size desired by the consumer by querying the {{domxref("ReadableStreamDefaultController.desiredSize")}} property on the controller. If it is too low, our ReadableStream can tell its underlying source to stop sending data, and we backpressure along the stream chain. +To use backpressure in a {{domxref("ReadableStream")}} , we can ask the controller for the chunk size desired by the consumer by querying the {{domxref("ReadableStreamDefaultController.desiredSize")}} property on the controller. If it is too low, our `ReadableStream` can tell its underlying source to stop sending data, and we backpressure along the stream chain. If later on the consumer again wants to receive data, we can use the pull method in the stream creation to tell our underlying source to feed our stream with data. @@ -94,14 +96,15 @@ As mentioned earlier, the chunks in a stream that have not yet been processed an Internal queues employ a **queuing strategy**, which dictates how to signal backpressure based on the **internal queue state.** -In general, the strategy compares the size of the chunks in the queue to a value called the **high water mark**, which is the largest total chunk size that the queue can realistically manage. +In general, the strategy compares the size of the chunks in the queue to a value called the **high water mark**, which is the largest total chunk size that the queue would prefer to manage. The calculation performed is `high water mark - total size of chunks in queue = desired size` -The **desired size** is the size of chunks the stream can still accept to keep the stream flowing but below the high water mark in size. After the calculation is performed, chunk generation will be slowed down/sped up as appropriate to keep the stream flowing as fast as possible while keeping the desired size above zero. If the value falls to zero (or below in the case of writable streams), it means that chunks are being generated faster than the stream can cope with, which results in problems. - -> **Note:** What happens in the case of zero or negative desired size hasn't really been defined in the spec so far. Patience is a virtue. +The **desired size** is the number of chunks the stream can still accept to keep the stream flowing but below the high water mark in size. +Chunk generation will be slowed down/sped up as appropriate to keep the stream flowing as fast as possible while keeping the desired size above zero. +If the value falls to zero (or below), it means that chunks are being generated faster than the stream can cope with, which may result in problems. -As an example, let's take a chunk size of 1, and a high water mark of 3. This means that up to 3 chunks can be enqueued before the high water mark is reached and backpressure is applied. +As an example, let's take a chunk size of 1, and a high water mark of 3. +This means that up to 3 chunks can be enqueued before the high water mark is reached and backpressure is applied. diff --git a/files/en-us/web/api/streams_api/index.md b/files/en-us/web/api/streams_api/index.md index 341ecc27b293357..8d5d983df6dfcae 100644 --- a/files/en-us/web/api/streams_api/index.md +++ b/files/en-us/web/api/streams_api/index.md @@ -37,7 +37,7 @@ More complicated uses involve creating your own stream using the {{domxref("Read You can also write data to streams using {{domxref("WritableStream")}}. -> **Note:** You can find a lot more details about the theory and practice of streams in our articles — [Streams API concepts](/en-US/docs/Web/API/Streams_API/Concepts), [Using readable streams](/en-US/docs/Web/API/Streams_API/Using_readable_streams), and [Using writable streams](/en-US/docs/Web/API/Streams_API/Using_writable_streams). +> **Note:** You can find a lot more details about the theory and practice of streams in our articles — [Streams API concepts](/en-US/docs/Web/API/Streams_API/Concepts), [Using readable streams](/en-US/docs/Web/API/Streams_API/Using_readable_streams), [Using readable byte streams](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams), and [Using writable streams](/en-US/docs/Web/API/Streams_API/Using_writable_streams). ## Stream interfaces @@ -116,4 +116,5 @@ Examples from other developers: - [Streams API concepts](/en-US/docs/Web/API/Streams_API/Concepts) - [Using readable streams](/en-US/docs/Web/API/Streams_API/Using_readable_streams) +- [Using readable byte streams](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams) - [Using writable streams](/en-US/docs/Web/API/Streams_API/Using_writable_streams) diff --git a/files/en-us/web/api/streams_api/using_readable_byte_streams/index.md b/files/en-us/web/api/streams_api/using_readable_byte_streams/index.md new file mode 100644 index 000000000000000..f708afe671cb9e2 --- /dev/null +++ b/files/en-us/web/api/streams_api/using_readable_byte_streams/index.md @@ -0,0 +1,1090 @@ +--- +title: Using readable byte streams +slug: Web/API/Streams_API/Using_readable_byte_streams +page-type: guide +tags: + - API + - Controller + - Fetch + - Guide + - ReadableStreams + - Streams + - pipe chains + - readable streams + - readable byte streams + - reader + - tee +--- +{{apiref("Streams")}} + +Readable _byte streams_ are [readable streams](/en-US/docs/Web/API/Streams_API/Using_readable_streams) that have an underlying byte source of `type: "bytes"`, and which support efficient zero-copy transfer of data from the underlying source to a consumer (bypassing the stream's internal queues). +They are intended for use cases where data might be supplied or requested in arbitrary sized and potentially very large chunks, and hence where avoiding making copies is likely to improve efficiency. + +This article explains how readable byte streams compare to normal "default" streams, and how you create and consume them. + +> **Note:** Readable byte streams are almost identical to "normal" readable streams and almost all of the concepts are the same. +> This article assumes that you already understand those concepts and will only be covering them superficially (if at all). +> If you're not familiar with the relevent concepts, please first read: [Using readable streams](/en-US/docs/Web/API/Streams_API/Using_readable_streams), [Streams concepts and usage overview](/en-US/docs/Web/API/Streams_API#concepts_and_usage), and [Streams API concepts](/en-US/docs/Web/API/Streams_API/Concepts). + +## Overview + +Readable streams provides a consistent interface for streaming data from some underlying source, such as a file or socket, to a consumer, such as a reader, transform stream or writable stream. +In a normal readable stream, data from the underlying source always passes to a consumer through the internal queues. +A readable byte stream differs in that if the internal queues are empty, the underlying source can write directly to the consumer (an efficient zero-copy transfer). + +A readable byte stream is created by specifying `type: "bytes"` in the `underlyingSource` object that may be passed as the first parameter to the [`ReadableStream()` constructor](/en-US/docs/Web/API/ReadableStream/ReadableStream). +With this value set, the stream is created with a {{domxref("ReadableByteStreamController")}}, and this is the object that is passed to the underlying source when the `start(controller)` and `pull(controller)` callback functions are invoked. + +The main difference between {{domxref("ReadableByteStreamController")}} and the default controller ({{domxref("ReadableStreamDefaultController")}}) is that it has an additional property {{domxref("ReadableByteStreamController.byobRequest")}} of type {{domxref("ReadableStreamBYOBRequest")}}. +This represents a pending read request by a consumer that will be made as a zero-copy transfer from the underlying source. +The property will be `null` if there is no pending request. + +A `byobRequest` is only made available when a read request is made on a readable byte stream and there is no data in the stream's internal queues (if there is data then the request is satisfied from those queues). + +An underlying byte source that needs to transfer data must check the `byobRequest` property and, if it is available, use it to transfer data. +If the property is `null`, incoming data should instead be added to the stream's internal queues using {{domxref("ReadableByteStreamController.enqueue()")}} (this is the only way to transfer data when using a "default" stream). + +The {{domxref("ReadableStreamBYOBRequest")}} has a {{domxref("ReadableStreamBYOBRequest.view","view")}} property, which is a view on the buffer allocated for transfer. +Data from an underlying source should be written into this property, and then the underlying source must call {{domxref("ReadableStreamBYOBRequest.respond()","respond()")}} indicating the number of bytes written. +This signals that the data should be transferred, and the pending read request by the consumer resolved. +After calling `respond()` the `view` can no longer be written. + +There is also an additional method {{domxref("ReadableStreamBYOBRequest.respondWithNewView()")}} to which an underlying source can pass a "new" view containing data to be transferred. +This new view must be over the _same_ memory buffer as the original, and from the same starting offset. +This method might be used if the underlying byte source needs to first transfer the view to a worker thread to populate (for example) and then get it back before responding to the `byobRequest`. +In most cases this method will not be needed. + +Readable byte streams are normally read using a {{domxref("ReadableStreamBYOBReader")}}, which can be obtained by calling {{domxref("ReadableStream.getReader()")}} on the stream, specifying `mode: "byob"` in the options parameter. + +A readable byte stream can also be read using a default reader ({{domxref("ReadableStreamDefaultReader")}}), but in this case `byobRequest` objects are only created when automatic buffer allocation is enabled for the stream ([`autoAllocateChunkSize`](/en-US/docs/Web/API/ReadableStream/ReadableStream#autoallocatechunksize) was set for the stream's `underlyingSource`). +Note that the size indicated by `autoAllocateChunkSize` is used for the buffer size in this case; for a byte reader the buffer used is supplied by the consumer. +If the property was not specified, the default reader will still "work" but the underlying source will never be offered a `byobRequest`, and all data will be transferred through the stream's internal queues. + +Other than the differences outlined above, the controller and underlying source for bytes streams are very similar to those for default streams, [and are used in much the same way](/en-US/docs/Web/API/Streams_API/Using_readable_streams). + +## Examples + +### Underlying push source with byte reader + +This live example shows how to create a readable byte stream with a _push_ underlying byte source, and read it using a byte reader. + +Unlike with a pull underlying byte source, data can arrive at any time. +Therefore the underlying source must use `controller.byobRequest` to transfer incoming data if one exists, and otherwise enqueue the data into the stream's internal queues. +Further, since the data can arrive at any time the monitoring behaviour is set up in the `underlyingSource.start()` callback function. + +The example is highly influenced by a push byte source example in the stream specification. +It uses a mocked "hypothetical socket" source that supplies data of arbitrary sizes. +The reader is deliberately delayed at various points to allow the underlying source to use both transfer and enqueing to send data to the stream. +Backpressure support is not demonstrated. + +> **Note:** An underlying byte source can also be used with a default reader. +> If automatic buffer allocation is enabled the controller will supply fixed-size buffers for zero-copy transfers when there is an outstanding request from a reader and the stream's internal queues are empty. +> If automatic buffer allocation is not enabled then all data from the byte stream will always be enqueued. +> This is similar to the behaviour shown in the "pull: underlying byte source examples. + +#### Mocked underlying socket source + +The mocked underlying source has three important methods: + +- `select2()` represents an outstanding request on the socket. + It returns a promise that is resolved when data is available. +- `readInto()` reads data from the socket into a supplied buffer and then clears the data. +- `close()` closes the socket. + +The implementation is very simplistic. +As shown below, `select2()` creates a randomly sized buffer of random data on a timeout. +The created data is read into a buffer then cleared in `readInto()`. + +```js +class MockHypotheticalSocket { + constructor() { + this.max_data = 800; // total amount of data to stream from "socket" + this.max_per_read = 100; // max data per read + this.min_per_read = 40; // min data per read + this.data_read = 0; // total data read so far (capped is maxdata) + this.socketdata = null; // + } + + /* Method returning promise when this socket is readable. */ + select2() { + // Object used to resolve promise + const resultobj = {}; + resultobj["bytesRead"] = 0; + + return new Promise((resolve/*, reject*/) => { + if (this.data_read >= this.max_data) { //out of data + resolve(resultobj); + return; + } + + // Emulate slow read of data + window.setTimeout(() => { + const numberBytesReceived = this.getNumberRandomBytesSocket(); + this.data_read += numberBytesReceived; + this.socketdata = this.randomByteArray(numberBytesReceived); + resultobj["bytesRead"] = numberBytesReceived; + resolve(resultobj); + }, 500); + }); + } + + /* Read data into specified buffer offset */ + readInto(buffer, offset, length) { + let length_data = 0; + if (this.socketdata) { + length_data = this.socketdata.length; + const myview = new Uint8Array(buffer, offset, length); + // Write the length of data specified into buffer + // Code assumes buffer always bigger than incoming data + for (let i = 0; i < length_data; i++) { + myview[i]=this.socketdata[i]; + } + this.socketdata = null; // Clear "socket" data after reading + } + return length_data; + } + + /* Dummy close function */ + close() { + return + } + + /* Return random number bytes in this call of socket */ + getNumberRandomBytesSocket() { + //Capped to remaining data and the max min return-per-read range + const remaining_data = this.max_data - this.data_read; + let numberBytesRecieved = 0; + if (remaining_data < this.min_per_read) { + numberBytesRecieved = remaining_data; + } else { + numberBytesRecieved = this.getRandomIntInclusive(this.min_per_read, Math.min(this.max_per_read, remaining_data)); + } + return numberBytesRecieved; + } + + /* Return random number between two values */ + getRandomIntInclusive(min, max) { + min = Math.ceil(min); + max = Math.floor(max); + return Math.floor(Math.random() * (max - min + 1) + min); + } + + /* Return random character string */ + randomChars(length = 8) { + let string = ""; + let choices = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789!@#$%^&*()"; + + for (let i = 0; i < length; i++) { + string += choices.charAt(Math.floor(Math.random() * choices.length)); + } + return string; + } + + /* Return random Uint8Array of bytes */ + randomByteArray(bytes = 8) { + const textEncoder = new TextEncoder(); + return textEncoder.encode(this.randomChars(bytes)); + } +}; +``` + + + +```css hidden +.input { + float: left; + width: 50%; +} +.output { + float: right; + width: 50%; + overflow-wrap: break-word; +} +button { + display: block; +} +``` + +```html hidden + +
+

Underlying source

+ +
+
+

Consumer

+ +
+``` + +```js hidden +// Store reference to lists, paragraph and button +const list1 = document.querySelector('.input ul'); +const list2 = document.querySelector('.output ul'); +const button = document.querySelector('button'); + +// Create empty string in which to store final result +let result = ""; + +// Function to log data from underlying source +function logSource(result) { + const listItem = document.createElement('li'); + listItem.textContent = result; + list1.appendChild(listItem); +} + +// Function to log data from consumer +function logConsumer(result) { + const listItem = document.createElement('li'); + listItem.textContent = result; + list2.appendChild(listItem); +} +``` + +#### Creating a readable socket push byte stream + +The following code shows how to define a readable socket "push" byte stream. + +The `underlyingSource` object definition is passed as the first parameter to the [`ReadableStream()` constructor](/en-US/docs/Web/API/ReadableStream/ReadableStream). +To make this a readable "byte" stream, we specify `type: "bytes"` as a property of the object. +This ensures that the stream is handed a {{domxref("ReadableByteStreamController")}} (instead of the default controller ({{domxref("ReadableStreamDefaultController")}})) + +Since data can arrive at the socket before the consumer is ready to handle it, everything about reading the underlying source is configured in the `start()` callback method (we don't wait on a pull to start handling data). +The implementation opens the "socket" and calls `select2()` to request data. +When the retured promise resolves the code checks if `controller.byobRequest` exists (is not `null`), and if so calls `socket.readInto()` to copy data into the request and transfer it. +If `byobRequest` does not exist there is no outstanding request from a consuming stream that can be satisifed as as zero-copy transfer. +In this case, `constroller.enqueue()` used to copy data to the stream internal queues. + +The `select2()` request for more data is reposted until a request is returned with no data. +A this point the controller is used to close the stream. + +```js +const stream = makeSocketStream("dummy host", "dummy port") + +const DEFAULT_CHUNK_SIZE = 400; + +function makeSocketStream(host, port) { + const socket = new MockHypotheticalSocket(); + + return new ReadableStream({ + type: "bytes", + + start(controller) { + readRepeatedly().catch(e => controller.error(e)); + + function readRepeatedly() { + return socket.select2().then(() => { + // Since the socket can become readable even when there’s + // no pending BYOB requests, we need to handle both cases. + let bytesRead; + if (controller.byobRequest) { + const v = controller.byobRequest.view; + bytesRead = socket.readInto(v.buffer, v.byteOffset, v.byteLength); + if (bytesRead === 0) { + controller.close(); + } + controller.byobRequest.respond(bytesRead); + logSource(`byobRequest with ${bytesRead} bytes`); + } else { + const buffer = new ArrayBuffer(DEFAULT_CHUNK_SIZE); + bytesRead = socket.readInto(buffer, 0, DEFAULT_CHUNK_SIZE); + if (bytesRead === 0) { + controller.close(); + } else { + controller.enqueue(new Uint8Array(buffer, 0, bytesRead)); + } + logSource(`enqueue() ${bytesRead} bytes (no byobRequest)`); + } + + if (bytesRead === 0) { + return; + // no more bytes in source + } + return readRepeatedly(); + }); + } + }, + + cancel() { + socket.close(); + logSource(`cancel(): socket closed`); + } + }); +} +``` + +Note that `readRepeatedly()` returns a promise, and we use this to catch any errors from setting up or handling the read operation. +The errors are then passed to the controller as shown above (see `readRepeatedly().catch(e => controller.error(e));`). + +A `cancel()` method is provided at the end to close the underlying source; the `pull()` callback is not needed, and is therefore not implemented. + +#### Consuming the push byte stream + +The following code creates a `ReadableStreamBYOBReader` for the socket byte stream and uses it read data into a buffer. +Note `processText()` is called recursively to read more data until the buffer is filled. +When the underlying source signals that it has no more data, the `reader.read()` will have `done` set to true, which in turn completes the read operation. + +This code is almost exactly the same as for the [Underlying pull source with byte reader](#underlying_pull_source_with_byte_reader) example above. +The only difference is that the reader includes some code to slow down reading, so the the log output can demonstrate that data will be enqueued if not read fast enough. + +```js +const reader = stream.getReader({mode: "byob"}); +let buffer = new ArrayBuffer(4000); +readStream(reader); + +function readStream(reader) { + let bytesReceived = 0; + let offset = 0; + + while (offset < buffer.byteLength) { + // read() returns a promise that resolves when a value has been received + reader.read( new Uint8Array(buffer, offset, buffer.byteLength - offset) ).then(async function processText({ done, value }) { + // Result objects contain two properties: + // done - true if the stream has already given all its data. + // value - some data. Always undefined when done is true. + + if (done) { + logConsumer(`readStream() complete. Total bytes: ${bytesReceived}`); + return; + } + + buffer = value.buffer; + offset += value.byteLength; + bytesReceived += value.byteLength; + + //logConsumer(`Read ${bytesReceived} bytes: ${value}`); + logConsumer(`Read ${bytesReceived} bytes`); + result += value; + + // Add delay to emulate when data can't be read and data is enqueued + if (bytesReceived > 300 && bytesReceived < 600) { + logConsumer(`Delaying read to emulate slow stream reading`); + const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms)); + await delay(1000); + } + + // Read some more, and call this function again + return reader.read( new Uint8Array(buffer, offset, buffer.byteLength - offset) ).then(processText); + }); + } +} +``` + +#### Cancelling the stream using the reader + +We can use {{domxref("ReadableStreamBYOBReader.cancel()")}} to cancel the stream. +For this example we call the method if a button is clicked with a reason "user choice" (other HTML and code for the button not shown). +We also log when the cancel operation completes. + +```js +button.addEventListener('click', () => { reader.cancel("user choice").then( () => { logConsumer(`reader.cancel complete`) }) } ); +``` + +{{domxref("ReadableStreamBYOBReader.releaseLock()")}} can be used to release the reader without cancelling the stream. +Note however that any outstanding read requests will immediately be rejected. +A new reader can be acquired later on to read the remaining chunks. + +#### Monitoring for stream for close/error + +The {{domxref("ReadableStreamBYOBReader.closed")}} property returns a promise that will resolve when the stream is closed, and reject if there is an error. +While no errors are expected in this case, the following code should log the completion case. + +```js +reader.closed + .then( () => { logConsumer("ReadableStreamBYOBReader.closed: resolved")} ) + .catch( () => { logConsumer("ReadableStreamBYOBReader.closed: rejected:")} ); +``` + +#### Result + +The logging from the underlying push source (left) and consumer (right) are shown below. +Not the period in the middle where data is equeued rather than transferred as a zero-copy operation. + +{{EmbedLiveSample("Underlying push source with default reader","100%","500px")}} + +### Underlying pull source with byte reader + +This live example shows how data might be read from an "pull" underlying byte source, such as a file, and transferred by a stream as a zero-copy transfer to a {{domxref("ReadableStreamBYOBReader")}}. + +#### Mocked underlying file source + +For the underlying pull source we use the following class to (_very_ superficially) mock a nodejs [`FileHandle`](https://nodejs.org/api/fs.html#class-filehandle), and in particular the [`read()`](https://nodejs.org/api/fs.html#filehandlereadbuffer-offset-length-position) method. +The class generates random data to represent a file. +The `read()` method reads from this data into a provided buffer from the specified position. +The `close()` method does nothing: it is only provided to show where you might close the source when defining the constructor for the stream. + +> **Note:** This same class is used for all the "pull source" examples. +> It is shown here for information only (so that it is obvious that it is a mock). + +```js +class MockUnderlyingFileHandle { + constructor() { + this.maxdata = 1300; // "file size" + this.filedata = this.randomByteArray(this.maxdata); + this.position = 0; + } + + /* Read data from "file" at position/length into specified buffer offset */ + read(buffer, offset, length, position) { + // Object used to resolve promise + const resultobj = {}; + resultobj["buffer"] = buffer; + resultobj["bytesRead"] = 0; + + return new Promise((resolve/*, reject*/) => { + if (position >= this.maxdata) { //out of data + resolve(resultobj); + return; + } + + // Read random data into supplied buffer + const myview = new Uint8Array(buffer, offset, length); + // Write the length of data specified + for (let i = 0; i < length; i++) { + myview[i]=this.filedata[position + i]; + resultobj["bytesRead"] = i; + if (position + i >= this.maxdata) { + break; + } + } + // Emulate slow read of data + window.setTimeout(() => { resolve(resultobj); }, 1000); + }); + } + + /* Dummy close function */ + close() { + return + } + + /* Return random character string */ + randomChars(length = 8) { + let string = ""; + let choices = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789!@#$%^&*()"; + + for (let i = 0; i < length; i++) { + string += choices.charAt(Math.floor(Math.random() * choices.length)); + } + return string; + } + + /* Return random Uint8Array of bytes */ + randomByteArray(bytes = 8) { + const textEncoder = new TextEncoder(); + return textEncoder.encode(this.randomChars(bytes)); + } +}; +``` + + + +```css hidden +.input { + float: left; + width: 50%; +} +.output { + float: right; + width: 50%; + overflow-wrap: break-word; +} +button { + display: block; +} +``` + +```html hidden + +
+

Underlying source

+ +
+
+

Consumer

+ +
+``` + +```js hidden +// Store reference to lists, paragraph and button +const list1 = document.querySelector('.input ul'); +const list2 = document.querySelector('.output ul'); +const button = document.querySelector('button'); + +// Create empty string in which to store final result +let result = ""; + +// Function to log data from underlying source +function logSource(result) { + const listItem = document.createElement('li'); + listItem.textContent = result; + list1.appendChild(listItem); +} + +// Function to log data from consumer +function logConsumer(result) { + const listItem = document.createElement('li'); + listItem.textContent = result; + list2.appendChild(listItem); +} +``` + +#### Creating a readable file byte stream + +The following code shows how to define a readable file byte stream. + +Just as for the previous example, the `underlyingSource` object definition is passed as the first parameter to the [`ReadableStream()` constructor](/en-US/docs/Web/API/ReadableStream/ReadableStream). +To make this a readable "byte" stream, we specify `type: "bytes"` as a property of the object. +This ensures that the stream is handed a {{domxref("ReadableByteStreamController")}}. + +The `start()` function simply opens the file handle, which is then closed in the `cancel()` callback. +`cancel()` is provided to clean up any resources if {{domxref("ReadableStream.cancel()")}} or {{domxref("ReadableStreamDefaultController.close()")}} are called. + +Most of the interesting code is in the `pull()` callback. +This copies data from the file into the pending read request ({{domxref("ReadableByteStreamController.byobRequest")}}) and then calls {{domxref("ReadableStreamBYOBRequest.respond()","respond()")}} to indicate how much data is in the buffer and transfer it. +If 0 bytes were transferred from the file then we know it has all been copied, and call {{domxref("ReadableStreamDefaultController.close()","close()")}} on the controller, which in turn will result in `cancel()` being called on the underlying source. + +```js +const stream = makeReadableByteFileStream("dummy file.txt") + +function makeReadableByteFileStream(filename) { + let fileHandle; + let position = 0; + return new ReadableStream({ + type: "bytes", // An underlying byte stream! + start(controller) { + // Called to initialise the underlying source. + // For a file source open a file handle (here we just create the mocked object). + fileHandle = new MockUnderlyingFileHandle(); + logSource(`start(): ${controller.constructor.name}.byobRequest = ${controller.byobRequest}`) + }, + async pull(controller) { + // Called when there is a pull request for data + const theview = controller.byobRequest.view; + const {bytesRead, buffer} = + await fileHandle.read(theview.buffer, theview.offset, theview.length, position) + if (bytesRead === 0) { + await fileHandle.close(); + controller.close(); + controller.byobRequest.respond(0); + logSource(`pull() with byobRequest. Close controller (read bytes: ${bytesRead})`); + } else { + position += bytesRead; + controller.byobRequest.respond(bytesRead); + logSource(`pull() with byobRequest. Transfer ${bytesRead} bytes`); + } + }, + cancel(reason) { + // This is called if the stream is cancelled (via reader or controller). + // Clean up any resources + fileHandle.close(); + logSource(`cancel() with reason: ${reason}`); + } + }); +} +``` + +#### Consuming the byte stream + +The following code creates a `ReadableStreamBYOBReader` for the file byte stream and uses it read data into a buffer. +Note `processText()` is called recursively to read more data until the buffer is filled. +When the underlying source signals that it has no more data, the `reader.read()` will have `done` set to true, which in turn completes the read operation. + +```js +const reader = stream.getReader({mode: "byob"}); +let buffer = new ArrayBuffer(4000); +readStream(reader); + +function readStream(reader) { + let bytesReceived = 0; + let offset = 0; + + while (offset < buffer.byteLength) { + // read() returns a promise that resolves when a value has been received + reader.read( new Uint8Array(buffer, offset, buffer.byteLength - offset) ).then(function processText({ done, value }) { + // Result objects contain two properties: + // done - true if the stream has already given all its data. + // value - some data. Always undefined when done is true. + + if (done) { + logConsumer(`readStream() complete. Total bytes: ${bytesReceived}`); + return; + } + + buffer = value.buffer; + offset += value.byteLength; + bytesReceived += value.byteLength; + + logConsumer(`Read ${bytesReceived} bytes: ${value}`); + result += value; + + // Read some more, and call this function again + return reader.read( new Uint8Array(buffer, offset, buffer.byteLength - offset) ).then(processText); + }); + } +} +``` + +Lastly, we add a handler that will cancel the stream if a button is clicked (other HTML and code for the button not shown). + +```js +button.addEventListener('click', () => { reader.cancel("user choice").then( () => { logConsumer(`reader.cancel complete`) }) } ); +``` + +#### Result + +The logging from the underlying pull source (left) and consumer (right) are shown below. +Of particular note are that the: + +- `start()` function is passed a `ReadableByteStreamController` +- the buffer passed to the reader is large enough to encompass the whole "file", so the whole file is transferred in one operation. + +{{EmbedLiveSample("Underlying pull source","100%","500px")}} + +### Underlying pull source with default reader + +This live example shows how the same data might be read as a zero-copy transfer using a default reader ({{domxref("ReadableStreamDefaultReader")}}). +This uses the same same [mocked underlying file source](#mocked_underlying_file_source) as in the preceding example. + +```js hidden +class MockUnderlyingFileHandle { + constructor() { + this.maxdata = 1300; // "file size" + this.filedata = this.randomByteArray(this.maxdata); + this.position = 0; + } + + /* Read data from "file" at position/length into specified buffer offset */ + read(buffer, offset, length, position) { + // Object used to resolve promise + const resultobj = {}; + resultobj["buffer"] = buffer; + resultobj["bytesRead"] = 0; + + return new Promise((resolve/*, reject*/) => { + if (position >= this.maxdata) { //out of data + resolve(resultobj); + return; + } + + // Read random data into supplied buffer + const myview = new Uint8Array(buffer, offset, length); + // Write the length of data specified + for (let i = 0; i < length; i++) { + myview[i]=this.filedata[position + i]; + resultobj["bytesRead"] = i; + if (position + i >= this.maxdata) { + break; + } + } + // Emulate slow read of data + window.setTimeout(() => { resolve(resultobj); }, 1000); + }); + } + + /* Dummy close function */ + close() { + return + } + + /* Return random character string */ + randomChars(length = 8) { + let string = ""; + let choices = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789!@#$%^&*()"; + + for (let i = 0; i < length; i++) { + string += choices.charAt(Math.floor(Math.random() * choices.length)); + } + return string; + } + + /* Return random Uint8Array of bytes */ + randomByteArray(bytes = 8) { + const textEncoder = new TextEncoder(); + return textEncoder.encode(this.randomChars(bytes)); + } +}; +``` + + + +```css hidden +.input { + float: left; + width: 50%; +} +.output { + float: right; + width: 50%; + overflow-wrap: break-word; +} +button { + display: block; +} +``` + +```html hidden + +
+

Underlying source

+ +
+
+

Consumer

+ +
+ +``` + +```js hidden +// Store reference to lists, paragraph and button +const list1 = document.querySelector('.input ul'); +const list2 = document.querySelector('.output ul'); +const button = document.querySelector('button'); + +// Create empty string in which to store final result +let result = ""; + +// Function to log data from underlying source +function logSource(result) { + const listItem = document.createElement('li'); + listItem.textContent = result; + list1.appendChild(listItem); +} + +// Function to log data from consumer +function logConsumer(result) { + const listItem = document.createElement('li'); + listItem.textContent = result; + list2.appendChild(listItem); +} +``` + +#### Creating a readable file byte stream with automatic buffer allocation + +The only difference in our underlying source is that we must specify `autoAllocateChunkSize`, and that the size will be used as the view buffer size for `controller.byobRequest`, rather than one supplied by the consumer. + +```js +const DEFAULT_CHUNK_SIZE = 200; +const stream = makeReadableByteFileStream("dummy file.txt") + +function makeReadableByteFileStream(filename) { + let fileHandle; + let position = 0; + return new ReadableStream({ + type: "bytes", // An underlying byte stream! + start(controller) { + // Called to initialise the underlying source. + // For a file source open a file handle (here we just create the mocked object). + fileHandle = new MockUnderlyingFileHandle(); + logSource(`start(): ${controller.constructor.name}.byobRequest = ${controller.byobRequest}`) + }, + async pull(controller) { + // Called when there is a pull request for data + const theview = controller.byobRequest.view; + const {bytesRead, buffer} = + await fileHandle.read(theview.buffer, theview.offset, theview.length, position) + if (bytesRead === 0) { + await fileHandle.close(); + controller.close(); + controller.byobRequest.respond(0); + logSource(`pull() with byobRequest. Close controller (read bytes: ${bytesRead})`); + } else { + position += bytesRead; + controller.byobRequest.respond(bytesRead); + logSource(`pull() with byobRequest. Transfer ${bytesRead} bytes`); + } + }, + cancel(reason) { + // This is called if the stream is cancelled (via reader or controller). + // Clean up any resources + fileHandle.close(); + logSource(`cancel() with reason: ${reason}`); + }, + autoAllocateChunkSize: DEFAULT_CHUNK_SIZE // Only relevant if using a default reader + }); +} +``` + +#### Consuming the byte stream with default reader + +The following code creates a {{domxref("ReadableStreamDefaultReader")}} for the file byte stream by calling `stream.getReader();` without specifying the mode, and uses it read data into a buffer. +The operation of the code is the same as the previous example except that the buffer is supplied by the stream rather than the consumer. + +```js +const reader = stream.getReader(); +readStream(reader); + +function readStream(reader) { + let bytesReceived = 0; + let result = ''; + + // read() returns a promise that resolves + // when a value has been received + reader.read().then(function processText({ done, value }) { + // Result objects contain two properties: + // done - true if the stream has already given you all its data. + // value - some data. Always undefined when done is true. + if (done) { + logConsumer(`readStream() complete. Total bytes: ${bytesReceived}`); + return; + } + + bytesReceived += value.length; + logConsumer(`Read ${bytesReceived} bytes so far. Current bytes = ${value}`); + result += value; + + // Read some more, and call this function again + return reader.read().then(processText); + }); +} +``` + +Lastly, we add a handler that will cancel the stream if a button is clicked (other HTML and code for the button not shown). + +```js +button.addEventListener('click', () => { reader.cancel("user choice").then( () => { logConsumer(`reader.cancel complete`) }) } ); +``` + +#### Result + +The logging from the underlying bye pull source (left) and consumer (right) are shown below. + +Note that the chunks are now 200-byte wide, as specified in the underlying byte source. +These are made as zero-copy transfers. + +{{EmbedLiveSample("Underlying pull source with default reader","100%","500px")}} + +### Underlying pull source with default reader and no allocation + +For completeness, we can also use a default reader with a byte source that does not support automatic buffer allocation. + +```js hidden +class MockUnderlyingFileHandle { + constructor() { + this.maxdata = 1300; // "file size" + this.filedata = this.randomByteArray(this.maxdata); + this.position = 0; + } + + /* Read data from "file" at position/length into specified buffer offset */ + read(buffer, offset, length, position) { + // Object used to resolve promise + const resultobj = {}; + resultobj["buffer"] = buffer; + resultobj["bytesRead"] = 0; + + return new Promise((resolve/*, reject*/) => { + if (position >= this.maxdata) { //out of data + resolve(resultobj); + return; + } + + // Read random data into supplied buffer + const myview = new Uint8Array(buffer, offset, length); + // Write the length of data specified + for (let i = 0; i < length; i++) { + myview[i]=this.filedata[position + i]; + resultobj["bytesRead"] = i; + if (position + i >= this.maxdata) { + break; + } + } + // Emulate slow read of data + window.setTimeout(() => { resolve(resultobj); }, 1000); + }); + } + + /* Dummy close function */ + close() { + return + } + + /* Return random character string */ + randomChars(length = 8) { + let string = ""; + let choices = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789!@#$%^&*()"; + + for (let i = 0; i < length; i++) { + string += choices.charAt(Math.floor(Math.random() * choices.length)); + } + return string; + } + + /* Return random Uint8Array of bytes */ + randomByteArray(bytes = 8) { + const textEncoder = new TextEncoder(); + return textEncoder.encode(this.randomChars(bytes)); + } +}; +``` + + + +```css hidden +.input { + float: left; + width: 50%; +} +.output { + float: right; + width: 50%; + overflow-wrap: break-word; +} + button { + display: block; +} +``` + +```html hidden + +
+

Underlying source

+ +
+
+

Consumer

+ +
+``` + +```js hidden +// Store reference to lists, paragraph and button +const list1 = document.querySelector('.input ul'); +const list2 = document.querySelector('.output ul'); +const button = document.querySelector('button'); + +// Create empty string in which to store final result +let result = ""; + +// Function to log data from underlying source +function logSource(result) { + const listItem = document.createElement('li'); + listItem.textContent = result; + list1.appendChild(listItem); +} + +// Function to log data from consumer +function logConsumer(result) { + const listItem = document.createElement('li'); + listItem.textContent = result; + list2.appendChild(listItem); +} +``` + +However in this case the controller will not supply a `byobRequest` for the underlying source to write into. +Instead the underlying source would have to enqueue the data. +Note below that to support this case, in `pull()` we need to check if the `byobRequest` exists. + +```js +const stream = makeReadableByteFileStream("dummy file.txt") +const DEFAULT_CHUNK_SIZE = 300; + +function makeReadableByteFileStream(filename) { + let fileHandle; + let position = 0; + return new ReadableStream({ + type: "bytes", // An underlying byte stream! + start(controller) { + // Called to initialise the underlying source. + // For a file source open a file handle (here we just create the mocked object). + fileHandle = new MockUnderlyingFileHandle(); + logSource(`start(): ${controller.constructor.name}.byobRequest = ${controller.byobRequest}`) + }, + async pull(controller) { + // Called when there is a pull request for data + if (controller.byobRequest) { + const theview = controller.byobRequest.view; + const {bytesRead, buffer} = await fileHandle.read(theview.buffer, theview.offset, theview.length, position) + if (bytesRead === 0) { + await fileHandle.close(); + controller.close(); + controller.byobRequest.respond(0); + logSource(`pull() with byobRequest. Close controller (read bytes: ${bytesRead})`); + } else { + position += bytesRead; + controller.byobRequest.respond(bytesRead); + logSource(`pull() with byobRequest. Transfer ${bytesRead} bytes`); + } + } else { + // No BYOBRequest so enqueue data to stream + // NOTE, this branch would only execute for a default reader if autoAllocateChunkSize is not defined. + const mynewBuffer = new Uint8Array(DEFAULT_CHUNK_SIZE); + const {bytesRead, buffer} = await fileHandle.read(mynewBuffer.buffer, mynewBuffer.offset, mynewBuffer.length, position); + if (bytesRead === 0) { + await fileHandle.close(); + controller.close(); + controller.enqueue(mynewBuffer); + logSource(`pull() with no byobRequest. Close controller (read bytes: ${bytesRead})`); + } else { + position += bytesRead; + controller.enqueue(mynewBuffer); + logSource(`pull() with no byobRequest. enqueue() ${bytesRead} bytes`); + } + + } + }, + cancel(reason) { + // This is called if the stream is cancelled (via reader or controller). + // Clean up any resources + fileHandle.close(); + logSource(`cancel() with reason: ${reason}`); + } + }); +} +``` + +```js hidden +const reader = stream.getReader(); + +readStream(reader); + +function readStream(reader) { + let bytesReceived = 0; + let result = ''; + + // read() returns a promise that resolves + // when a value has been received + reader.read().then(function processText({ done, value }) { + // Result objects contain two properties: + // done - true if the stream has already given you all its data. + // value - some data. Always undefined when done is true. + if (done) { + logConsumer(`readStream() complete. Total bytes: ${bytesReceived}`); + return; + } + + bytesReceived += value.length; + logConsumer(`Read ${bytesReceived} bytes so far. Current bytes = ${value}`); + result += value; + + // Read some more, and call this function again + return reader.read().then(processText); + }); +} +``` + +```js hidden +button.addEventListener('click', () => { reader.cancel("user choice").then( () => { logConsumer(`reader.cancel complete`) }) } ); +``` + +#### Result + +The logging from the underlying pull source (left) and consumer (right) are shown below. +Note that the underlying source side shows that the data has been enqueued rather than zero-byte transferred. + +{{EmbedLiveSample("Underlying pull source with default reader and no allocation","100%","500px")}} + +## See also + +- [Streams API concepts](/en-US/docs/Web/API/Streams_API/Concepts) +- [Streams concepts and usage overview](/en-US/docs/Web/API/Streams_API#concepts_and_usage) +- [Using readable streams](/en-US/docs/Web/API/Streams_API/Using_readable_streams) diff --git a/files/en-us/web/api/streams_api/using_readable_streams/index.md b/files/en-us/web/api/streams_api/using_readable_streams/index.md index e97442c5905c641..b3e22032eb5b7ce 100644 --- a/files/en-us/web/api/streams_api/using_readable_streams/index.md +++ b/files/en-us/web/api/streams_api/using_readable_streams/index.md @@ -24,7 +24,8 @@ As a JavaScript developer, programmatically reading and manipulating streams of ## Browser support -You can consume Fetch body objects as streams and create your own custom readable streams in Firefox 65+ and Chrome 42+ (and equivalent Chromium-based browsers). [Pipe chains](/en-US/docs/Web/API/Streams_API/Concepts#pipe_chains) are only supported in Chrome at the moment, and that functionality is subject to change. +You can consume Fetch body objects as streams and create your own custom readable streams most current browsers. +[Pipe chain](/en-US/docs/Web/API/Streams_API/Concepts#pipe_chains) support is still not universal, and it may be worth checking compatibility tables (for example, see {{domxref("ReadableStream.pipeThrough()")}}). ## Finding some examples @@ -331,11 +332,9 @@ function teeStream() { ## Pipe chains -One very experimental feature of streams is the ability to pipe streams into one another (called a [pipe chain](/en-US/docs/Web/API/Streams_API/Concepts#pipe_chains)). This involves two methods — {{domxref("ReadableStream.pipeThrough()")}}, which pipes a readable stream through a writer/reader pair to transform one data format into another, and {{domxref("ReadableStream.pipeTo()")}}, which pipes a readable stream to a writer acting as an end point for the pipe chain. +Another feature of streams is the ability to pipe streams into one another (called a [pipe chain](/en-US/docs/Web/API/Streams_API/Concepts#pipe_chains)). This involves two methods — {{domxref("ReadableStream.pipeThrough()")}}, which pipes a readable stream through a writer/reader pair to transform one data format into another, and {{domxref("ReadableStream.pipeTo()")}}, which pipes a readable stream to a writer acting as an end point for the pipe chain. -This functionality is at a very experimental stage and is subject to change, so we have no explored it too deeply as of yet. - -We have created an example called [Unpack Chunks of a PNG](https://github.com/mdn/dom-examples/tree/master/streams/png-transform-stream) ([see it live also](https://mdn.github.io/dom-examples/streams/png-transform-stream/)) that fetches an image as a stream, then pipes it through to a custom PNG transform stream that retrieves PNG chunks out of a binary data stream. +We do have have a simple example called [Unpack Chunks of a PNG](https://github.com/mdn/dom-examples/tree/master/streams/png-transform-stream) ([see it live also](https://mdn.github.io/dom-examples/streams/png-transform-stream/)) that fetches an image as a stream, then pipes it through to a custom PNG transform stream that retrieves PNG chunks out of a binary data stream. ```js // Fetch the original image @@ -348,6 +347,10 @@ fetch('png-logo.png') .then(rs => logReadableStream('PNG Chunk Stream', rs)) ``` +We don't yet have an example that uses {{domxref("TransformStream")}}. + ## Summary -That explains the basics of "default" readable streams. We'll explain bytestreams in a separate future article, once they are available in browsers. +That explains the basics of "default" readable streams. + +See [Using readable byte streams](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams) for information about how to use readable _byte_ streams: streams with an underlying byte source that can perform efficient zero-copy transfers to a consumer, bypassing the stream's internal queues. diff --git a/files/en-us/web/api/streams_api/using_writable_streams/index.md b/files/en-us/web/api/streams_api/using_writable_streams/index.md index cc2b254846d33cf..501039bd3b0ae8c 100644 --- a/files/en-us/web/api/streams_api/using_writable_streams/index.md +++ b/files/en-us/web/api/streams_api/using_writable_streams/index.md @@ -18,7 +18,7 @@ As a JavaScript developer, programmatically writing data to a stream is very use > **Note:** This article assumes that you understand the use cases of writable streams, and are aware of the high-level concepts. > If not, we suggest that you first read the [Streams concepts and usage overview](/en-US/docs/Web/API/Streams_API#concepts_and_usage) and dedicated [Streams API concepts](/en-US/docs/Web/API/Streams_API/Concepts) article, then come back. -> **Note:** If you are looking for information about readable streams, try [Using readable streams](/en-US/docs/Web/API/Streams_API/Using_readable_streams) instead. +> **Note:** If you are looking for information about readable streams, try [Using readable streams](/en-US/docs/Web/API/Streams_API/Using_readable_streams) and [Using readable byte streams](/en-US/docs/Web/API/Streams_API/Using_readable_byte_streams) instead. ## Introducing an example diff --git a/files/en-us/web/api/transformstream/index.md b/files/en-us/web/api/transformstream/index.md index 03bddb49dca076f..bbd0b451492e4c2 100644 --- a/files/en-us/web/api/transformstream/index.md +++ b/files/en-us/web/api/transformstream/index.md @@ -11,21 +11,27 @@ browser-compat: api.TransformStream --- {{APIRef("Streams")}} -The `TransformStream` interface of the [Streams API](/en-US/docs/Web/API/Streams_API) represents a set of transformable data. +The `TransformStream` interface of the [Streams API](/en-US/docs/Web/API/Streams_API) represents a concrete implementation of the [pipe chain](/en-US/docs/Web/API/Streams_API/Concepts#pipe_chains) *transform stream* concept. + +It may be passed to the {{domxref("ReadableStream.pipeThrough()")}} method in order to transform a stream of data from one format into another. +For example, it might be used to decode (or encode) video frames, decompress data, or convert the stream from XML to JSON. + +A transformation algorithm may be provided as an optional argument to the object constructor. +If not supplied, data is not modified when piped through the stream. `TransformStream` is a {{glossary("Transferable objects","transferable object")}}. ## Constructor - {{domxref("TransformStream.TransformStream", "TransformStream()")}} - - : Creates and returns a transform stream object from the given handlers. + - : Creates and returns a transform stream object, optionally specifying a transformation object and queuing strategies for the streams. ## Properties - {{domxref("TransformStream.readable")}} {{readonlyInline}} - - : The `readable` end of a TransformStream. + - : The `readable` end of a `TransformStream`. - {{domxref("TransformStream.writable")}} {{readonlyInline}} - - : The `writable` end of a TransformStream. + - : The `writable` end of a `TransformStream`. ## Methods diff --git a/files/jsondata/GroupData.json b/files/jsondata/GroupData.json index a366863d8c35184..fded694671ac90c 100644 --- a/files/jsondata/GroupData.json +++ b/files/jsondata/GroupData.json @@ -1253,6 +1253,7 @@ "guides": [ "/docs/Web/API/Streams_API/Concepts", "/docs/Web/API/Streams_API/Using_readable_streams", + "/docs/Web/API/Streams_API/Using_readable_byte_streams", "/docs/Web/API/Streams_API/Using_writable_streams" ], "interfaces": [