Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds sendWithStream method in MessageHandler. #8430

Merged
merged 1 commit into from
Jun 9, 2017

Conversation

mukulmishra18
Copy link
Contributor

This is second PR for the project Streams API in PDF.js.

This PR adds a method sendWithStream in MessageHandler at shared/utils. We can use this method to pass data between main and worker thread using Streams API. This method returns a ReadableStream.
The behaviour of stream is just like the standard ReadableStream Constructor.

This PR also contains tests for the functionalities of stream underlyingSource(pull, cancel) and streamSink(read data with blocking promise, error).

Copy link
Collaborator

@Snuffleupagus Snuffleupagus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've left a couple of comments, based on a very cursory look at the code.

Also, I'm wondering if there's any reason that the new unit-tests cannot be run on Travis?
Otherwise, let's add the new test file to https://github.com/mozilla/pdf.js/blob/master/test/unit/clitests.json.

@@ -1219,16 +1219,111 @@ function MessageHandler(sourceName, targetName, comObj) {
this.targetName = targetName;
this.comObj = comObj;
this.callbackIndex = 1;
this.streamId = 1;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just above there's callbackIndex, so can we please use the same same suffix for both callback and stream to reduce confusion here!?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, sure. I will change it to streamIndex

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, sure. I will change it to streamIndex

Thanks, but I now notice that this may become problematic with respect to the line length limit; it may be simpler to keep this.streamId and just change this.callbackIndex -> this.callbackId instead :-)

Looking at the existing code, the callbackId name is actually already used in most places, except here: https://github.com/mozilla/pdf.js/blob/master/src/shared/util.js#L1313.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but I now notice that this may become problematic with respect to the line length limit

"line length limit"? I think i am not getting this point, can you please elaborate?

Copy link
Collaborator

@Snuffleupagus Snuffleupagus May 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just noticed that there's a couple of fairly long lines in the patch already (for example this one), and it's possible that using streamIndex instead of streamId everywhere could perhaps unnecessarily go over the maximum limit of 80 characters per line (as enforced by ESLint).

However, it seems that we already use callbackId in most places in the existing code, hence I'm suggesting to keep this.streamId in this patch and just change the two existing occurrences of this.callbackIndex to this.callbackId instead.

Sorry about giving confusing/conflicting advice here!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm suggesting to keep this.streamId in this patch and just change the two existing occurrences of this.callbackIndex to this.callbackId instead.

Okay, sure. Thanks for clarification here.

let streamId = data.streamId;
switch (data.stream) {
case 'start_complete':
if (data.success === true || data.success === 'true') {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks slightly awkward to me; and why would/should success ever be anything other than a boolean?
Looking below at the postMessage calls, success is always a boolean, so please simplify this to if (data.success) { instead.

Please note: This comment applies in multiple places below as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to be at safer side, i thought maybe somehow while sending data between main and worker, success will change to string. But i am not sure.
Thanks for your suggestion, i will change this to if (data.success) {. This looks more clear.

Copy link
Collaborator

@Snuffleupagus Snuffleupagus May 20, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to be at safer side, i thought maybe somehow while sending data between main and worker, success will change to string. But i am not sure.

That shouldn't be a problem, since the structured cloning algorithm maintains types (and throws on unsupported types).

If you want to check for yourself, look at the value/type of data.isReply in the current code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That shouldn't be a problem, since the structured cloning algorithm maintains types (and throws on unsupported types).

Okay right, we are using structured cloning algorithm in postMessage.
Thanks for clarification.

data.desiredSize > 0) {
streamSinks[data.streamId].sinkCapability.resolve();
}
// reset desiredSize property of sink on every pull.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: For improved readability of comments, please remember to use proper capitalization at the start of sentences.

Please note: This probably applies elsewhere in the patch as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, sure. Thanks.

break;
case 'pull':
// pull increases the desiredSize property of sink, so when it
// changes from -ve to +ve set ready property as resolved promise.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless one already knows what ve means, then this comment is unfortunately not very helpful. (This applies to another comment below as well.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, i will make it negative/positive instead.

let streamSink = {
enqueue(chunk, size) {
var lastDesiredSize = this.desiredSize;
this.desiredSize -= size !== undefined ? size : 1;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not change the signature to enqueue(chunk, size = 1) { instead, since this line can then be simplified to this.desiredSize -= size?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, this is short and more clear. We will do this way.

* @param {Object} queueingStrategy strategy to signal backpressure based on
* internal queue.
* @param {Array} [transfers] Optional list of transfers/ArrayBuffers.
* @return {ReadableStream} ReadableStream to read data in chunks.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Let's make this more consistent with how we generally write JSDoc comments (note that the other JSDoc comments in MessageHandler aren't entirely correct):

* @param {string} actionName - Action to call.
* @param {JSON} data - JSON data to send.
* @param {Object} queueingStrategy - strategy to signal backpressure based on
*   internal queue.
* @param {Array} [transfers] - Optional list of transfers/ArrayBuffers.
* @return {ReadableStream} ReadableStream to read data in chunks.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note that the other JSDoc comments in MessageHandler aren't entirely correct

I referred those comments to write this :)
Okay, i will do as you suggested, and also change other comments in MessageHandler to follow this.

* @return {ReadableStream} ReadableStream to read data in chunks.
*/
sendWithStream(actionName, data, queueingStrategy, transfers) {
var streamId = this.streamId++;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please reduce the indentation to only two spaces within this method.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, sure.

@@ -14,52 +14,52 @@
*/

import {
ReadableStream, removeNullCharacters, stringToPDFString
ReadableStream, removeNullCharacters, stringToPDFString
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unnecessary change, please revert.


import { MessageHandler } from '../../src/shared/util';

describe('util stream', function () {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Please make this util_stream instead, since that's consistent with existing tests.

}

postMessage(obj) {
let e = {data: obj};
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use a more descriptive variable name than e.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, Thanks for your suggestions.

@mukulmishra18 mukulmishra18 force-pushed the sendWithStream branch 2 times, most recently from 0d9797b to 2dd23b0 Compare May 21, 2017 09:34
@mukulmishra18
Copy link
Contributor Author

Okay, but why it is failing the tests :-(

gulp unittestcli is just running fine on my local machine, with result:
412 specs, 0 failures, 11 pending specs

@Snuffleupagus
Copy link
Collaborator

Snuffleupagus commented May 21, 2017

It seems that one of the tests fail on Travis, and I've noticed that while all tests pass locally, there still seem to be some errors regarding accessing removed properties.

Note that you can easily run all (or part) of the unit-tests locally in a browser, by executing gulp server and then pointing your browser to http://localhost:8888/test/unit/unit_test.html. It's also possible to run part of the tests as well, using e.g. http://localhost:8888/test/unit/unit_test.html?spec=util_stream.
Doing that locally, I get two TypeErrors in the browser console, as indicated below.

TypeError: streamControllers[data.streamId] is undefined[Learn More] system.js%20line%204%20%3E%20eval:1206:19
TypeError: streamSinks[data.streamId] is undefined[Learn More] system.js%20line%204%20%3E%20eval:1215:1

This suggests that currently there's code trying to access certain properties after a delete ... expression has been run. Those errors need to be dealt with, since it indicates that the ordering of the various stream events isn't the expected one. (Fixing this might also fix the failure on Travis, since that error looks similar.)

@mukulmishra18 mukulmishra18 force-pushed the sendWithStream branch 3 times, most recently from 31e7fe6 to 572a661 Compare May 21, 2017 20:30
@mukulmishra18
Copy link
Contributor Author

This suggests that currently there's code trying to access certain properties after a delete ... expression has been run.

Yes, you are right. Actually, as desiredSize property of sink in greater than sum of size of all data chunks, so it is not blocking the stream(we are using sink.ready property to block the stream when queue is full). Due to this, it is calling sink.close here, before calling pull. Decreasing the size of desiredSize is solving this problem.

Thanks for your help here :)

@Snuffleupagus
Copy link
Collaborator

Snuffleupagus commented May 22, 2017

Testing the latest version of the patch locally (on Windows), I get different behaviour in Firefox vs Chrome:

  • In Firefox Nightly (version 55), there's a Expected '012p' to equal '012' failure in the should read data with blocking promise and buffer whole data into stream test.
  • In Chrome Beta (version 59), the new unit-tests pass.

This unfortunately suggests that there're issues either in the streams polyfill, the MessageHandler implementation, or the unit-tests; or perhaps some combination of them.
(It might just be a timing issue with the e.g. the sleep function used in the unit-tests, or it might point to a deeper issue with regards to cross-browser compatibility.)

@mukulmishra18
Copy link
Contributor Author

mukulmishra18 commented May 22, 2017

Testing the latest version of the patch locally (on Windows), I get different behaviour in Firefox vs Chrome:

  • In Firefox Nightly (version 55), there's a Expected '012p' to equal '012' failure in the should read data with blocking promise and buffer whole data into stream test.
  • In Chrome Beta (version 59), the new unit-tests pass.

Yes, you are right, i have Chrome(version 52) locally, and it is failing on this also.

But in my opinion, 012p would be the correct value for log, as value of highWaterMark is 8, enqueue of chunk here does not fill the internal queue completely, so it will call the pull to get next chunk of data.

@mukulmishra18
Copy link
Contributor Author

It might just be a timing issue with the e.g. the sleep function used in the unit-tests, or it might point to a deeper issue with regards to cross-browser compatibility.

I am not sure, but potentially this seems to be a timing problem. Running the test with different timeout value(e.g. sleep(0), or sleep(1)[1]), gives different log values. I also think, this is not a fault of sleep function itself(as instead of sleep, i tried setTimeout function also).

[1] note: sleep(1) and sleep(10) has no difference on log values

@mukulmishra18 mukulmishra18 force-pushed the sendWithStream branch 3 times, most recently from 2147c2b to 6dc7dc5 Compare May 29, 2017 08:20
Copy link
Contributor

@yurydelendik yurydelendik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also rebase to new master.

}).then(() => {
sink.close();
});
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return sleep(5); before exiting handler -- we need to be sure first sink.ready promise is resolved. Few place below needs that too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

expect(result.value).toEqual([1, 2, 3, 4]);
expect(result.done).toEqual(false);
return sleep(10);
}).then(() => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add reason param here and use it instead of log for expect below.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay sure, done in new commit

@mukulmishra18 mukulmishra18 force-pushed the sendWithStream branch 2 times, most recently from 34ae2f5 to 834e634 Compare May 31, 2017 21:09
});

it('should ignore any pull after close is called', function (done) {
function createPromiseCapability() {
Copy link
Collaborator

@Snuffleupagus Snuffleupagus May 31, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are you duplicating this here? Please import it from shared/util.js instead!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, sure. Thanks.

let streamId = data.streamId;
let sendStreamResponse = ({ stream, success }) => {
comObj.postMessage({ sourceName, targetName, stream,
success, streamId });
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: The indentation is off here, please alight the parameters like this:

comObj.postMessage({ sourceName, targetName, stream,
                     success, streamId });

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

let capability = createPromiseCapability();
let sendStreamRequest = ({ stream, chunk, reason, success }) => {
comObj.postMessage({ sourceName, targetName, stream, streamId,
chunk, reason, success });
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

@mukulmishra18 mukulmishra18 May 31, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@timvandermeij
Copy link
Contributor

timvandermeij commented May 31, 2017

I have a remark that is related to, but more general than this pull request. The unit tests are clear and help understand what the Streams API does (because I'm not familiar with it yet myself), but I think it would really help (since this is going to be one of the core components) to have some documentation about the terminology, such as "high watermark", "sink" and "queueing strategy". I think you have a really good reference with your write-up at https://docs.google.com/document/d/1yTPvPBhGkFn7XUntAt8wdVPckxHovYylNAIJOGnGRXc/edit and just extending that should already suffice. This is not something that needs to be done right away, but I just wanted to let you know that I think that it would help a lot with understanding the code.

Other than that, this looks promising. Keep up the nice work!

if (data.isReply) {
var callbackId = data.callbackId;
if (data.stream) {
let sourceName = this.sourceName;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extract entire if (data.stream) into separate method e.g. '_processStreamMessage(data)'

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, done in new commit.

@@ -1269,6 +1357,51 @@ function MessageHandler(sourceName, targetName, comObj) {
error: reason
});
});
} else if (data.streamId) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extract entire if (data.streamId) into separate method e.g. '_createStreamSink(data)'

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, thanks for suggestions.

let sourceName = this.sourceName;
let targetName = data.sourceName;
let streamId = data.streamId;
let sendStreamResponse = ({ stream, success }) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here and at sendStreamRequest add 'reason' parameter after success -- it shall be populated/used when success == false. reject on xxx_complete side shall accept this reason as argument.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

success, streamId });
};
let deleteStreamController = () => {
Promise.all([streamControllers[data.streamId].startCall,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: startCall on the new line

add .cancelCall promise here too

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@mukulmishra18
Copy link
Contributor Author

but I think it would really help (since this is going to be one of the core components) to have some documentation about the terminology, such as "high watermark", "sink" and "queueing strategy".

Yes you are right, I am also thinking about the same. Having documentation about the code and terminology will help to understand it better, even i can understand later what i have done :-D
I will try to update the Streams API for PDF.js proposal as we proceed with the project. I think for documenting the terminology related to Streams API, i can take reference from Steams Spec.

Other than that, this looks promising. Keep up the nice work!

Thank You :)

@timvandermeij
Copy link
Contributor

I did not notice the specification yet, but it's surprisingly good to read and understand. It's a really helpful resource (that I see you already mentioned in your proposal) and I think extracting the most important terminology from it is a good idea. Thank you for considering that; I'll leave it for now (to not digress from the topic of this particular PR) and watch how to proposal progresses.

Copy link
Contributor

@yurydelendik yurydelendik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To improve readability and error handling

let callbacksCapabilities = this.callbacksCapabilities = Object.create(null);
let ah = this.actionHandler = Object.create(null);

let _processStreamMessage = (data) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, but I wonder if we can move _processStreamMessage and _createStreamSink as a method of MessageHandler (we have this.comObj, this.streamSinks and this.streamControllers available to us)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in new commit.

Copy link
Collaborator

@Snuffleupagus Snuffleupagus Jun 2, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might be totally misunderstanding, but I'm interpreting #8430 (comment) as suggesting that they be moved into MessageHandler.prototype instead (since the updated patch doesn't really change anything otherwise)!?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might be totally misunderstanding, but I'm interpreting #8430 (comment) as suggesting that they be moved into MessageHandler.prototype instead (since the updated patch doesn't really change anything otherwise)!?

I think i misunderstood #8439 (comment), sorry for that. I will move _processStreamMessage and _createStreamSink to MessageHandler.prototype

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just thought earlier that maybe doing something like: let _createStreamSink = this._createStreamSink = () => {...} will make _createStreamSink a method of MessageHandler.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it will, but from performance point of view this methods without closure (in this case members of the prototype) will be more effective and easier to debug.

streamControllers[data.streamId].startCall.resolve();
} else {
streamControllers[data.streamId].startCall.reject();
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add utility function let resolveOrReject = (capability, success, reason) => { if (success) { capability.resolve(); ... and use it at xxxx_complete branches

Copy link
Contributor Author

@mukulmishra18 mukulmishra18 Jun 1, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we do this like: let resolveOrReject = ({ capability, success, reason }) => { if (success) { capability.resolve(); ..., as i think we don't have reason for success == true, and for consistency with sendStreamRequest and sendStreamResponse

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's fine as 3 args, you can add undefined for success == true or just don't supply it

streamSinks[data.streamId].desiredSize = data.desiredSize;
let pullResult = streamSinks[data.streamId].onPull !== null ?
streamSinks[data.streamId].onPull() : undefined;
Promise.resolve(pullResult).then(() => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add let resolveCall = (fn, args) => { if (!fn) return Promise.resolve(undefined); return new Promise( (resolve, reject) => { resolve(fn.apply(null, args)); }); } and use it in pull, cancel and start cases.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We require to call action[0] with this = action[1] in start case here. So maybe this function shall be let resolveCall = ({ fn, _this, args}) => { if (!fn) return Promise.resolve(undefined); return new Promise( (resolve, reject) => { resolve(fn.apply(_this, args)); }); } instead.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, you make make its signature (fn, args, thisArg) and for pull/cancel call use thisArg == null.

break;
case 'error':
streamControllers[data.streamId].controller.error(data.reason);
break;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

call deleteStreamController when error

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Promise.resolve(actionResult).then(() => {
sendStreamRequest({ stream: 'start_complete', success: true });
}, () => {
sendStreamRequest({ stream: 'start_complete', success: false });
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add reason here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

let capability = createPromiseCapability();
let sendStreamRequest = ({ stream, chunk, success, reason }) => {
comObj.postMessage({ sourceName, targetName, stream, streamId,
chunk, reason, success });
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: switch 'success' and 'reason' places

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

// Delete streamController only when start, pull and
// cancel callbacks are resolved, to avoid "TypeError".
Promise.all([
streamControllers[data.streamId].startCall,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add let finalize = (promise) => { return Promise.resolve(promise).catch(() => {}); }; utility function and use it to wrap streamControllers[data.streamId].xxxCall here

Copy link
Contributor Author

@mukulmishra18 mukulmishra18 Jun 1, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think i am not getting this utility function properly, can you please explain a bit.
We are going to use this function something like:
Promise.all([finalize(streamControllers[data.streamId].startCall), finalize(streamControllers[data.streamId].pullCall), finalize(streamControllers[data.streamId].cancelCall)]).then(() => {...}) ?

this.ready = this.sinkCapability.promise;
}
sendStreamRequest({ stream: 'enqueue', chunk });
},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Adding a newline here would aid readability of the code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

close() {
sendStreamRequest({ stream: 'close' });
delete that.streamSinks[streamId];
},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Adding a newline here would aid readability of the code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

},
error(reason) {
sendStreamRequest({ stream: 'error', reason });
},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Adding a newline here would aid readability of the code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

_processStreamMessage(data) {
let sourceName = this.sourceName;
let targetName = data.sourceName;
let streamId = data.streamId;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Adding a newline here would aid readability of the code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

let sendStreamResponse = ({ stream, success, reason }) => {
this.comObj.postMessage({ sourceName, targetName, stream,
success, streamId, reason });
};
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Adding a newline here would aid readability of the code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

} else {
capability.reject(reason);
}
};
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Adding a newline here would aid readability of the code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

};
let finalize = (promise) => {
return Promise.resolve(promise).catch(() => {});
};
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Adding a newline here would aid readability of the code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

this.comObj.postMessage({ sourceName, targetName, stream,
success, streamId, reason });
};
let resolveOrReject = (capability, success, reason) => {
Copy link
Collaborator

@Snuffleupagus Snuffleupagus Jun 2, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can do function resolveOrReject(capability, success, reason) { instead here.

Edit: Please note that if you're not accessing this an arrow function isn't necessary!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, thanks for the suggestion. I will take care of that.

capability.reject(reason);
}
};
let finalize = (promise) => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can do function finalize(promise) { instead here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

onCancel: null,
desiredSize,
ready: null,
};
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Adding a newline here would aid readability of the code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

}, queueingStrategy);
},

resolveCall(fn, args, thisArg = null) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolveCall does not need to be a member of MessageHandler. Can you move it out of the class (e.g. just before MessageHandler)?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolveOrReject and finalize can go there too

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in new commit. Thanks.

Copy link
Contributor

@yurydelendik yurydelendik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good (with resolveOrReject, finalize and resolveCall before MessageHandler constructor)

@mukulmishra18 mukulmishra18 force-pushed the sendWithStream branch 2 times, most recently from fc5b48a to ce08007 Compare June 6, 2017 09:28
}
}

function finalize(promise) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless I'm missing something, you're not actually passing Promises to this function (at line https://github.com/mozilla/pdf.js/pull/8430/files#diff-8d2b578d58d4d8c8a2975020fbbc84d9R1492 and below).

Rather, it seems that what you're passing is actually promiseCapabilities, which means that this function currently doesn't really make sense to me; what am I missing here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, yes. You are right. Thanks for pointing out.

// Delete streamController only when start, pull and
// cancel callbacks are resolved, to avoid "TypeError".
Promise.all([
finalize(this.streamControllers[data.streamId].startCall),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry. let's map all capabilities to promises Promise.all([this.streamControllers[data.streamId].startCall, this.streamControllers[data.streamId].pullCall,....].map((capability) => { return capability && finalize(capability.promise); })).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in new commit. Thanks.

Adds functionality to accept Queueing Strategy in
sendWithStream method. Using Queueing Strategy we
can control the data that is enqueued into the sink,
and hence regulated the flow of chunks from worker
to main thread.

Adds capability in pull and cancel methods.
Adds ready and desiredSize property in streamSink.

Adds unit test for ReadableStream and sendWithStream.
Copy link
Collaborator

@Snuffleupagus Snuffleupagus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r=me, modulo passing tests.

@Snuffleupagus
Copy link
Collaborator

/botio test

@pdfjsbot
Copy link

pdfjsbot commented Jun 7, 2017

From: Bot.io (Windows)


Received

Command cmd_test from @Snuffleupagus received. Current queue size: 0

Live output at: http://54.215.176.217:8877/b6e3d28eb33b99f/output.txt

@pdfjsbot
Copy link

pdfjsbot commented Jun 7, 2017

From: Bot.io (Linux m4)


Received

Command cmd_test from @Snuffleupagus received. Current queue size: 0

Live output at: http://54.67.70.0:8877/be28eabda275c01/output.txt

@pdfjsbot
Copy link

pdfjsbot commented Jun 7, 2017

From: Bot.io (Linux m4)


Success

Full output at http://54.67.70.0:8877/be28eabda275c01/output.txt

Total script time: 17.14 mins

  • Font tests: Passed
  • Unit tests: Passed
  • Regression tests: Passed

@pdfjsbot
Copy link

pdfjsbot commented Jun 7, 2017

From: Bot.io (Windows)


Success

Full output at http://54.215.176.217:8877/b6e3d28eb33b99f/output.txt

Total script time: 29.94 mins

  • Font tests: Passed
  • Unit tests: Passed
  • Regression tests: Passed

@yurydelendik yurydelendik merged commit 9342054 into mozilla:master Jun 9, 2017
@yurydelendik
Copy link
Contributor

Thank you for the patch.

movsb pushed a commit to movsb/pdf.js that referenced this pull request Jul 14, 2018
Adds sendWithStream method in MessageHandler.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants