Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement streaming using moz-chunk-arraybuffer #5263

Merged
merged 2 commits into from
Sep 25, 2014

Conversation

yurydelendik
Copy link
Contributor

Implements #4739 with moz-chunk-arraybuffer. Support of the http://www.w3.org/TR/streams-api/ will be easier to add.

The PDFJS.disableStream option added to disable this functionality. Currently it autodetects presence of moz-chunk-arraybuffer feature, and if enabled turns on the streaming of the PDF (in addition to range requests). The disableAutoFetch is true when streaming in process.

@@ -903,6 +937,8 @@ PdfStreamConverter.prototype = {
aRequest.contentLength >= 0 &&
hash.indexOf('disableRange=true') < 0;
}
var streamRequest = !getBoolPref(PREF_PREFIX + '.disableStream', false) &&
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add this pref to default_preferences.js?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep

@Snuffleupagus
Copy link
Collaborator

This looks really nice, but I haven't really had time to dig into the code yet!

I've done some quick testing and found that for large files, it seems that this patch can significantly increase the time to render the first page in some cases.
Testing locally (i.e. using node make server) with the file pdf.pdf from the testsuite, it seems to take approximately 3 times longer to render the first page compared to the current master.
It appears that in this case (node make server) the entire file is downloaded before it begins to render.
Is this expected behaviour, or an issue with the implementation?

@yurydelendik
Copy link
Contributor Author

Testing locally (i.e. using node make server) with the file pdf.pdf from the testsuite, it seems to take approximately 3 times longer to render the first page compared to the current master.

pdf.pdf has issue with the named destination read -- it's too huge for us, so processing of that takes most of the time. We need to file a separate bug for that. (I think the same issue you will have with disalbeRange=true)

Another benefit I forgot to mention that, the cache will be used for the next load attempts.

@Snuffleupagus
Copy link
Collaborator

(I think the same issue you will have with disalbeRange=true)

That is a bit slower on master, but this patch is still (comparatively) a lot slower even than that case.

Another benefit I forgot to mention that, the cache will be used for the next load attempts.

That's great!

@timvandermeij
Copy link
Contributor

@yurydelendik extensions/chromium/preferences_schema.json also needs to be updated to keep it in sync with default_preferences.js (as noted at https://github.com/yurydelendik/pdf.js/blob/d2a86de8fc0dfce1b6588f70c3fa16df320f79c7/web/default_preferences.js#L21).

@yurydelendik
Copy link
Contributor Author

@yurydelendik extensions/chromium/preferences_schema.json also needs to be updated

Thanks

@yurydelendik
Copy link
Contributor Author

for performance of pdf.pdf, #5266 is filed

Math.floor(position / this.chunkSize);
var curChunk;
for (curChunk = beginChunk; curChunk < endChunk; ++curChunk) {
if (!(curChunk in this.loadedChunks)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace this line with if (!this.loadedChunks[curChunk]) {

@@ -903,6 +937,8 @@ PdfStreamConverter.prototype = {
aRequest.contentLength >= 0 &&
hash.indexOf('disableRange=true') < 0;
}
var streamRequest = !getBoolPref(PREF_PREFIX + '.disableStream', false) &&
hash.indexOf('disableStream=true') < 0;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If range requests are disabled then hash won't be defined and thus throw a TypeError here. Let's move line yurydelendik@87b1298#diff-985966130e4e9f07e74f562c737dee81R934 above the if block.

Also, this way you can move yurydelendik@87b1298#diff-985966130e4e9f07e74f562c737dee81R938 to line yurydelendik@87b1298#diff-985966130e4e9f07e74f562c737dee81R923 instead (saving a few ops when disableRange === true).

@CodingFabian
Copy link
Contributor

code looks ok to me. I wonder how often that progress event fires. can one control that? could anyone fire up a preview instance to look at?

@yurydelendik
Copy link
Contributor Author

/botio-windows preview

@pdfjsbot
Copy link

From: Bot.io (Windows)


Received

Command cmd_preview from @yurydelendik received. Current queue size: 0

Live output at: http://107.22.172.223:8877/f2a38c22acbe978/output.txt

@CodingFabian
Copy link
Contributor

In my chrome 37 on mac os, there is now a significant "lag" until the loading of the pdf starts. the dev tools show that pdf.worker.js is executing.
FF32 works very nice.

@yurydelendik
Copy link
Contributor Author

Hmmm, this PR shall not affect Chrome since moz-chunk-arraybuffer is not avialable there

@yurydelendik
Copy link
Contributor Author

@CodingFabian cache?

@CodingFabian
Copy link
Contributor

same lag on windows 7. chrome 36 and IE 11.
somewhere between 5 and 10 seconds.
does not exist on master. there the pdf starts loading instantly

@yurydelendik
Copy link
Contributor Author

@CodingFabian "the pdf" is test/pdfs/pdf.pdf ? see #5266

@CodingFabian
Copy link
Contributor

no, just the tracemonkey paper. and in network tab i do not even see it starting for a while

@yurydelendik
Copy link
Contributor Author

Looks good here

@CodingFabian
Copy link
Contributor

hmm. now its gone. could see it on two machines. maybe a server side issue/slowdown?

@yurydelendik
Copy link
Contributor Author

107.22.172.223? yes, that's a slow virtual machine

@CodingFabian
Copy link
Contributor

yeah, all good here, also with range requests disabled

brendandahl added a commit that referenced this pull request Sep 25, 2014
Implement streaming using moz-chunk-arraybuffer
@brendandahl brendandahl merged commit 9c56c6f into mozilla:master Sep 25, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants