Estimate the size of decoded streams in advance. #4444
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| @@ -800,6 +808,7 @@ var JpegStream = (function JpegStreamClosure() { | |||
| var jpegImage = new JpegImage(); | |||
| if (this.colorTransform != -1) | |||
| jpegImage.colorTransform = this.colorTransform; | |||
| dump('x1\n'); | |||
There was a problem hiding this comment.
It looks like this is a leftover from debugging.
When decoding a stream, the decode buffer is often grown multiple times, its byte size increasing like so: 512, 1024, 2048, etc. This patch estimates the minimum size in advance (using the length of the encoded stream), often allowing the smaller sizes to be skipped. It also renames numerous |length| variables as |maybeLength| to make it clear that they can be |null|. I measured this change on eight documents. This change reduces the cumulative size of decode buffer allocations by 0--32%, with 10--20% being typical. This reduces peak RSS by 10 or 20 MiB for several of them.
|
I removed the stray dump() call. |
|
/botio test |
From: Bot.io (Linux)ReceivedCommand cmd_test from @Snuffleupagus received. Current queue size: 0 Live output at: http://107.21.233.14:8877/5bd5b07dfad125b/output.txt |
From: Bot.io (Windows)ReceivedCommand cmd_test from @Snuffleupagus received. Current queue size: 0 Live output at: http://107.22.172.223:8877/47f982c4bbc8866/output.txt |
From: Bot.io (Linux)SuccessFull output at http://107.21.233.14:8877/5bd5b07dfad125b/output.txt Total script time: 26.26 mins
|
From: Bot.io (Windows)FailedFull output at http://107.22.172.223:8877/47f982c4bbc8866/output.txt Total script time: 36.95 mins
Image differences available at: http://107.22.172.223:8877/47f982c4bbc8866/reftest-analyzer.html#web=eq.log |
yurydelendik
added a commit
that referenced
this pull request
Mar 13, 2014
Estimate the size of decoded streams in advance.
|
Looks good. Thanks |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When decoding a stream, the decode buffer is often grown multiple times, its
byte size increasing like so: 512, 1024, 2048, etc. This patch estimates the
minimum size in advance (using the length of the encoded stream), often
allowing the smaller sizes to be skipped. It also renames numerous |length|
variables as |maybeLength| to make it clear that they can be |null|.
I measured this change on eight documents. This change reduces the cumulative
size of decode buffer allocations by 0--32%, with 10--20% being typical. This
reduces peak RSS by 10 or 20 MiB for several of them.
BTW, I think Ascii85 and AsciiHex streams are the only ones for which the decoded data is regularly smaller than the encoded data, and I've accounted for that. Please let me know if any of the other stream kinds have the same behaviour.