Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a unit-test to check that ProblematicCharRanges contains valid entries #7540

Merged
merged 1 commit into from
Aug 27, 2016
Merged

Add a unit-test to check that ProblematicCharRanges contains valid entries #7540

merged 1 commit into from
Aug 27, 2016

Conversation

Snuffleupagus
Copy link
Collaborator

When adding new entries to ProblematicCharRanges, you have to be careful to not make any mistakes since that could cause glyph mapping issues.
Currently the existing reference tests should probably help catch any errors, but based on experience I think that having a unit-test which specifically checks ProblematicCharRanges would be both helpful and timesaving when modifying/reviewing changes to this code.

Hence this patch which adds a function (and unit-test) that is used to validate the entries in ProblematicCharRanges, and also checks that we don't accidentally add more character ranges than the Private Use Area can actually contain.
The way that the validation code, and thus the unit-test, is implemented also means that we have an easy way to tell how much of the Private Use Area is potentially utilized by re-mapped characters.

Note: I suppose that the test function could also have been placed inside an IIFE that is only invoked in !PRODUCTION mode, but having a unit-test seemed like a good approach to me.

@Snuffleupagus
Copy link
Collaborator Author

/botio unittest

@pdfjsbot
Copy link

From: Bot.io (Windows)


Received

Command cmd_unittest from @Snuffleupagus received. Current queue size: 0

Live output at: http://107.22.172.223:8877/db3fc490f8aee16/output.txt

@pdfjsbot
Copy link

From: Bot.io (Linux)


Received

Command cmd_unittest from @Snuffleupagus received. Current queue size: 0

Live output at: http://107.21.233.14:8877/0ce4c90407ed3b0/output.txt

@pdfjsbot
Copy link

From: Bot.io (Windows)


Success

Full output at http://107.22.172.223:8877/db3fc490f8aee16/output.txt

Total script time: 2.09 mins

  • Unit Tests: Passed

@pdfjsbot
Copy link

From: Bot.io (Linux)


Failed

Full output at http://107.21.233.14:8877/0ce4c90407ed3b0/output.txt

Total script time: 2.11 mins

  • Unit Tests: FAILED

@Snuffleupagus
Copy link
Collaborator Author

That test failure shouldn't be the fault of this PR, trying again...

/botio unittest

@pdfjsbot
Copy link

From: Bot.io (Windows)


Received

Command cmd_unittest from @Snuffleupagus received. Current queue size: 0

Live output at: http://107.22.172.223:8877/c3ea71317998d82/output.txt

@pdfjsbot
Copy link

From: Bot.io (Linux)


Received

Command cmd_unittest from @Snuffleupagus received. Current queue size: 0

Live output at: http://107.21.233.14:8877/600ffc521137e12/output.txt

@pdfjsbot
Copy link

From: Bot.io (Linux)


Success

Full output at http://107.21.233.14:8877/600ffc521137e12/output.txt

Total script time: 1.72 mins

  • Unit Tests: Passed

@pdfjsbot
Copy link

From: Bot.io (Windows)


Success

Full output at http://107.22.172.223:8877/c3ea71317998d82/output.txt

Total script time: 1.88 mins

  • Unit Tests: Passed

…entries

When adding new entries to `ProblematicCharRanges`, you have to be careful to not make any mistakes since that could cause glyph mapping issues.
Currently the existing reference tests should probably help catch any errors, but based on experience I think that having a unit-test which specifically checks `ProblematicCharRanges` would be both helpful and timesaving when modifying/reviewing changes to this code.

Hence this patch which adds a function (and unit-test) that is used to validate the entries in `ProblematicCharRanges`, and also checks that we don't accidentally add more character ranges than the Private Use Area can actually contain.
The way that the validation code, and thus the unit-test, is implemented also means that we have an easy way to tell how much of the Private Use Area is potentially utilized by re-mapped characters.
@Snuffleupagus
Copy link
Collaborator Author

/botio unittest

@pdfjsbot
Copy link

From: Bot.io (Windows)


Received

Command cmd_unittest from @Snuffleupagus received. Current queue size: 0

Live output at: http://107.22.172.223:8877/6b3e3527c9fd22e/output.txt

@pdfjsbot
Copy link

From: Bot.io (Linux)


Received

Command cmd_unittest from @Snuffleupagus received. Current queue size: 0

Live output at: http://107.21.233.14:8877/9ce3782ea35d981/output.txt

@pdfjsbot
Copy link

From: Bot.io (Linux)


Success

Full output at http://107.21.233.14:8877/9ce3782ea35d981/output.txt

Total script time: 1.69 mins

  • Unit Tests: Passed

@pdfjsbot
Copy link

From: Bot.io (Windows)


Success

Full output at http://107.22.172.223:8877/6b3e3527c9fd22e/output.txt

Total script time: 2.04 mins

  • Unit Tests: Passed

@Snuffleupagus
Copy link
Collaborator Author

@timvandermeij Is this something that you'd feel comfortable reviewing?
Note that while this patch does add a function in fonts.js, it isn't used anywhere within that file (i.e. not while parsing fonts) and isn't even included in built versions of PDF.js (thanks to the !PRODUCTION preprocessor tags).

@timvandermeij
Copy link
Contributor

I'm a bit confused about why this has to live in src/core/fonts.js, since it's only for testing. Why can't we export ProblematicCharRanges and place this function in the unit test code as helper function?


describe('Fonts', function() {
it('checkProblematicCharRanges', function() {
var EXPECTED_PERCENTAGE = 45;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where does this value come from?

Copy link
Collaborator Author

@Snuffleupagus Snuffleupagus Aug 27, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It comes from the fact that we currently utilize approximately 44% of the Private Use Area (PUA).
Currently we have no way of knowing how much of the PUA that the remapped characters can potentially make use of, and by having this test fail when you add a number of new char ranges it becomes a lot clearer how the PUA utilization increases over time.

We obviously want to ensure, first of all, that we don't try to add more char ranges than the PUA can actually hold.
When you add new char ranges, you might have to increase this constant, which I feel should help both the developer and reviewer to notice that we're now using more of the PUA than before.

So by having this unit-test that will fail if you add a bunch of new char ranges, we will get advance warning if in the future we're about to run out of space in the PUA.

A bit long-winded perhaps, but I hope it makes sense! It not, just ask :-)

@timvandermeij
Copy link
Contributor

timvandermeij commented Aug 27, 2016

Aside from the comments above, this looks really good. I recently merged one of your other PRs that touched ProblematicCharRanges and I had to really pay attention to make sure the range was correct, primarily because the second code is the first _non_problematic code. Fortunately the comment above it was very clear, but it's nice to have an automated test as well.

@timvandermeij timvandermeij self-assigned this Aug 27, 2016
@Snuffleupagus
Copy link
Collaborator Author

I'm a bit confused about why this has to live in src/core/fonts.js, since it's only for testing. Why can't we export ProblematicCharRanges and place this function in the unit test code as helper function?

First of all, I prefer having all of the validation consolidated in just one simple function that the unit-test calls, since it reduces clutter in the test file.
Second of all, it wouldn't actually be enough to just export ProblematicCharRanges, we'd also need to export PRIVATE_USE_OFFSET_END and PRIVATE_USE_OFFSET_START as well.
Hence why I'd much rather just export one validation function from fonts.js, rather than a bunch of private "constants".

[...] I recently merged one of your other PRs that touched ProblematicCharRanges and I had to really pay attention to make sure the range was correct, primarily because the second code is the first nonproblematic code. Fortunately the comment above it was very clear, but it's nice to have an automated test as well.

Agreed, my intent with this patch was to simplify the life of both developers and reviewers :-)

@timvandermeij
Copy link
Contributor

/botio unittest

@pdfjsbot
Copy link

From: Bot.io (Windows)


Received

Command cmd_unittest from @timvandermeij received. Current queue size: 0

Live output at: http://107.22.172.223:8877/c845fa23fbb32be/output.txt

@pdfjsbot
Copy link

From: Bot.io (Linux)


Received

Command cmd_unittest from @timvandermeij received. Current queue size: 0

Live output at: http://107.21.233.14:8877/2b1a0cc5813dcd6/output.txt

@pdfjsbot
Copy link

From: Bot.io (Linux)


Success

Full output at http://107.21.233.14:8877/2b1a0cc5813dcd6/output.txt

Total script time: 1.71 mins

  • Unit Tests: Passed

@pdfjsbot
Copy link

From: Bot.io (Windows)


Success

Full output at http://107.22.172.223:8877/c845fa23fbb32be/output.txt

Total script time: 1.96 mins

  • Unit Tests: Passed

@timvandermeij timvandermeij merged commit d944c32 into mozilla:master Aug 27, 2016
@timvandermeij
Copy link
Contributor

Thank you!

@Snuffleupagus Snuffleupagus deleted the ProblematicCharRanges-unit-test branch August 28, 2016 09:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants