Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix PR curve plugin to display correct thresholds #5191

Merged
merged 2 commits into from
Aug 10, 2021

Conversation

dcower
Copy link
Contributor

@dcower dcower commented Aug 4, 2021

  • Motivation for features / changes:

    • The thresholds generated and shown in the PR curve plugin are incorrect (AFAICT) compared to the thresholds used in the PR curve summary.op.
    • The thresholds used in summary.op are:
      [0.0, 1/(num_thresholds-1), 2/(num_thresholds-1), ..., 1.0]
      ... but prior to this change, the thresholds generated in _make_pr_entry and displayed in TensorBoard are:
      [1/num_thresholds, 2/num_thresholds, ..., 1.0].
  • Technical description of changes:

    • Modified _make_pr_entry to instead generate thresholds like sumamry.op, as described above.
    • Cleaned up the code in _make_pr_entry to be a bit easier to read / check for correctness.
    • Added tests for summary.op that verify the thresholds used are indeed what are described above.
    • Changed the number of thresholds used in pr_curve_demo from 50 to 51, since we're now including 0 in the thresholds.
  • Screenshots of UI changes: N/A (UI is unchanged)

  • Detailed steps to verify changes work correctly (as executed by you):

    • Ran TensorBoard with PR curve demo data and verified thresholds are displayed correctly.
    • Ran all tests in pr_curve directory.
  • Alternate designs / implementations considered: N/A

@google-cla google-cla bot added the cla: yes label Aug 4, 2021
@dcower dcower marked this pull request as ready for review August 4, 2021 19:10
@stephanwlee stephanwlee requested a review from nfelt August 6, 2021 22:35
@stephanwlee
Copy link
Contributor

Assigning to Nick who, amongst our team member, is the most knowledgeable person on PR curve. While the description sounds correct, I may not know about the historical background of the plugin.

Copy link
Contributor

@nfelt nfelt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much for this fix, and for cleaning up the code and adding tests as well! A bit embarrassing this was overlooked for so long 😅 but glad it's correct now.

@nfelt nfelt merged commit f2392db into tensorflow:master Aug 10, 2021
@dcower
Copy link
Contributor Author

dcower commented Aug 11, 2021

No problem, and thank you for the review! It took me a while to even tell anything was off, so I can definitely understand why it was overlooked 😄

yatbear pushed a commit to yatbear/tensorboard that referenced this pull request Mar 27, 2023
* Fix PR curve plugin to display correct thresholds

* Fix linter error in _make_pr_entry
dna2github pushed a commit to dna2fork/tensorboard that referenced this pull request May 1, 2023
* Fix PR curve plugin to display correct thresholds

* Fix linter error in _make_pr_entry
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants