Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

statistics: add metrics for unneeded analyze table #54822

Merged
merged 4 commits into from
Jul 26, 2024
Merged

statistics: add metrics for unneeded analyze table #54822

merged 4 commits into from
Jul 26, 2024

Conversation

hawkingrei
Copy link
Member

@hawkingrei hawkingrei commented Jul 23, 2024 •
edited
Loading

What problem does this PR solve?

Issue Number: close #54823

Problem Summary:

What changed and how does it work?

The primary changes in pkg/statistics/handle/cache/statscache.go file.
the rest are code migrations to avoid cyclic imports.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)

I test in the 1m tables cluster. we can see this metrics in the grafana.

image
  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot ti-chi-bot bot added do-not-merge/needs-linked-issue do-not-merge/needs-tests-checked release-note-none Denotes a PR that doesn't merit a release note. sig/planner SIG: Planner size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 23, 2024
@codecov Codecov
Copy link

codecov bot commented Jul 23, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 90.00000% with 2 lines in your changes missing coverage. Please review.

Project coverage is 56.9952%. Comparing base (f774ef6) to head (2a88ef3).
Report is 38 commits behind head on master.

Additional details and impacted files 
@@                Coverage Diff                @@
##             master     #54822         +/-   ##
=================================================
- Coverage   74.7207%   56.9952%   -17.7256%     
=================================================
  Files          1557       1684        +127     
  Lines        363500     631235     +267735     
=================================================
+ Hits         271610     359774      +88164     
- Misses        72240     247422     +175182     
- Partials      19650      24039       +4389
Flag Coverage Δ
integration 37.6885% <85.0000%> (?)
unit 72.1299% <90.0000%> (-1.5151%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 53.3749% <ø> (-1.8246%) ⬇️
parser ∅ <ø> (∅)
br 51.9165% <ø> (+4.2288%) ⬆️

@hawkingrei hawkingrei force-pushed the add_health_for_uneligibleAnalyze branch from 69d932a to aa6287a Compare July 23, 2024 05:41
@hawkingrei
Copy link
Member Author

/retest

@hawkingrei hawkingrei requested a review from winoros July 23, 2024 09:55
@hawkingrei
update
0907f8a 
Signed-off-by: Weizhen Wang <[email protected]>
@hawkingrei
Copy link
Member Author

/retest

1 similar comment
@hawkingrei
Copy link
Member Author

/retest

@ti-chi-bot ti-chi-bot bot added approved needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jul 24, 2024
@hawkingrei hawkingrei requested a review from Rustin170506 July 25, 2024 03:55
Rustin170506
Rustin170506 reviewed Jul 25, 2024
View reviewed changes
Copy link
Member

@Rustin170506 Rustin170506 left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to update the grafana json file?

@hawkingrei @Rustin170506
Update pkg/statistics/table.go
2a88ef3 
Co-authored-by: 二手掉包工程师 <[email protected]>
@hawkingrei
Copy link
Member Author

hawkingrei commented Jul 25, 2024 •
edited
Loading

Do we need to update the grafana json file?

No, because this type is generated dynamically by the grafana.

@ti-chi-bot ti-chi-bot
Copy link

ti-chi-bot bot commented Jul 25, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hi-rustin, winoros

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jul 25, 2024
@ti-chi-bot ti-chi-bot
Copy link

ti-chi-bot bot commented Jul 25, 2024

[LGTM Timeline notifier]

Timeline:

  • 2024-07-24 12:41:39.274428107 +0000 UTC m=+1048921.265369577: ☑️ agreed by winoros.
  • 2024-07-25 07:53:23.497835995 +0000 UTC m=+1118025.488777466: ☑️ agreed by hi-rustin.

@hawkingrei
Copy link
Member Author

/retest

1 similar comment
@hawkingrei
Copy link
Member Author

/retest

@ti-chi-bot ti-chi-bot bot merged commit 7e73ddc into pingcap:master Jul 26, 2024
23 checks passed
morgo added a commit to morgo/tidb that referenced this pull request Jul 28, 2024
@morgo
Merge remote-tracking branch 'upstream/master' into add-default-literal
4424521 
* upstream/master: (93 commits)
  disjoinset: add generic impl (pingcap#54917)
  planner: derive index filters for mv index paths (pingcap#54877)
  br: cli refactor backup error handling logic (pingcap#54697)
  expression: fix infinity loop in `timestampadd` (pingcap#54916)
  planner: import more expand test. (pingcap#54962)
  planner: use code-gen to generate CloneForPlanCache method for some operators (pingcap#54957)
  test: fix flaky test TestFailSchemaSyncer (pingcap#54958)
  planner: move logical show into logicalop pkg. (pingcap#54928)
  privilege: Remove TestAbnormalMySQLTable (pingcap#54925)
  resource_control: support unlimited keyword when setting the resource group (pingcap#54704)
  ddl: fix a data race on localRowCntListener Written() (pingcap#54484)
  lightning: fix SET SESSION on connection pool (pingcap#54927)
  planner: move logical mem-table to logicalop pkg. (pingcap#54903)
  table: Add `CachedTableSupport` and `TemporaryTableSupport` for `MutateContext` (pingcap#54900)
  executor: fix index_hash_join hang when context canceled (pingcap#54855)
  statistics: add metrics for unneeded analyze table (pingcap#54822)
  *: refine pipelined dml benchmarks (pingcap#54844)
  ddl: assign table IDs for jobs submitted to queue (pingcap#54880)
  *: avoid using Tables field of model.DBInfo, use API instead (pingcap#52302)
  planner: remove useless check (pingcap#54907)
  ...
@hawkingrei
Copy link
Member Author

/cherrypick release-8.1
/cherrypick release-7.5

@ti-chi-bot
Copy link
Member

@hawkingrei: new pull request created to branch release-7.5: #55577.

In response to this:

/cherrypick release-8.1
/cherrypick release-7.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Aug 22, 2024
@hawkingrei @ti-chi-bot
This is an automated cherry-pick of pingcap#54822
2848d33 
Signed-off-by: ti-chi-bot <[email protected]>
@ti-chi-bot ti-chi-bot mentioned this pull request Aug 22, 2024
Merged
13 tasks
@ti-chi-bot
Copy link
Member

@hawkingrei: new pull request created to branch release-8.1: #55578.

In response to this:

/cherrypick release-8.1
/cherrypick release-7.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Aug 22, 2024
@hawkingrei @ti-chi-bot
This is an automated cherry-pick of pingcap#54822
2f8e42c 
Signed-off-by: ti-chi-bot <[email protected]>
@ti-chi-bot ti-chi-bot mentioned this pull request Aug 22, 2024
Closed
13 tasks
@hawkingrei hawkingrei deleted the add_health_for_uneligibleAnalyze branch August 22, 2024 03:36
ti-chi-bot bot pushed a commit that referenced this pull request Aug 25, 2024
@ti-chi-bot
statistics: add metrics for unneeded analyze table (#54822) (#55577)
47279f5 
close #54823
@hawkingrei
Copy link
Member Author

/cherrypick release-8.1

@ti-chi-bot
Copy link
Member

@hawkingrei: new pull request created to branch release-8.1: #57100.

In response to this:

/cherrypick release-8.1

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Nov 4, 2024
@hawkingrei @ti-chi-bot
This is an automated cherry-pick of pingcap#54822
ab6e036 
Signed-off-by: ti-chi-bot <[email protected]>
@ti-chi-bot ti-chi-bot mentioned this pull request Nov 4, 2024
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this pull request Nov 4, 2024
@ti-chi-bot
statistics: add metrics for unneeded analyze table (#54822) (#57100)
152b102 
close #54823
@hawkingrei
Copy link
Member Author

/cherry-pick release-8.1-20241203-v8.1.1

@ti-chi-bot
Copy link
Member

@hawkingrei: new pull request created to branch release-8.1-20241203-v8.1.1: #57966.

In response to this:

/cherry-pick release-8.1-20241203-v8.1.1

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Dec 4, 2024
@hawkingrei @ti-chi-bot
This is an automated cherry-pick of pingcap#54822
3e9752d 
Signed-off-by: ti-chi-bot <[email protected]>
@ti-chi-bot ti-chi-bot mentioned this pull request Dec 4, 2024
Merged
13 tasks
hawkingrei pushed a commit to ti-chi-bot/tidb that referenced this pull request Dec 4, 2024
@ti-chi-bot @hawkingrei
statistics: add metrics for unneeded analyze table (pingcap#54822) (p…
430072a 
…ingcap#57100)

close pingcap#54823
ti-chi-bot bot pushed a commit that referenced this pull request Dec 4, 2024
@ti-chi-bot
statistics: add metrics for unneeded analyze table (#54822) (#57966)
238b0d8 
close #54823
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@winoros winoros winoros approved these changes

@Rustin170506 Rustin170506 Rustin170506 approved these changes

Assignees
No one assigned
Labels
approved lgtm release-note-none Denotes a PR that doesn't merit a release note. sig/planner SIG: Planner size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

add metrics for unneeded analyze
4 participants
@hawkingrei @ti-chi-bot @winoros @Rustin170506