Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[primer] stop overwriting --rcfile #6897

Merged

Conversation

jacobtylerwalls
Copy link
Member

@jacobtylerwalls jacobtylerwalls commented Jun 8, 2022

Type of Changes

Type
🔨 Refactoring

Description

Move everything into the new primer and increase the timeout on the main job. (3.7 is about 2x as slow as 3.10)
Follow up to #6761 and #6753.

Never mind, we will have to discuss that separately.

@jacobtylerwalls jacobtylerwalls marked this pull request as draft June 8, 2022 15:23
@coveralls
Copy link

coveralls commented Jun 8, 2022

Pull Request Test Coverage Report for Build 2470403195

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage remained the same at 95.525%

Totals Coverage Status
Change from base Build 2466932541: 0.0%
Covered Lines: 16352
Relevant Lines: 17118

💛 - Coveralls

@jacobtylerwalls jacobtylerwalls marked this pull request as ready for review June 8, 2022 17:36
@Pierre-Sassoulas
Copy link
Member

The error of the CI is strange, it seems there's a mix up between django and black

@DanielNoord
Copy link
Collaborator

@jacobtylerwalls Did you test this on your local branch to see the time differences?

@jacobtylerwalls jacobtylerwalls force-pushed the add-more-packages-to-primer branch from 058fe87 to 1c51047 Compare June 9, 2022 12:52
@jacobtylerwalls
Copy link
Member Author

Good point, I hadn't. I just did. I removed keras to try to get the job under 45m. With just django and music21, added, the 3.10 job finishes in 30 minutes. 3.7 seems to take 1.5x as long, which just barely exceeds 45m (hard to tell, but that's my guess.)

Neither of the two projects I'm adding support 3.7 anymore. I'm wondering if we even need that job?

@jacobtylerwalls jacobtylerwalls changed the title Restore home-assistant to old primer and move others to new primer Move all packages to new primer and increase timeout for main job Jun 9, 2022
@jacobtylerwalls jacobtylerwalls force-pushed the add-more-packages-to-primer branch from f62d06e to 5b6f229 Compare June 9, 2022 18:28
@jacobtylerwalls jacobtylerwalls changed the title Move all packages to new primer and increase timeout for main job [primer] stop overwriting --rcfile Jun 9, 2022
@github-actions
Copy link
Contributor

github-actions bot commented Jun 9, 2022

🤖 Effect of this PR on checked open source code: 🤖

Effect on astroid:
The following messages are now emitted:

  1. too-many-branches:
    Too many branches (32/25)
    https://github.com/PyCQA/astroid/blob/main/astroid/filter_statements.py#L42
  2. too-many-statements:
    Too many statements (83/60)
    https://github.com/PyCQA/astroid/blob/main/astroid/filter_statements.py#L42
  3. too-many-public-methods:
    Too many public methods (80/20)
    https://github.com/PyCQA/astroid/blob/main/astroid/rebuilder.py#L56
  4. too-many-branches:
    Too many branches (27/25)
    https://github.com/PyCQA/astroid/blob/main/astroid/protocols.py#L655
  5. suppressed-message:
    Suppressed 'too-many-statements' (from line 670)
    https://github.com/PyCQA/astroid/blob/main/astroid/protocols.py#L655
  6. invalid-name:
    Method name "get_return_nodes_skip_functions" doesn't conform to '[a-z][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/mixins.py#L138
  7. too-many-branches:
    Too many branches (27/25)
    https://github.com/PyCQA/astroid/blob/main/astroid/brain/brain_gi.py#L57
  8. too-many-statements:
    Too many statements (62/60)
    https://github.com/PyCQA/astroid/blob/main/astroid/brain/brain_gi.py#L57
  9. suppressed-message:
    Suppressed 'no-member' (from line 336)
    https://github.com/PyCQA/astroid/blob/main/astroid/interpreter/objectmodel.py#L337
  10. invalid-name:
    Method name "get_return_nodes_skip_functions" doesn't conform to '[a-z][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/node_ng.py#L587
  11. invalid-name:
    Method name "eq" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/node_ng.py#L646
  12. too-many-statements:
    Too many statements (65/60)
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/node_ng.py#L653
  13. too-many-public-methods:
    Too many public methods (29/20)
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/node_ng.py#L59
  14. too-many-public-methods:
    Too many public methods (83/20)
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/as_string.py#L34
  15. too-many-lines:
    Too many lines in module (5443/3000)
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/node_classes.py#L1
  16. too-many-lines:
    Too many lines in module (3098/3000)
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/scoped_nodes/scoped_nodes.py#L1
  17. too-many-public-methods:
    Too many public methods (21/20)
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/scoped_nodes/scoped_nodes.py#L177
  18. too-many-public-methods:
    Too many public methods (36/20)
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/scoped_nodes/scoped_nodes.py#L1918
  19. suppressed-message:
    Suppressed 'no-member' (from line 1193)
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/scoped_nodes/scoped_nodes.py#L1196

The following messages are no longer emitted:

  1. too-many-branches:
    Too many branches (32/27)
    https://github.com/PyCQA/astroid/blob/main/astroid/filter_statements.py#L42
  2. too-many-lines:
    Too many lines in module (2084/2000)
    https://github.com/PyCQA/astroid/blob/main/astroid/rebuilder.py#L1
  3. too-many-public-methods:
    Too many public methods (80/25)
    https://github.com/PyCQA/astroid/blob/main/astroid/rebuilder.py#L56
  4. bad-builtin:
    Used builtin function 'map'. Using a list comprehension can be clearer.
    https://github.com/PyCQA/astroid/blob/main/astroid/modutils.py#L287
  5. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/bases.py#L177
  6. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/bases.py#L467
  7. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/bases.py#L479
  8. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/bases.py#L491
  9. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/bases.py#L498
  10. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/bases.py#L507
  11. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/bases.py#L516
  12. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/bases.py#L520
  13. invalid-name:
    Attribute name "op" doesn't conform to '[a-z_][a-z0-9_]{2,}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/util.py#L68
  14. invalid-name:
    Argument name "op" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/util.py#L66
  15. invalid-name:
    Attribute name "op" doesn't conform to '[a-z_][a-z0-9_]{2,}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/util.py#L104
  16. invalid-name:
    Argument name "op" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/util.py#L101
  17. invalid-name:
    Argument name "op" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/inference.py#L565
  18. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/inference.py#L573
  19. invalid-name:
    Argument name "op" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/inference.py#L580
  20. invalid-name:
    Argument name "op" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/inference.py#L594
  21. invalid-name:
    Variable name "op" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/inference.py#L652
  22. bad-builtin:
    Used builtin function 'map'. Using a list comprehension can be clearer.
    https://github.com/PyCQA/astroid/blob/main/astroid/inference.py#L743
  23. invalid-name:
    Argument name "op" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/inference.py#L819
  24. invalid-name:
    Variable name "op" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/inference.py#L872
  25. invalid-name:
    Argument name "op" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/protocols.py#L79
  26. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/protocols.py#L519
  27. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/protocols.py#L539
  28. useless-suppression:
    Useless suppression of 'too-many-statements'
    https://github.com/PyCQA/astroid/blob/main/astroid/protocols.py#L670
  29. too-many-ancestors:
    Too many ancestors (8/7)
    https://github.com/PyCQA/astroid/blob/main/astroid/objects.py#L261
  30. too-many-ancestors:
    Too many ancestors (8/7)
    https://github.com/PyCQA/astroid/blob/main/astroid/objects.py#L312
  31. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/manager.py#L177
  32. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/manager.py#L209
  33. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/manager.py#L255
  34. bad-builtin:
    Used builtin function 'map'. Using a list comprehension can be clearer.
    https://github.com/PyCQA/astroid/blob/main/astroid/helpers.py#L182
  35. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/helpers.py#L189
  36. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/helpers.py#L279
  37. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/helpers.py#L281
  38. no-member:
    Instance of 'FilterStmtsMixin' has no 'statement' member
    https://github.com/PyCQA/astroid/blob/main/astroid/mixins.py#L51
  39. no-member:
    Instance of 'AssignTypeMixin' has no 'statement' member
    https://github.com/PyCQA/astroid/blob/main/astroid/mixins.py#L71
  40. no-member:
    Instance of 'ParentAssignTypeMixin' has no 'parent' member
    https://github.com/PyCQA/astroid/blob/main/astroid/mixins.py#L80
  41. no-member:
    Instance of 'ImportFromMixin' has no 'root' member
    https://github.com/PyCQA/astroid/blob/main/astroid/mixins.py#L95
  42. no-member:
    Instance of 'ImportFromMixin' has no 'modname' member
    https://github.com/PyCQA/astroid/blob/main/astroid/mixins.py#L98
  43. no-member:
    Instance of 'ImportFromMixin' has no 'names' member
    https://github.com/PyCQA/astroid/blob/main/astroid/mixins.py#L111
  44. no-member:
    Instance of 'MultiLineBlockMixin' has no '_multi_line_block_fields' member
    https://github.com/PyCQA/astroid/blob/main/astroid/mixins.py#L136
  45. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/test_utils.py#L28
  46. invalid-name:
    Argument name "f" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/test_utils.py#L35
  47. invalid-name:
    Variable name "m" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/test_utils.py#L70
  48. suppressed-message:
    Suppressed 'invalid-name' (from line 266)
    https://github.com/PyCQA/astroid/blob/main/astroid/brain/brain_builtin_inference.py#L750
  49. suppressed-message:
    Suppressed 'invalid-name' (from line 266)
    https://github.com/PyCQA/astroid/blob/main/astroid/brain/brain_builtin_inference.py#L741
  50. suppressed-message:
    Suppressed 'invalid-name' (from line 266)
    https://github.com/PyCQA/astroid/blob/main/astroid/brain/brain_builtin_inference.py#L311
  51. invalid-name:
    Variable name "m" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/brain/brain_gi.py#L177
  52. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/brain/brain_namedtuple_enum.py#L194
  53. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/brain/brain_namedtuple_enum.py#L474
  54. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/brain/brain_typing.py#L279
  55. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/brain/brain_typing.py#L360
  56. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/interpreter/objectmodel.py#L314
  57. too-many-ancestors:
    Too many ancestors (8/7)
    https://github.com/PyCQA/astroid/blob/main/astroid/interpreter/objectmodel.py#L792
  58. too-many-ancestors:
    Too many ancestors (8/7)
    https://github.com/PyCQA/astroid/blob/main/astroid/interpreter/objectmodel.py#L835
  59. useless-suppression:
    Useless suppression of 'no-member'
    https://github.com/PyCQA/astroid/blob/main/astroid/interpreter/objectmodel.py#L336
  60. invalid-name:
    Method name "eq" doesn't conform to '[a-z_][a-z0-9_]{2,}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/node_ng.py#L646
  61. too-many-public-methods:
    Too many public methods (29/25)
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/node_ng.py#L59
  62. too-many-public-methods:
    Too many public methods (83/25)
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/as_string.py#L34
  63. too-many-lines:
    Too many lines in module (5443/2000)
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/node_classes.py#L1
  64. too-many-instance-attributes:
    Too many instance attributes (15/11)
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/node_classes.py#L576
  65. invalid-name:
    Attribute name "op" doesn't conform to '[a-z_][a-z0-9_]{2,}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/node_classes.py#L1340
  66. invalid-name:
    Attribute name "op" doesn't conform to '[a-z_][a-z0-9_]{2,}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/node_classes.py#L1450
  67. invalid-name:
    Attribute name "op" doesn't conform to '[a-z_][a-z0-9_]{2,}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/node_classes.py#L1549
  68. invalid-name:
    Attribute name "op" doesn't conform to '[a-z_][a-z0-9_]{2,}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/node_classes.py#L4227
  69. too-many-lines:
    Too many lines in module (3098/2000)
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/scoped_nodes/scoped_nodes.py#L1
  70. bad-builtin:
    Used builtin function 'map'. Using a list comprehension can be clearer.
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/scoped_nodes/scoped_nodes.py#L151
  71. invalid-name:
    Argument name "n" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/scoped_nodes/scoped_nodes.py#L166
  72. too-many-instance-attributes:
    Too many instance attributes (12/11)
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/scoped_nodes/scoped_nodes.py#L177
  73. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/scoped_nodes/scoped_nodes.py#L1721
  74. too-many-ancestors:
    Too many ancestors (8/7)
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/scoped_nodes/scoped_nodes.py#L1799
  75. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/scoped_nodes/scoped_nodes.py#L2244
  76. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/scoped_nodes/scoped_nodes.py#L2258
  77. invalid-name:
    Variable name "e" doesn't conform to '[a-z_][a-z0-9_]{2,30}$' pattern
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/scoped_nodes/scoped_nodes.py#L2981
  78. too-many-public-methods:
    Too many public methods (36/25)
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/scoped_nodes/scoped_nodes.py#L1918
  79. useless-suppression:
    Useless suppression of 'no-member'
    https://github.com/PyCQA/astroid/blob/main/astroid/nodes/scoped_nodes/scoped_nodes.py#L1193

This comment was generated for commit 5b6f229

@jacobtylerwalls
Copy link
Member Author

jacobtylerwalls commented Jun 9, 2022

The changes in astroid are to be expected because we're using the correct --rcfile now.

I plan to merge this tonight based on Pierre's approval earlier of the cleanups.

I'll move discussion about adding packages to #5359, since I think it's clear we need to look at parallelization instead of just adding packages to a single run. The 3.10 (the faster one!) errored out on CI (on my fork, with adding all packages) with an out of memory error, probably trying to deal with the massive list of messages. The 3.7 run would have no hope.

@jacobtylerwalls jacobtylerwalls merged commit d9233fe into pylint-dev:main Jun 9, 2022
@jacobtylerwalls jacobtylerwalls deleted the add-more-packages-to-primer branch June 9, 2022 22:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants