Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancement #3413 consider-using-with #4372

Merged

Conversation

DudeNr33
Copy link
Collaborator

Steps

  • Add a ChangeLog entry describing what your PR does.
  • If it's a new feature or an important bug fix, add a What's New entry in
    doc/whatsnew/<current release.rst>.
  • Write a good description on what the PR does.

Description

This PR implements a new refactoring message consider-using-with inside the existing RefactoringChecker class.
This message is triggered if certain constructs may be replaced with a with block to ensure the correct release of allocated resources.
The following cases are covered (I took this blog post as a reference):

Resource-allocating assignments

  • Opening files, basically everything inheriting from io.IOBase (open(), codecs.open(), etc.)
  • subprocess.Popen
  • multiprocessing.context.BaseContext.Pool
  • thread.ThreadPoolExecutor and process.ProcessPoolExecutor from concurrent.futures

Methods that are also called when using the object as context manager

  • the different lock-acquisition methods of the threading module, e.g. threading.Lock.acquire()
  • BaseManager.start() and SyncManager.start() from multiprocessing.managers

The current implementation works by checking the assignments/calls against predefined sets of callables of the Python standard library.

After adding this new check some files in the codebase were flagged when running Pylint on itself.
These files have been fixed as well, or the message disabled for the offending lines if transforming into a with statement was not possible.

Trying to dynamically determine if an assignment or call could be replaced by a with block seems possible for easy cases (e.g. if __enter__ returns just self or calls the same method the currently checked call does), but I think that could become either very incomplete (missing more complex cases) or error prone (reporting false positives).
However, it could then also check code outside standard library.
Maybe this could be implemented as an optional checker with a separate PR.

Type of Changes

Type
✨ New feature
🔨 Refactoring

Related Issue

Closes #3413

@DudeNr33 DudeNr33 changed the title Enhancement #3414 consider-using-with Enhancement #3413 consider-using-with Apr 17, 2021
@coveralls
Copy link

coveralls commented Apr 17, 2021

Coverage Status

Coverage increased (+0.008%) to 91.59% when pulling c4f6010 on DudeNr33:enhancement-3414-consider-using-with into 95c05e0 on PyCQA:master.

@DudeNr33
Copy link
Collaborator Author

Hm, it seems that the ordinary open() call is uninferrable when running under pypy.

@DudeNr33 DudeNr33 marked this pull request as ready for review April 18, 2021 13:30
Copy link
Member

@Pierre-Sassoulas Pierre-Sassoulas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, great new checker and a lot of work done !

@Pierre-Sassoulas Pierre-Sassoulas merged commit 922f389 into pylint-dev:master Apr 23, 2021
@DudeNr33 DudeNr33 deleted the enhancement-3414-consider-using-with branch April 23, 2021 19:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Detect open() calls without with operator
3 participants