pylint crashed with a IndexError

[ismagilli]

Bug description

I was writing code to process a file and accidentally made an error in it -- I forgot the argument of the next function. On checking such a pylint crash code. I tried to create an MRE, it turned out the following:

def data(filename):
    with open(filename) as file:
        next() # skip header

        for line in file:
            yield line

if I replace next() to next(file) then code does not crash. If i remove for loop pylynt also does not crash.

Information from pylint-crash-*.txt file:
pylint crashed with a AstroidError and with the following stacktrace:

Traceback (most recent call last):
  File "/home/ismagilov/.local/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 790, in _lint_file
    check_astroid_module(module)
  File "/home/ismagilov/.local/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 1060, in check_astroid_module
    retval = self._check_astroid_module(
  File "/home/ismagilov/.local/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 1110, in _check_astroid_module
    walker.walk(node)
  File "/home/ismagilov/.local/lib/python3.10/site-packages/pylint/utils/ast_walker.py", line 93, in walk
    self.walk(child)
  File "/home/ismagilov/.local/lib/python3.10/site-packages/pylint/utils/ast_walker.py", line 93, in walk
    self.walk(child)
  File "/home/ismagilov/.local/lib/python3.10/site-packages/pylint/utils/ast_walker.py", line 93, in walk
    self.walk(child)
  [Previous line repeated 1 more time]
  File "/home/ismagilov/.local/lib/python3.10/site-packages/pylint/utils/ast_walker.py", line 90, in walk
    callback(astroid)
  File "/home/ismagilov/.local/lib/python3.10/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 1071, in visit_call
    self._check_raising_stopiteration_in_generator_next_call(node)
  File "/home/ismagilov/.local/lib/python3.10/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 1148, in _check_raising_stopiteration_in_generator_next_call
    and not _looks_like_infinite_iterator(node.args[0])
IndexError: list index out of range

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ismagilov/.local/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 755, in _lint_files
    self._lint_file(fileitem, module, check_astroid_module)
  File "/home/ismagilov/.local/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 792, in _lint_file
    raise astroid.AstroidError from e
astroid.exceptions.AstroidError

Command used

pylint code.py

Pylint output

************* Module code
code.py:1:0: C0114: Missing module docstring (missing-module-docstring)
code.py:1:0: C0116: Missing function or method docstring (missing-function-docstring)
code.py:2:9: W1514: Using open without explicitly specifying an encoding (unspecified-encoding)
Exception on node <Call l.3 at 0x7f6220ee7670> in file '/home/ismagilov/code.py'
Traceback (most recent call last):
  File "/home/ismagilov/.local/lib/python3.10/site-packages/pylint/utils/ast_walker.py", line 90, in walk
    callback(astroid)
  File "/home/ismagilov/.local/lib/python3.10/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 1071, in visit_call
    self._check_raising_stopiteration_in_generator_next_call(node)
  File "/home/ismagilov/.local/lib/python3.10/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 1148, in _check_raising_stopiteration_in_generator_next_call
    and not _looks_like_infinite_iterator(node.args[0])
IndexError: list index out of range
code.py:1:0: F0002: code.py: Fatal error while checking 'code.py'. Please open an issue in our bug tracker so we address this. There is a pre-filled template that you can use in '/home/ismagilov/.cache/pylint/pylint-crash-2022-11-23-12-43-53.txt'. (astroid-error)

-----------------------------------
Your code has been rated at 0.00/10

Expected behavior

pylint does not crash

Pylint version

$ pylint --version
pylint 2.15.5
astroid 2.12.12
Python 3.10.8 (main, Oct 13 2022, 21:13:48) [GCC 12.2.0]

OS / Environment

GNU/Linux

[clavedeluna]

🤦‍♀️ and we thought this was fixed!

[Pierre-Sassoulas]

Could you try upgrading to pylint 2.15.6 @a-is, please ?

[clavedeluna]

Sorry I wasn't specific enough, I'm pretty sure this is legitimate. The PR that touched code around this https://github.com/PyCQA/pylint/pull/7622/files didn't specifically handle for this. I'll take a look now.

[clavedeluna]

Oh wait a minute, I'm re-reading the issue description. so we know calling next() is invalid python, but I guess pylint shouldn't crash at that 🤔
@Pierre-Sassoulas how do we handle cases when users write incorrect python code, such as in this case, do we do our best to make pylint not crash like this?

That is, should pylint handle the len(args) == 0 if next is called as next() even though that's invalid Python code?

[Pierre-Sassoulas]

Yeah we should do our best so pylint does not crash. (As an aside I really dislike when C compiler equate "undefined behavior" with "the compiler can crash and roast your computer in the process".). But we don't go out of our way to never crash either by trying to anticipate everything, if it's never reported then.. who cares. (For example the "utf13" crash was not reported for a very long time)

[clavedeluna]

Great, I'll open a PR shortly

[ismagilli]

I will be able to check the code for 2.15.6 a little later.

I want to note that calling next without arguments in some cases pylint works correctly. It seems to me that telling the user that there are errors in his code should not be so radical :)

[Pierre-Sassoulas]

I will be able to check the code for 2.15.6 a little later.

Don't worry about that I misunderstood what @clavedeluna was saying. Thank you for reporting so fast 👍