Read test debug configuration from settings not just launch.json

[ElPincheTopo]

Issue Type: Bug

Behaviour

Expected vs. Actual

Adding this to my .vscode/launch.json allows me to use breakpoints in third-party libraries when debbuging unit tests:

{
	"name": "Debug Unit Test",
	"type": "python",
	"request": "test",
	"justMyCode": false
}

I expect that adding the same to the launch section of my workspace settings would produce the same behaviour but this is not true.

Steps to reproduce:

1. Add the above debug config to your .vscode/launch.json file.
2. Add a breakpoint to a third party tool.
3. Debug a unit test that calls the third party code and the debbuger will stop in the breakpoint.
4. Delete the .vscode/launch.json file.
5. Add the above debug config in your workspace settings.
6. Add a breakpoint to a third party tool.
7. Debug a unit test that calls the third party code and the debbuger will skip the breakpoint.

Diagnostic data

• Python version (& distribution if applicable, e.g. Anaconda): 3.10.0
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Venv
• Value of the python.languageServer setting: Default

User Settings

defaultLS: {"defaultLSType":"Pylance"}

downloadLanguageServer: true

envFile: "<placeholder>"

venvPath: "<placeholder>"

venvFolders: "<placeholder>"

condaPath: "<placeholder>"

pipenvPath: "<placeholder>"

poetryPath: "<placeholder>"

languageServer: "Pylance"

linting
• enabled: true
• cwd: "<placeholder>"
• Flake8Args: "<placeholder>"
• flake8Enabled: true
• flake8Path: "<placeholder>"
• lintOnSave: true
• banditArgs: "<placeholder>"
• banditEnabled: false
• banditPath: "<placeholder>"
• mypyArgs: "<placeholder>"
• mypyEnabled: true
• mypyPath: "<placeholder>"
• pycodestyleArgs: "<placeholder>"
• pycodestyleEnabled: false
• pycodestylePath: "<placeholder>"
• prospectorArgs: "<placeholder>"
• prospectorEnabled: false
• prospectorPath: "<placeholder>"
• pydocstyleArgs: "<placeholder>"
• pydocstyleEnabled: false
• pydocstylePath: "<placeholder>"
• pylamaArgs: "<placeholder>"
• pylamaEnabled: false
• pylamaPath: "<placeholder>"
• pylintArgs: "<placeholder>"
• pylintPath: "<placeholder>"

sortImports
• args: "<placeholder>"
• path: "<placeholder>"

formatting
• autopep8Args: "<placeholder>"
• autopep8Path: "<placeholder>"
• provider: "autopep8"
• blackArgs: "<placeholder>"
• blackPath: "<placeholder>"
• yapfArgs: "<placeholder>"
• yapfPath: "<placeholder>"

testing
• cwd: "<placeholder>"
• debugPort: 3000
• nosetestArgs: "<placeholder>"
• nosetestsEnabled: undefined
• nosetestPath: "<placeholder>"
• promptToConfigure: true
• pytestArgs: "<placeholder>"
• pytestEnabled: true
• pytestPath: "<placeholder>"
• unittestArgs: "<placeholder>"
• unittestEnabled: false
• autoTestDiscoverOnSaveEnabled: true

terminal
• activateEnvironment: true
• executeInFileDir: "<placeholder>"
• launchArgs: "<placeholder>"

experiments
• enabled: true
• optInto: []
• optOutFrom: []

insidersChannel: "off"

tensorBoard
• logDirectory: "<placeholder>"

Extension version: 2022.2.1924087327
VS Code version: Code 1.65.2 (c722ca6c7eed3d7987c0d5c3df5c45f6b15e77d1, 2022-03-10T15:36:26.048Z)
OS version: Linux x64 4.14.240
Restricted Mode: No

[luabud]

Hi @ElPincheTopo, the launch.json configuration has changed for debugging tests (see https://code.visualstudio.com/docs/python/testing#_debug-tests). Can you try the configuration below?

 "launch": {
        "configurations": [ {
            "name": "Python: Debug Tests",
            "type": "python",
            "request": "launch",
            "program": "${file}",
            "purpose": ["debug-test"],
            "console": "integratedTerminal",
            "justMyCode": false
        }
        ],
        "compounds": []
    }

And just out of pure curiosity, what is the reason behind using it on your workspace configuration rather than launch.json?

[karthiknadig]

We currently only read this from the launch.json. We could expand the scope of where we read it from, but that has some things that we have not decided yet. Like how we handle multiple test configurations, which one takes priority, etc.

[ElPincheTopo]

Thanks for the hint @luabud. It's good to know that that has changed, but it still behaves on the same way as my original example.
My original description might have not been super clear. I'm storing my workspace settings in a PROJECT.code-workspace file outside of the repository. I use this in some projects to be able to have multiple folders in the same workspace, and if I'm already putting my settings in there it makes sense to put all the settings in there unless I need to override something for one of the folders in the project.

@karthiknadig I think that test configurations defined in the Folder settings (.vscode/launch.json) should take precedence, then the ones on the Workspace settings (PROJECT.code-workspace) and then the ones in the User settings. I think that this would be consistent with the way all the other settings get applied in vscode, which is why I expected this to work.

Currently, if you have more than one test configuration defined in your launch.json the first one gets used and the rest are ignored, so making this change would preserve the existing behavior but expand the places the config can be read from to make it consistent with other settings in vscode.

[MetRonnie]

It doesn't help that the command Open 'launch.json' in the command palette just opens settings.json if you haven't got a launch.json. That makes you think there would be no difference between where you add a configuration

[coretl]

I have the same workflow, vscode settings in a workspace settings rather than in the repository, so I'd love to see a solution to this. I found this issue after reading microsoft/vscode-python#693 and the last comment in that thread seems to be about this issue, so commenting to link the two.

[eleanorjboyd]

The above fix is out on the pre-release version on the debugger extension and will be released to stable next week. If anyone is able to try it in the meantime on pre-release and let me know if it fixes your issue that would be great! Thanks

[coretl]

Since my original comment I've now moved on to using devcontainers, so I tried testing that with the prerelease debugpy. With the following in my devcontainer.json:

    "customizations": {
        "vscode": {
            "settings": {
                "python.defaultInterpreterPath": "/venv/bin/python",
                "python.testing.unittestEnabled": false,
                "python.testing.pytestEnabled": true,
            },
            "launch": {
                "version": "0.2.0",
                "configurations": [
                    {
                        "name": "Debug Unit Test",
                        "type": "debugpy",
                        "request": "launch",
                        "justMyCode": false,
                        "program": "${file}",
                        "purpose": [
                            "debug-test"
                        ],
                        "console": "integratedTerminal",
                    }
                ]
            }
        }
    },

it doesn't seem to be allowing me to step into library code, so it doesn't appear to work for me.

Putting the launch section of that file into launch.json works correctly.

Would you expect to be able to read launch config from devcontainer.json? If not, should I raise a ticket to request it?

[eleanorjboyd]

hm great question- I will have to investigate, thank you for pointing that out!