• Interpreter is shown in the status bar
• An interpreter can be manually specified using the Select Interpreter command
• Detected system-installed interpreters
• Detected an Anaconda installation
• (Linux/macOS) Detected all interpreters installed w/ pyenv detected
• "python.pythonPath" triggers an update in the status bar
• Run Python File in Terminal
• Run Selection/Line in Python Terminal
  • Right-click
  • Command
  • Shift+Enter

ALWAYS:

• Use the latest version of Anaconda

• Realize that conda is slow

• Create an environment with a space in their path somewhere as well as upper and lowercase characters

• Make sure that you do not have python.pythonPath specified in your settings.json when testing automatic detection

• Do note that the Select Interpreter drop-down window scrolls

• Detected a single virtual environment at the top-level of the workspace folder (if you created this after opening VS Code, then run Reload Window to pick up the new environment)

  • Appropriate suffix label specified in status bar (e.g. (venv))
  • Create Terminal works
    • Steals focus
    • "python.terminal.activateEnvironment": false deactivates automatically running the activation script in the terminal
  • After the language server downloads it is able to complete its analysis of the environment w/o requiring a restart

• Detect multiple virtual environments contained in the directory specified in "python.venvPath"

• Detected all conda environments created with an interpreter

  • Appropriate suffix label specified in status bar (e.g. (condaenv))
  • Prompted to install Pylint
    • Asked whether to install using conda or pip
    • Installs into environment
  • Create Terminal works
    • "python.terminal.activateEnvironment": false deactivates automatically running the activation script in the terminal
  • After the language server downloads it is able to complete its analysis of the environment w/o requiring a restart

• (Linux/macOS until -m is supported) Detected the virtual environment created by pipenv

  • Appropriate suffix label specified in status bar (e.g. (pipenv))
  • Prompt to install Pylint uses pipenv install --dev
  • Create Terminal works
    • "python.terminal.activateEnvironment": false deactivates automatically running the activation script in the terminal
  • After the language server downloads it is able to complete its analysis of the environment w/o requiring a restart

• (Linux/macOS) Virtual environments created under {workspaceFolder}/.direnv/python-{python_version} are detected (for direnv and its layout python3 support)

  • Appropriate suffix label specified in status bar (e.g. (venv))

Sample files:

# example.py
import os
print('Hello,', os.environ.get('WHO'), '!')

# .env
WHO=world
PYTHONPATH=some/path/somewhere

ALWAYS:

• Make sure to use Reload Window between tests to reset your environment

• Note that environment files only apply under the debugger and Jedi

• Environment variables in a .env file are exposed when running under the debugger

• "python.envFile" allows for specifying an environment file manually (e.g. Jedi picks up PYTHONPATH changes)

• envFile in a launch.json configuration works

• pythonPath setting in your launch.json overrides your python.pythonPath default setting

ALWAYS:

• Check under the Problems tab to see e.g. if a linter is raising errors

[Prompting to install Pylint is covered under Environments above]

For testing the disablement of the default linting rules for Pylint:

# pylintrc
[MESSAGES CONTROL]
enable=bad-names

# example.py
foo = 42  # Marked as a blacklisted name.

• Installation via the prompt installs Pylint as appropriate
  • Uses --user for system-install of Python
  • Installs into a virtual environment environment directly
• Pylint works
• "python.linting.pylintUseMinimalCheckers": false turns off the default rules w/ no pylintrc file present
• The existence of a pylintrc file turns off the default rules

Note:

• You can use the Run Linting command to run a newly installed linter

• When the extension installs a new linter, it turns off all other linters

• flake8 works

  • Select linter lists the linter and installs it if necessary

• mypy works

  • Select linter lists the linter and installs it if necessary

• pep8 works

  • Select linter lists the linter and installs it if necessary

• prospector works

  • Select linter lists the linter and installs it if necessary

• pydocstyle works

  • Select linter lists the linter and installs it if necessary

• pylama works

  • Select linter lists the linter and installs it if necessary

• 3 or more linters work simultaneously (make sure you have turned on the linters in your settings.json)

  • Run Linting runs all activated linters
  • "python.linting.enabled": false disables all linters
  • The Enable Linting command changes "python.linting.enabled"

• "python.linting.lintOnSave works

Please also test for general accuracy on the most "interesting" code you can find.

• "python.autoComplete.extraPaths" works
• "python.autocomplete.addBrackets": true causes auto-completion of functions to append ()

Sample file:

# There should be _some_ change after running `Format Document`.
def foo():pass

• Prompted to install a formatter if none installed and Format Document is run
  • Installing autopep8 works
  • Installing black works
  • Installing yapf works
• autopep8 works
• black works
• yapf works
• "editor.formatOnType": true works and has expected results

• Extract Variable works
  • You are prompted to install rope if it is not already available
• Extract method works
  • You are prompted to install rope if it is not already available
• Sort Imports works

ALWAYS:

• Test the current debugger

• Test the experimental debugger (and note whether it is at least as fast as the old debugger)

• Configurations work (see package.json and the "configurationSnippets" section for all of the possible configurations)

• Running code from start to finish w/ no special debugging options (e.g. no breakpoints)

• Breakpoint-like things

  • Breakpoint
    • Set
    • Hit
  • Conditional breakpoint
    • Expression
      • Set
      • Hit
    • Hit count
      • Set
      • Hit
  • Logpoint
    • Set
    • Hit

• Stepping

  • Over
  • Into
  • Out

• Can inspect variables

  • Through hovering over variable in code
  • Variables section of debugger sidebar

• Remote debugging works

  • ... over SSH

• App Engine

import unittest

class PassingTests(unittest.TestCase):

    def test_passing(self):
        self.assertEqual(42, 42)

    def test_passing_still(self):
        self.assertEqual("silly walk", "silly walk")


class FailingTests(unittest.TestCase):

    def test_failure(self):
        self.assertEqual(42, -13)

    def test_failure_still(self):
        self.assertEqual("I'm right!", "no, I am!")

• Run All Unit Tests triggers the prompt to configure the test runner
• Tests are discovered (as shown by code lenses on each test)
  • Code lens for a class runs all tests for that class
  • Code lens for a method runs just that test
    • Run Test works
    • Debug Test works

def test_passing():
    assert 42 == 42

def test_failure():
    assert 42 == -13

• Run All Unit Tests triggers the prompt to configure the test runner
  • pytest gets installed
• Tests are discovered (as shown by code lenses on each test)
  • Run Test works
  • Debug Test works

def test_passing():
    assert 42 == 42

def test_failure():
    assert 42 == -13

• Run All Unit Tests triggers the prompt to configure the test runner
  • Nose gets installed
• Tests are discovered (as shown by code lenses on each test)
  • Run Test works
  • Debug Test works

• Code lenses appears
  • Run Test lens works (and status bar updates as appropriate)
  • Debug Test lens works
  • Appropriate ✔/❌ shown for each test
• Status bar is functioning
  • Appropriate test results displayed
  • Run All Unit Tests works
  • Discover Unit Tests works (resets tests result display in status bar)
  • Run Unit Test Method ... works
  • View Unit Test Output works
  • After having at least one failure, Run Failed Tests works