Skip to content

Test execution randomly ends prematurely #17282

Closed
#17313
Closed
Test execution randomly ends prematurely#17282
#17313
Assignees
karthiknadig
Labels
area-testingbugIssue identified by VS Code Team member as probable bug
Milestone
August Recovery
@pewscorner

Description

@pewscorner
pewscorner
opened on Sep 4, 2021

Environment data

  • VS Code version: 1.60.0
  • Extension version (available under the Extensions sidebar): v2021.9.1191016588
  • OS and version: Windows 10
  • Python version (& distribution if applicable, e.g. Anaconda): 3.9.7
  • Type of virtual environment used (N/A | venv | virtualenv | conda | ...): N/A
  • Relevant/affected Python packages and their versions: N/A
  • Relevant/affected Python-related VS Code extensions and their versions: Pylance v2021.9.0
  • Value of the python.languageServer setting: Pylance

Expected behaviour

When clicking on the "Run Tests" button, all discovered tests should run.

Actual behaviour

A random number of tests are run (sometimes all of them). There is nothing in the UI indicating why the testing stopped (but there's an error in the Python output log, see below). This behavior started after upgrading VS Code to 1.60.0.

Steps to reproduce:

[NOTE: Self-contained, minimal reproducing code samples are extremely helpful and will expedite addressing your issue]

I simply run my 64-test test suite.

Logs

Details

Test output terminal:

Running tests (unittest): d:\Documents\PythonProjects\PythonQuickRefTest\unittests
Running tests: d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py::TestClasses::test_about_classes_types_and_objects
./classes_unittest.py::TestClasses::test_about_classes_types_and_objects Passed
Running tests: d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py::TestClasses::test_class_creation_and_instantiation
./classes_unittest.py::TestClasses::test_class_creation_and_instantiation Passed
Running tests: d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py::TestClasses::test_class_inheritance
./classes_unittest.py::TestClasses::test_class_inheritance Passed
Running tests: d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py::TestClasses::test_descriptors
./classes_unittest.py::TestClasses::test_descriptors Passed
Running tests: d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py::TestClasses::test_property_attributes
Total number of tests expected to run: 64
Total number of tests run: 64
Total number of tests passed: 4
Total number of tests failed: 0
Total number of tests failed with errors: 0
Total number of tests skipped: 0
Finished running tests!

Python output

User belongs to experiment group 'pythonaa'
User belongs to experiment group 'pythonJediLSP'
User belongs to experiment group 'pythonSendEntireLineToREPL'
User belongs to experiment group 'pythonNotDisplayLinterPrompt'
User belongs to experiment group 'pythonDiscoveryModuleWithoutWatcher'
User belongs to experiment group 'pythonTensorboardExperiment'
User belongs to experiment group 'PythonPyTorchProfiler'
Python interpreter path: D:\Python\Python39\python.exe
> pyenv root
> conda --version
> python3.7 -c "import sys;print(sys.executable)"
> python3.6 -c "import sys;print(sys.executable)"
> python3 -c "import sys;print(sys.executable)"
> python2 -c "import sys;print(sys.executable)"
> python -c "import sys;print(sys.executable)"
> py -3.7 -c "import sys;print(sys.executable)"
> py -3.6 -c "import sys;print(sys.executable)"
> py -3 -c "import sys;print(sys.executable)"
> py -2 -c "import sys;print(sys.executable)"
> D:\Python\Python39\python.exe -c "import sys;print(sys.executable)"
> conda info --json
Error 2021-09-04 19:21:02: Detection of Python Interpreter for Command python3 and args  failed as file  does not exist
Error 2021-09-04 19:21:03: Failed to get interpreter information for 'D:\Python\Python25\python.exe' [Error: Command failed: "D:\\Python\Python25\python.exe" "c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\interpreterInfo.py"
Traceback (most recent call last):
  File "c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\interpreterInfo.py", line 4, in <module>
    import json
ImportError: No module named json

	at ChildProcess.exithandler (child_process.js:317:12)
	at ChildProcess.emit (events.js:315:20)
	at maybeClose (internal/child_process.js:1048:16)
	at Process.ChildProcess._handle.onexit (internal/child_process.js:288:5)] {
  killed: false,
  code: 1,
  signal: null,
  cmd: '"D:\\\\Python\\Python25\\python.exe" "c:\\Users\\User\\.vscode\\extensions\\ms-python.python-2021.9.1191016588\\pythonFiles\\interpreterInfo.py"'
}
Error 2021-09-04 19:21:03: Failed to get interpreter information for 'D:\Python\Python24\python.exe' [Error: Command failed: "D:\\Python\Python24\python.exe" "c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\interpreterInfo.py"
Traceback (most recent call last):
  File "c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\interpreterInfo.py", line 4, in ?
    import json
ImportError: No module named json

	at ChildProcess.exithandler (child_process.js:317:12)
	at ChildProcess.emit (events.js:315:20)
	at maybeClose (internal/child_process.js:1048:16)
	at Process.ChildProcess._handle.onexit (internal/child_process.js:288:5)] {
  killed: false,
  code: 1,
  signal: null,
  cmd: '"D:\\\\Python\\Python24\\python.exe" "c:\\Users\\User\\.vscode\\extensions\\ms-python.python-2021.9.1191016588\\pythonFiles\\interpreterInfo.py"'
}
Starting Pylance language server.
Error 2021-09-04 19:21:04: Failed to get interpreter information for 'C:\Users\User\AppData\Local\Microsoft\WindowsApps\python.exe' [Error: Command failed: "C:\\Users\User\AppData\Local\Microsoft\WindowsApps\python.exe" "c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\interpreterInfo.py"
Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases.

	at ChildProcess.exithandler (child_process.js:317:12)
	at ChildProcess.emit (events.js:315:20)
	at maybeClose (internal/child_process.js:1048:16)
	at Socket.<anonymous> (internal/child_process.js:439:11)
	at Socket.emit (events.js:315:20)
	at Pipe.<anonymous> (net.js:673:12)] {
  killed: false,
  code: 9009,
  signal: null,
  cmd: '"C:\\\\Users\\User\\AppData\\Local\\Microsoft\\WindowsApps\\python.exe" "c:\\Users\\User\\.vscode\\extensions\\ms-python.python-2021.9.1191016588\\pythonFiles\\interpreterInfo.py"'
}
Error 2021-09-04 19:21:04: Failed to get interpreter information for 'C:\Users\User\AppData\Local\Microsoft\WindowsApps\python3.exe' [Error: Command failed: "C:\\Users\User\AppData\Local\Microsoft\WindowsApps\python3.exe" "c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\interpreterInfo.py"
Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases.

	at ChildProcess.exithandler (child_process.js:317:12)
	at ChildProcess.emit (events.js:315:20)
	at maybeClose (internal/child_process.js:1048:16)
	at Socket.<anonymous> (internal/child_process.js:439:11)
	at Socket.emit (events.js:315:20)
	at Pipe.<anonymous> (net.js:673:12)] {
  killed: false,
  code: 9009,
  signal: null,
  cmd: '"C:\\\\Users\\User\\AppData\\Local\\Microsoft\\WindowsApps\\python3.exe" "c:\\Users\\User\\.vscode\\extensions\\ms-python.python-2021.9.1191016588\\pythonFiles\\interpreterInfo.py"'
}
Error 2021-09-04 19:21:04: Failed to get interpreter information for 'C:\Users\User\AppData\Local\Microsoft\WindowsApps\python3.exe' [Error: Command failed: "C:\\Users\User\AppData\Local\Microsoft\WindowsApps\python3.exe" "c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\interpreterInfo.py"
Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases.

	at ChildProcess.exithandler (child_process.js:317:12)
	at ChildProcess.emit (events.js:315:20)
	at maybeClose (internal/child_process.js:1048:16)
	at Socket.<anonymous> (internal/child_process.js:439:11)
	at Socket.emit (events.js:315:20)
	at Pipe.<anonymous> (net.js:673:12)] {
  killed: false,
  code: 9009,
  signal: null,
  cmd: '"C:\\\\Users\\User\\AppData\\Local\\Microsoft\\WindowsApps\\python3.exe" "c:\\Users\\User\\.vscode\\extensions\\ms-python.python-2021.9.1191016588\\pythonFiles\\interpreterInfo.py"'
}
Error 2021-09-04 19:21:04: Failed to get interpreter information for 'C:\Users\User\AppData\Local\Microsoft\WindowsApps\python.exe' [Error: Command failed: "C:\\Users\User\AppData\Local\Microsoft\WindowsApps\python.exe" "c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\interpreterInfo.py"
Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases.

	at ChildProcess.exithandler (child_process.js:317:12)
	at ChildProcess.emit (events.js:315:20)
	at maybeClose (internal/child_process.js:1048:16)
	at Socket.<anonymous> (internal/child_process.js:439:11)
	at Socket.emit (events.js:315:20)
	at Pipe.<anonymous> (net.js:673:12)] {
  killed: false,
  code: 9009,
  signal: null,
  cmd: '"C:\\\\Users\\User\\AppData\\Local\\Microsoft\\WindowsApps\\python.exe" "c:\\Users\\User\\.vscode\\extensions\\ms-python.python-2021.9.1191016588\\pythonFiles\\interpreterInfo.py"'
}
> conda --version
> D:\Python\Python39\python.exe -c "
import unittest
import inspect

def get_sourceline(obj):
    s, n = inspect.getsourcelines(obj)
    for i, v in enumerate(s):
        if v.strip().startswith('def'):
            return str(n+i)
    return '*'

def generate_test_cases(suite):
    for test in suite:
        if isinstance(test, unittest.TestCase):
            yield test
        else:
            for test_case in generate_test_cases(test):
                yield test_case

loader = unittest.TestLoader()
suite = loader.discover("./unittests", pattern="*unittest.py")

print("start")  # Don't remove this line
loader_errors = []
for s in generate_test_cases(suite):
    tm = getattr(s, s._testMethodName)
    testId = s.id()
    if testId.startswith("unittest.loader._FailedTest"):
        loader_errors.append(s._exception)
    else:
        print(testId.replace(".", ":") + ":" + get_sourceline(tm))

for error in loader_errors:
    try:
        print("=== exception start ===")
        print(error.msg)
        print("=== exception end ===")
    except:
        pass
"
cwd: d:\Documents\PythonProjects\PythonQuickRefTest
> D:\Python\Python39\python.exe -c "
import unittest
import inspect

def get_sourceline(obj):
    s, n = inspect.getsourcelines(obj)
    for i, v in enumerate(s):
        if v.strip().startswith('def'):
            return str(n+i)
    return '*'

def generate_test_cases(suite):
    for test in suite:
        if isinstance(test, unittest.TestCase):
            yield test
        else:
            for test_case in generate_test_cases(test):
                yield test_case

loader = unittest.TestLoader()
suite = loader.discover("./unittests", pattern="*unittest.py")

print("start")  # Don't remove this line
loader_errors = []
for s in generate_test_cases(suite):
    tm = getattr(s, s._testMethodName)
    testId = s.id()
    if testId.startswith("unittest.loader._FailedTest"):
        loader_errors.append(s._exception)
    else:
        print(testId.replace(".", ":") + ":" + get_sourceline(tm))

for error in loader_errors:
    try:
        print("=== exception start ===")
        print(error.msg)
        print("=== exception end ===")
    except:
        pass
"
cwd: d:\Documents\PythonProjects\PythonQuickRefTest
> D:\Python\Python39\python.exe c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\visualstudio_py_testlauncher.py --us=./unittests --up=*unittest.py --uvInt=2 --result-port=49811 -tclasses_unittest.TestClasses.test_about_classes_types_and_objects --testFile=d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py
cwd: d:\Documents\PythonProjects\PythonQuickRefTest
> D:\Python\Python39\python.exe c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\visualstudio_py_testlauncher.py --us=./unittests --up=*unittest.py --uvInt=2 --result-port=49811 -tclasses_unittest.TestClasses.test_about_classes_types_and_objects --testFile=d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py
cwd: d:\Documents\PythonProjects\PythonQuickRefTest
Error 2021-09-04 19:21:21: Error: read ECONNRESET
> D:\Python\Python39\python.exe c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\visualstudio_py_testlauncher.py --us=./unittests --up=*unittest.py --uvInt=2 --result-port=49811 -tclasses_unittest.TestClasses.test_class_creation_and_instantiation --testFile=d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py
cwd: d:\Documents\PythonProjects\PythonQuickRefTest
> D:\Python\Python39\python.exe c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\visualstudio_py_testlauncher.py --us=./unittests --up=*unittest.py --uvInt=2 --result-port=49811 -tclasses_unittest.TestClasses.test_class_creation_and_instantiation --testFile=d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py
cwd: d:\Documents\PythonProjects\PythonQuickRefTest
Error 2021-09-04 19:21:21: Error: read ECONNRESET
> D:\Python\Python39\python.exe c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\visualstudio_py_testlauncher.py --us=./unittests --up=*unittest.py --uvInt=2 --result-port=49811 -tclasses_unittest.TestClasses.test_class_inheritance --testFile=d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py
cwd: d:\Documents\PythonProjects\PythonQuickRefTest
> D:\Python\Python39\python.exe c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\visualstudio_py_testlauncher.py --us=./unittests --up=*unittest.py --uvInt=2 --result-port=49811 -tclasses_unittest.TestClasses.test_class_inheritance --testFile=d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py
cwd: d:\Documents\PythonProjects\PythonQuickRefTest
Error 2021-09-04 19:21:21: Error: read ECONNRESET
> D:\Python\Python39\python.exe c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\visualstudio_py_testlauncher.py --us=./unittests --up=*unittest.py --uvInt=2 --result-port=49811 -tclasses_unittest.TestClasses.test_descriptors --testFile=d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py
cwd: d:\Documents\PythonProjects\PythonQuickRefTest
> D:\Python\Python39\python.exe c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\visualstudio_py_testlauncher.py --us=./unittests --up=*unittest.py --uvInt=2 --result-port=49811 -tclasses_unittest.TestClasses.test_descriptors --testFile=d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py
cwd: d:\Documents\PythonProjects\PythonQuickRefTest
Error 2021-09-04 19:21:22: Error: read ECONNRESET
> D:\Python\Python39\python.exe c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\visualstudio_py_testlauncher.py --us=./unittests --up=*unittest.py --uvInt=2 --result-port=49811 -tclasses_unittest.TestClasses.test_property_attributes --testFile=d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py
cwd: d:\Documents\PythonProjects\PythonQuickRefTest
> D:\Python\Python39\python.exe c:\Users\User\.vscode\extensions\ms-python.python-2021.9.1191016588\pythonFiles\visualstudio_py_testlauncher.py --us=./unittests --up=*unittest.py --uvInt=2 --result-port=49811 -tclasses_unittest.TestClasses.test_property_attributes --testFile=d:\Documents\PythonProjects\PythonQuickRefTest\unittests\classes_unittest.py
cwd: d:\Documents\PythonProjects\PythonQuickRefTest
Error 2021-09-04 19:21:22: Error: read ECONNRESET
Error 2021-09-04 19:21:22: undefined

Metadata

Metadata

Assignees

Labels

area-testingbugIssue identified by VS Code Team member as probable bug

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions