We recently added a Python script for running pytest discovery. It has plenty of unit tests, but no functional tests. Those are tests that use actual external dependencies (e.g. pytest, filesystem) and don't stub things out. (Technically, system tests are a kind of functional tests, but the distinction isn't important in this case).
Since we want to ensure that we've integrated properly with pytest, we should add functional tests for the adapter script. This will involve the following tasks:
These functional tests will go in pythonFiles/tests/testing_tools/adapter/test_functional.py (new file).
Simple Test Tree
pythonFiles/
tests/testing_tools/adapter/
.data/ # new
tests/ # test root
test_spam.py
#def test_simple():
# assert True
Complex Test Tree
Note: the following is probably not complete; there are other things worth adding.
pythonFiles/
tests/testing_tools/adapter/
.data/ # new
tests/ # test root
test_doctest.txt
* ...
test_pytest.py
* simple module-level test function
* module-level doctests
* marked test functions
+ skip, skipif, xfail, filterwarnings, custom # see https://docs.pytest.org/en/latest/reference.html#marks
+ one function for each
+ at least one function with multiple marks
* parameterized test functions
+ no parameters, single case
+ one parameter, single case
+ one parameter, multiple cases
+ multiple parameters, multiple cases
+ with *args & **kwargs parameters, multiple cases
+ 4 parameters, 3 decorators
+ decorator with "marks" arg (skip/xfail/etc)
* test function that calls pytest.skip()
* test function that calls pytest.fail()
* test function that raises Exception
* for loop that generates test functions dynamically
* suite (a test class)
+ simple test method
+ suite-level doctests
+ marked ("skip") test method
+ parameterized test method
+ nested suite
+ nested suite
+ simple test method
+ nested suite (no methods)
+ nested suite (no methods)
* another suite
+ a simple test method
* an empty suite
* suite using a fixture
test_unittest.py
* test suite inheriting from unittest.TestCase
+ simple test function
+ skipped test method (decorator)
+ skip-if'ed test method (decorator)
+ test method with expected failure decorator
+ test method that raises unittest.Skip
+ test method that calls unittest.skip()
+ test method that raises Exception
+ test method that uses one "with self.subtest():"
+ test method that uses multiple "with self.subtest():" (nested)
+ for loop that generates test methods dynamically
+ nested class (no base class)
+ nested class inheriting from unittest.TestCase
* another suite
+ a simple test method
* an empty suite
test_mixed.py
* module-level test function (a la pytest)
* marked ("skip") test function
* class with no base class (a la pytest)
+ a simple test method
* class inheriting from unittest.TestCase
+ simple test method
spam.py # note: no "test_" prefix
* module-level test function
test_foo.py
* module-level test function
test_42.py
* module-level test function
test_42-43.py # note the hyphen
* module-level test function
v/
__init__.py
spam.py
* pytest-style suite
* simple test function ("test_simple")
* simple test function ("test_simple")
test_eggs.py
* "from .spam import *"
test_ham.py
* "from .spam import test_simple"
* "from .spam import test_simple as test_not_hard"
test_spam.py
* "from .spam import test_simple"
* simple test function ("test_simpler")
w/
# no __init__.py
test_spam.py
* simple test function
test_spam_ex.py
* simple test function
x/y/z/ # each with a __init__.py
test_ham.py
* a simple test function
a/
test_spam.py
* a simple test function
b/
test_spam.py
* a simple test function
Also check the following prior art for more examples to include:
We recently added a Python script for running pytest discovery. It has plenty of unit tests, but no functional tests. Those are tests that use actual external dependencies (e.g. pytest, filesystem) and don't stub things out. (Technically, system tests are a kind of functional tests, but the distinction isn't important in this case).
Since we want to ensure that we've integrated properly with pytest, we should add functional tests for the adapter script. This will involve the following tasks:
pythonFiles/tests/__main__.pyfor opting in to (or out of) functional tests (e.g.--functionalor--no-functional, respectively)These functional tests will go in
pythonFiles/tests/testing_tools/adapter/test_functional.py(new file).Simple Test Tree
Complex Test Tree
Note: the following is probably not complete; there are other things worth adding.
Also check the following prior art for more examples to include: