|
| 1 | +# PyScript Test Suite |
| 2 | + |
| 3 | +There are three aspects to our test suite. These are reflected in the layout of |
| 4 | +the test directory: |
| 5 | + |
| 6 | +1. `python` - contains the Python based test suite to exercise Python code |
| 7 | + **within** PyScript. These tests are run four differeng ways to ensure all |
| 8 | + combination of MicroPython/Pyodide and main thread/worker contexts are |
| 9 | + checked. |
| 10 | +2. `javascript` - contains JavaScript tests to exercise PyScript _itself_, in |
| 11 | + the browser. |
| 12 | +3. `manual` - contains tests to run manually in a browser, due to the complex |
| 13 | + nature of the tests. |
| 14 | + |
| 15 | +We use [Playwright](https://playwright.dev/) to automate the running of the |
| 16 | +Python and JavaScript test suites. We use |
| 17 | +[uPyTest](https://github.com/ntoll/upytest) as a test framework for the Python |
| 18 | +test suite. uPyTest is a "PyTest inspired" framework for running tests in the |
| 19 | +browser on both MicroPython and Pyodide. |
| 20 | + |
| 21 | +The automated (Playwright) tests are specified in the `integration.spec.js` |
| 22 | +file in this directory. |
| 23 | + |
| 24 | +All automatic tests live in either the `python` or `javascript` folders. All |
| 25 | +the tests in these folder are run by CI or locally run by `make test` in the |
| 26 | +root of this project. Alternatively, run `npm run test:integration` in the |
| 27 | +PyScript source directory. |
| 28 | + |
| 29 | +Similarly, some tests can only be run manually (due to their nature or |
| 30 | +underlying complexity). These are in the `manual` directory and are in the form |
| 31 | +of separate directories (each containing an `index.html`) or individual `*.html` |
| 32 | +files to which you point your browser. Each separate test may exercise |
| 33 | +JavaScript or Python code (or both), and the context for each separate test is |
| 34 | +kept carefully isolated. |
| 35 | + |
| 36 | +Some rules of thumb: |
| 37 | + |
| 38 | +* We don't test upstream projects: we assume they have their own test suites, |
| 39 | + and if we find bugs, we file an issue upstream with an example of how to |
| 40 | + recreate the problem. |
| 41 | +* We don't test browser functionality, we just have to trust that browsers work |
| 42 | + as advertised. Once again, if we find an issue, we report upstream. |
| 43 | +* All test cases should include commentary describing the **intent** and |
| 44 | + context of the test. |
| 45 | +* Tests in Python use [uPyTest](https://github.com/ntoll/upytest) (see the |
| 46 | + README for documentation), an "inspired by PyTest" test framework that works |
| 47 | + with both MicroPython and Pyodide in the browser. This means that all |
| 48 | + Python tests should work with both interpreters. |
| 49 | +* Tests in JavaScript... (Andrea to explain). ;-) |
0 commit comments