| layout | doc |
|---|---|
| title | 12-ParallelExecution - Codeception - Documentation |
When execution time of your tests is longer than a coffee break, it is a good reason to think about making your tests faster. If you have already tried to run them on SSD drive, or to use PhantomJS instead of Selenium, and the execution time still upsets you, it might be a good idea to run your tests in parallel.
Codeception does not provide a command like run-parallel. There is no common solution that can play well for everyone. Here are the questions you will need to answer:
- How parallel processes will be executed?
- How parallel processes won't affect each other?
- Will they use different databases?
- Will they use different hosts?
- How should I split my tests across parallel processes?
There are two approaches to achieve parallelization. We can use Docker and run each process inside isolated containers, and have those containers executed simultaneously.
🚧 Section is under construction
dockeror Docker Toolbox
Run Docker image
docker run codeception/codeception
Running tests from a project, by mounting the current path as a host-volume into the container.
The default working directory in the container is /project.
docker run -v ${PWD}:/project codeception/codeception run
For local testing of the Codeception repository with Docker and docker-copmose, please refer to the testing documentation.
Parallel Test Execution consists of 3 steps:
- splitting tests
- running tests in parallel
- merging results
We propose to perform those steps using a task runner. In this guide we will use Robo task runner. It is a modern PHP task runner that is very easy to use. It uses Symfony Process to spawn background and parallel processes. Just what we need for the step 2! What about steps 1 and 3? We have created robo tasks for splitting tests into groups and merging resulting JUnit XML reports.
To conclude, we need:
- Robo, a task runner.
- robo-paracept - Codeception tasks for parallel execution.
Robo is recommended to be installed globally. You can either do a global install with Composer or download robo.phar and put it somewhere in PATH.
Execute robo in the root of your project
{% highlight bash %}
$ robo RoboFile.php not found in this dir Should I create RoboFile here? (y/n)
{% endhighlight %}
Confirm to create RoboFile.php.
{% highlight php %}
taskSplitTestFilesByGroups(5) ->projectRoot('.') ->testsFrom('tests/functional') ->groupsTo('tests/_log/p') ->run(); // alternatively $this->taskSplitTestsByGroups(5) ->projectRoot('.') ->testsFrom('tests/functional') ->groupsTo('tests/_log/p') ->run(); } {% endhighlight %} In second case `Codeception\TestLoader` class will be used and test classes will be loaded into memory. Let's prepare group files: {% highlight bash %} $ robo parallel:split-tests [Codeception\Task\SplitTestFilesByGroupsTask] Processing 33 files [Codeception\Task\SplitTestFilesByGroupsTask] Writing tests/_log/p1 [Codeception\Task\SplitTestFilesByGroupsTask] Writing tests/_log/p2 [Codeception\Task\SplitTestFilesByGroupsTask] Writing tests/_log/p3 [Codeception\Task\SplitTestFilesByGroupsTask] Writing tests/_log/p4 [Codeception\Task\SplitTestFilesByGroupsTask] Writing tests/_log/p5 {% endhighlight %} Now we have group files. We should update `codeception.yml` to load generated group files. In our case we have groups: *p1*, *p2*, *p3*, *p4*, *p5*. {% highlight yaml %} groups: p*: tests/_log/p* {% endhighlight %} Let's try to execute tests from the second group: {% highlight bash %} $ php codecept run functional -g p2 {% endhighlight %} #### Step 2: Running Tests As it was mentioned, Robo has `ParallelExec` task to spawn background processes. But you should not think of it as the only option. For instance, you can execute tests remotely via SSH, or spawn processes with Gearman, RabbitMQ, etc. But in our example we will use 5 background processes: {% highlight php %} taskParallelExec(); for ($i = 1; $i <= 5; $i++) { $parallel->process( $this->taskCodecept() // use built-in Codecept task ->suite('acceptance') // run acceptance tests ->group("p$i") // for all p* groups ->xml("tests/_log/result_$i.xml") // save XML results ); } return $parallel->run(); } {% endhighlight %} We missed something really important. We forgot to define different databases for different processes. This can be done using [Environments](http://codeception.com/docs/07-AdvancedUsage#Environments). Let's define 5 new environments in `acceptance.suite.yml`: {% highlight yaml %} class_name: AcceptanceTester modules: enabled: [WebDriver, Db] config: Db: dsn: 'mysql:dbname=testdb;host=127.0.0.1' user: 'root' dump: 'tests/_data/dump.sql' populate: true cleanup: true WebDriver: url: 'http://localhost/' env: p1: modules: config: Db: dsn: 'mysql:dbname=testdb_1;host=127.0.0.1' WebDriver: url: 'http://test1.localhost/' p2: modules: config: Db: dsn: 'mysql:dbname=testdb_2;host=127.0.0.1' WebDriver: url: 'http://test2.localhost/' p3: modules: config: Db: dsn: 'mysql:dbname=testdb_3;host=127.0.0.1' WebDriver: url: 'http://test3.localhost/' p4: modules: config: Db: dsn: 'mysql:dbname=testdb_4;host=127.0.0.1' WebDriver: url: 'http://test4.localhost/' p5: modules: config: Db: dsn: 'mysql:dbname=testdb_5;host=127.0.0.1' WebDriver: url: 'http://test5.localhost/' {% endhighlight %} Now, we should update our `parallelRun` method to use corresponding environment: {% highlight php %} taskParallelExec(); for ($i = 1; $i <= 5; $i++) { $parallel->process( $this->taskCodecept() // use built-in Codecept task ->suite('acceptance') // run acceptance tests ->group("p$i") // for all p* groups ->env("p$i") // in its own environment ->xml("tests/_log/result_$i.xml") // save XML results ); } return $parallel->run(); } {% endhighlight %} Now, we can execute tests with {% highlight bash %} $ robo parallel:run {% endhighlight %} #### Step 3: Merge Results We should not rely on console output when running our tests. In case of `parallelExec` task, some text can be missed. We recommend to save results as JUnit XML, which can be merged and plugged into Continuous Integration server. {% highlight php %} taskMergeXmlReports(); for ($i=1; $i<=5; $i++) { $merge->from("/tests/_log/result_$i.xml"); } $merge->into("/tests/_log/result.xml") ->run(); } {% endhighlight %} `result.xml` file will be generated. It can be processed and analyzed. #### All Together To create one command to rule them all we can define new public method `parallelAll` and execute all commands. We will save the result of `parallelRun` and use it for our final exit code: {% highlight php %} parallelSplitTests(); $result = $this->parallelRun(); $this->parallelMergeResults(); return $result; } {% endhighlight %} ## Conclusion Codeception does not provide tools for parallel test execution. This is a complex task and solutions may vary depending on a project. We use [Robo](http://robo.li) task runner as an external tool to perform all required steps. To prepare our tests to be executed in parallel we use Codeception features of dynamic groups and environments. To do even more we can create Extensions and Group classes to perform dynamic configuration depending on a test process. * **Previous Chapter: [< ContinuousIntegration](/docs/12-ContinuousIntegration)**