You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Allure 2 comes with three built-in hierarchies: suites, bdd, and packages. Each comes with some drawbacks.
package is a well-known label that is usually automatically filled by Allure integrations. When opening the report, Allure splits the package values by . and groups them into a tree. Each package value therefore is a path to the node containing the test result in that tree.
The drawbacks:
It groups tests on a higher level (e.g., on the file or the package levels, depending on the language and the framework).
It's a label, which means the value is stored as a single string. The path is restored by Allure by splitting the value by .. If a path fragment (e.g., a filename) contains ., it might lead to unexpected results.
suite and bdd are based on two sets of labels: parentSuite, suite, and subSuite and epic, feature, and story. Each label corresponds to one level of the corresponding hierarchy. Many integrations fill one of the hierarchies, leaving the other to a user.
The drawbacks:
It's not always clear which hierarchy a specific integration fills. Therefore, it may be unclear which hierarchy you should open first to get a better view of the test results.
Both hierarchies can only have up to three levels.
Both hierarchies are exposed to users: users may add labels to them. Therefore, it's possible to define more than one label of a specific level (e.g., two suite labels). It's not clear how to deal with this. Currently, Allure will show it multiple times in the tree, which may be confusing.
It's harder to combine different integrations as they may provide different hierarchies out of the box.
Describe the solution you'd like
A solution is to introduce the default hierarchy directly to Allure model. The TestResult type should include a new property, say, titlePath, which, should be an array of strings (most probably).
Rules for integrations:
An integration must translate the hierarchical structure of tests in the framework to titlePath values of the corresponding test results.
An integration must respect all levels. Two test results have the same titlePath if and only if they
a) correspond to the same test (e.g., are two retries of the test), or
b) correspond to sibling tests (i.e., tests that share all their parents: suites/files/packages/etc).
Note
A combination of a test's titlePath and name can also be used to calculate the fullName value.
Allure 3 will use titlePath values to display the test results in a tree view similar to how it uses package labels. The main difference is that no splitting is involved (values are already arrays), which means no ambiguous split characters and no need for encoding/decoding.
When run, the test results with the following titlePath values are created:
["foo.test.js","suite"]// for test-1 and test-2["dir","bar.test.js","suite"]// for test-3["dir","baz.test.js","suite 1"]// for test-4["dir","baz.test.js","suite 2"]// for test-5
Allure Report will display the results in the form of the following tree (it may compress nodes with a sinlge child):
foo.test.js
├─ suite
│ ├─ test-1
│ ├─ test-2
dir
├─ bar.test.js
│ ├─ suite
│ ├─ test-3
├─ baz.test.js
├─ suite 1
│ ├─ test-4
├─ suite 2
├─ test-5
Is your feature request related to a problem? Please describe.
Allure 2 comes with three built-in hierarchies:
suites,bdd, andpackages. Each comes with some drawbacks.packageis a well-known label that is usually automatically filled by Allure integrations. When opening the report, Allure splits thepackagevalues by.and groups them into a tree. Eachpackagevalue therefore is a path to the node containing the test result in that tree.The drawbacks:
.. If a path fragment (e.g., a filename) contains., it might lead to unexpected results.suiteandbddare based on two sets of labels:parentSuite,suite, andsubSuiteandepic,feature, andstory. Each label corresponds to one level of the corresponding hierarchy. Many integrations fill one of the hierarchies, leaving the other to a user.The drawbacks:
suitelabels). It's not clear how to deal with this. Currently, Allure will show it multiple times in the tree, which may be confusing.Describe the solution you'd like
A solution is to introduce the default hierarchy directly to Allure model. The
TestResulttype should include a new property, say,titlePath, which, should be an array of strings (most probably).Rules for integrations:
titlePathvalues of the corresponding test results.titlePathif and only if theya) correspond to the same test (e.g., are two retries of the test), or
b) correspond to sibling tests (i.e., tests that share all their parents: suites/files/packages/etc).
Note
A combination of a test's
titlePathand name can also be used to calculate thefullNamevalue.Allure 3 will use
titlePathvalues to display the test results in a tree view similar to how it usespackagelabels. The main difference is that no splitting is involved (values are already arrays), which means no ambiguous split characters and no need for encoding/decoding.Example
Suppose we have the following tests:
In
foo.test.js:In
dir/bar.test.js:In
dir/baz.test.js:When run, the test results with the following
titlePathvalues are created:Allure Report will display the results in the form of the following tree (it may compress nodes with a sinlge child):