Skip to content

Commit be39586

Browse files
jgebaljgebal
committed
Added support for --%name annotation, to name the contexts.
Changed behavior of ``--%context` annotation. The value f annotation now only indicates the context description. Resolves #1016
1 parent 2abd27e commit be39586

8 files changed

Lines changed: 506 additions & 153 deletions

File tree

‎docs/userguide/annotations.md‎

Lines changed: 138 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -21,9 +21,9 @@ We strongly recommend putting package level annotations at the very top of packa
2121
| --- | --- | --- |
2222
| `--%suite(<description>)` | Package | Mandatory. Marks package as a test suite. Optional suite description can be provided (see `displayname`). |
2323
| `--%suitepath(<path>)` | Package | Similar to java package. The annotation allows logical grouping of suites into hierarchies. |
24-
| `--%displayname(<description>)` | Package/procedure | Human-readable and meaningful description of a context/suite/test. Provides description to a `context` when used within `context`. When used with `test` or `suite` annotation, overrides the `<description>` provided with `suite`/`test`. |
24+
| `--%displayname(<description>)` | Package/procedure | Human-readable and meaningful description of a context/suite/test. Overrides the `<description>` provided with `suite`/`test`/`context` annotation. This annotation is redundant and might be removed in future releases. |
2525
| `--%test(<description>)` | Procedure | Denotes that the annotated procedure is a unit test procedure. Optional test description can by provided (see `displayname`). |
26-
| `--%throws(<exception>[,...])`| Procedure | Denotes that the annotated test procedure must throw one of the exceptions provided. Supported forms of exceptions are: numeric literals, numeric contant names, exception constant names, predefined Oracle exception names. |
26+
| `--%throws(<exception>[,...])`| Procedure | Denotes that the annotated test procedure must throw one of the exceptions provided. Supported forms of exceptions are: numeric literals, numeric constant names, exception constant names, predefined Oracle exception names. |
2727
| `--%beforeall` | Procedure | Denotes that the annotated procedure should be executed once before all elements of the suite. |
2828
| `--%beforeall([[<owner>.]<package>.]<procedure>[,...])` | Package | Denotes that the mentioned procedure(s) should be executed once before all elements of the suite. |
2929
| `--%afterall` | Procedure | Denotes that the annotated procedure should be executed once after all elements of the suite. |
@@ -36,7 +36,8 @@ We strongly recommend putting package level annotations at the very top of packa
3636
| `--%aftertest([[<owner>.]<package>.]<procedure>[,...])` | Procedure | Denotes that mentioned procedure(s) should be executed after the annotated `%test` procedure. |
3737
| `--%rollback(<type>)` | Package/procedure | Defines transaction control. Supported values: `auto`(default) - a savepoint is created before invocation of each "before block" is and a rollback to specific savepoint is issued after each "after" block; `manual` - rollback is never issued automatically. Property can be overridden for child element (test in suite) |
3838
| `--%disabled` | Package/procedure | Used to disable a suite or a test. Disabled suites/tests do not get executed, they are however marked and reported as disabled in a test run. |
39-
| `--%context(<name>)` | Package | Denotes start of a named context (sub-suite) in a suite package |
39+
| `--%context(<description>)` | Package | Denotes start of a named context (sub-suite) in a suite package an optional description for context can be provided. |
40+
| `--%name(<name>)` | Package | Denotes name for a context. Must be placed after the context annotation and before start of nested context. |
4041
| `--%endcontext` | Package | Denotes end of a nested context (sub-suite) in a suite package |
4142
| `--%tags` | Package/procedure | Used to label a test or a suite for purpose of identification |
4243

@@ -1008,14 +1009,15 @@ Contexts allow for creating sub-suites within a suite package and they allow for
10081009
In essence, context behaves like a suite within a suite.
10091010

10101011
Context have following characteristics:
1011-
- context starts with the `--%context` annotation and ends with `--%endcontext`
1012-
- can have a name provided as parameter for example `--%context(remove_rooms_by_name)`. This is different than with `suite` and `test` annotations, where name is taken from test `package/procedure`
1013-
- when no name is provided for context, the context is named `context_N` where `N` is the number of the context in suite or parent context
1014-
- context name must be unique within it's parent (suite or parent context)
1012+
- context starts with the `--%context` annotation and ends with `--%endcontext`. Everything placed between those two annotations belongs to that context
1013+
- can have a description provided as parameter for example `--%context(Some interesting stuff)`.
1014+
- can have a name provided with `--%name` annotation. This is different than with `suite` and `test` annotations, where name is taken from `package/procedure` name.
1015+
- contexts can be nested, you can place a context inside another context
1016+
- when no name is provided for context, the context is named `context_N` where `N` is the number of the context in suite or parent context.
1017+
- context name must be unique within it's parent (suite / parent context)
10151018
- if context name is not unique within it's parent, context and it's entire content is excluded from execution
10161019
- context name should not contain spaces or special characters
10171020
- context name cannot contain a `.` (full stop/period) character
1018-
- contexts can be nested, so a context can be nested within another context
10191021
- suite/context can have multiple nested sibling contexts in it
10201022
- contexts can have their own `--%beforeall`, `--%beforeeach`, `--%afterall` and `--%aftereach` procedures
10211023
- `--%beforeall`, `--%beforeeach`, `--%afterall` and `--%aftereach` procedures defined at ancestor level, propagate to context
@@ -1231,8 +1233,7 @@ Example of nested contexts test suite specification.
12311233
create or replace package queue_spec as
12321234
--%suite(Queue specification)
12331235

1234-
--%context(a_new_queue)
1235-
--%displayname(A new queue)
1236+
--%context(A new queue)
12361237

12371238
--%test(Is empty)
12381239
procedure is_empty;
@@ -1241,8 +1242,7 @@ create or replace package queue_spec as
12411242
--%test(Cannot be created with non positive bounding capacity)
12421243
procedure non_positive_bounding_cap;
12431244
--%endcontext
1244-
--%context(an_empty_queue)
1245-
--%displayname(An empty queue)
1245+
--%context(An empty queue)
12461246

12471247
--%test(Dequeues an empty value)
12481248
procedure deq_empty_value;
@@ -1251,19 +1251,16 @@ create or replace package queue_spec as
12511251
--%test(Becomes non empty when non null value enqueued)
12521252
procedure non_empty_after_enq;
12531253
--%endcontext
1254-
--%context(a_non_empty_queue)
1255-
--%displayname(A non empty queue)
1254+
--%context(A non empty queue)
12561255

1257-
--%context(that_is_not_full)
1258-
--%displayname(that is not full)
1256+
--%context(that is not full)
12591257

12601258
--%test(Becomes longer when non null value enqueued)
12611259
procedure grow_on_enq_non_null;
12621260
--%test(Becomes full when enqueued up to capacity)
12631261
procedure full_on_enq_to_cap;
12641262
--%endcontext
1265-
--%context(that_is_full)
1266-
--%displayname(That is full)
1263+
--%context(that is full)
12671264

12681265
--%test(Ignores further enqueued values)
12691266
procedure full_ignore_enq;
@@ -1312,6 +1309,122 @@ Finished in .088573 seconds
13121309

13131310
Suite nesting allows for organizing tests into human-readable specification of behavior.
13141311

1312+
### Name
1313+
The `--%name` annotation is currently only used only for naming a context.
1314+
If a context doesn't have explicit name specified, then the name is given automatically by framework.
1315+
1316+
The automatic name will be `context_#n` where `n` is a context number within a suite/parent context.
1317+
1318+
The `--%name` can be useful when you would like to run only a specific context or its items by `suitepath`.
1319+
1320+
Consider the below example.
1321+
1322+
```sql
1323+
create or replace package queue_spec as
1324+
--%suite(Queue specification)
1325+
1326+
--%context(A new queue)
1327+
1328+
--%test(Cannot be created with non positive bounding capacity)
1329+
procedure non_positive_bounding_cap;
1330+
--%endcontext
1331+
--%context(An empty queue)
1332+
1333+
--%test(Becomes non empty when non null value enqueued)
1334+
procedure non_empty_after_enq;
1335+
--%endcontext
1336+
--%context(A non empty queue)
1337+
1338+
--%context(that is not full)
1339+
1340+
--%test(Becomes full when enqueued up to capacity)
1341+
procedure full_on_enq_to_cap;
1342+
--%endcontext
1343+
--%context(that is full)
1344+
1345+
--%test(Becomes non full when dequeued)
1346+
procedure non_full_on_deq;
1347+
--%endcontext
1348+
1349+
--%endcontext
1350+
end;
1351+
```
1352+
1353+
In the above code, suitepaths, context names and context descriptions will be as follows.
1354+
1355+
| suitepath | description | name |
1356+
|-----------|------------|------|
1357+
| queue_spec | Queue specification | queue_spec |
1358+
| queue_spec.context_#1 | A new queue | context_#1 |
1359+
| queue_spec.context_#2 | An empty queue | context_#2 |
1360+
| queue_spec.context_#3 | A non empty queue | context_#3 |
1361+
| queue_spec.context_#3.context_#1 | that is not full | context_#1 |
1362+
| queue_spec.context_#3.context_#2 | that is full | context_#2 |
1363+
1364+
In order to run only the tests for the context `A non empty queue that is not full` you will need to call utPLSQL as below:
1365+
```sql
1366+
exec ut.run(':queue_spec.context_#3.context_#1');
1367+
```
1368+
1369+
You can use `--%name` annotation to explicitly name contexts on suitepath.
1370+
```sql
1371+
create or replace package queue_spec as
1372+
--%suite(Queue specification)
1373+
1374+
--%context(A new queue)
1375+
--%name(a_new_queue)
1376+
1377+
--%test(Cannot be created with non positive bounding capacity)
1378+
procedure non_positive_bounding_cap;
1379+
--%endcontext
1380+
--%context(An empty queue)
1381+
--%name(an_empty_queue)
1382+
1383+
--%test(Becomes non empty when non null value enqueued)
1384+
procedure non_empty_after_enq;
1385+
--%endcontext
1386+
--%context(A non empty queue)
1387+
--%name(a_non_empty_queue)
1388+
1389+
--%context(that is not full)
1390+
--%name(that_is_not_full)
1391+
1392+
--%test(Becomes full when enqueued up to capacity)
1393+
procedure full_on_enq_to_cap;
1394+
--%endcontext
1395+
--%context(that is full)
1396+
--%name(that_is_full)
1397+
1398+
--%test(Becomes non full when dequeued)
1399+
procedure non_full_on_deq;
1400+
--%endcontext
1401+
1402+
--%endcontext
1403+
end;
1404+
```
1405+
1406+
In the above code, suitepaths, context names and context descriptions will be as follows.
1407+
1408+
| suitepath | description | name |
1409+
|-----------|------------|------|
1410+
| queue_spec | Queue specification | queue_spec |
1411+
| queue_spec.a_new_queue | A new queue | a_new_queue |
1412+
| queue_spec.an_empty_queue | An empty queue | an_empty_queue |
1413+
| queue_spec.a_non_empty_queue | A non empty queue | a_non_empty_queue |
1414+
| queue_spec.a_non_empty_queue.that_is_not_full | that is not full | that_is_not_full |
1415+
| queue_spec.a_non_empty_queue.that_is_full | that is full | that_is_full |
1416+
1417+
1418+
The `--%name` annotation is only relevant for:
1419+
- running subsets of tests by given context suitepath
1420+
- some of test reports, like `ut_junit_reporter` that use suitepath or test-suite element names (not descriptions) for reporting
1421+
1422+
#### Name naming convention
1423+
1424+
The value of `--%name` annotation must follow the following naming rules:
1425+
- cannot contain spaces
1426+
- cannot contain a `.` (full stop/dot)
1427+
- is case-insensitive
13151428

13161429
### Tags
13171430

@@ -1440,8 +1553,9 @@ If you want to create tests for your application it is recommended to structure
14401553
* Payments recognition
14411554
* Payments set off
14421555

1443-
The `%suitepath` annotation is used for such grouping. Even though test packages are defined in a flat structure the `%suitepath` is used by the framework to form them into a hierarchical structure. Your payments recognition test package might look like:
1556+
The `--%suitepath` annotation is used for such grouping. Even though test packages are defined in a flat structure the `--%suitepath` is used by the framework to form them into a hierarchical structure.
14441557

1558+
Your payments recognition test package might look like:
14451559
```sql
14461560
create or replace package test_payment_recognition as
14471561

@@ -1476,8 +1590,8 @@ create or replace package test_payment_set_off as
14761590
end test_payment_set_off;
14771591
```
14781592

1479-
When you execute tests for your application, the framework constructs a test suite for each test package. Then it combines suites into grouping suites by the `%suitepath` annotation value so that the fully qualified path to the `recognize_by_num` procedure is `USER:payments.test_payment_recognition.test_recognize_by_num`. If any of its expectations fails then the test is marked as failed, also the `test_payment_recognition` suite, the parent suite `payments` and the whole run is marked as failed.
1480-
The test report indicates which expectation has failed on the payments module. The payments recognition submodule is causing the failure as `recognize_by_num` has not met the expectations of the test. Grouping tests into modules and submodules using the `%suitepath` annotation allows you to logically organize your project's flat structure of packages into functional groups.
1593+
When you execute tests for your application, the framework constructs a test suite for each test package. Then it combines suites into grouping suites by the `--%suitepath` annotation value so that the fully qualified path to the `recognize_by_num` procedure is `USER:payments.test_payment_recognition.test_recognize_by_num`. If any of its expectations fails then the test is marked as failed, also the `test_payment_recognition` suite, the parent suite `payments` and the whole run is marked as failed.
1594+
The test report indicates which expectation has failed on the payments module. The payments recognition submodule is causing the failure as `recognize_by_num` has not met the expectations of the test. Grouping tests into modules and submodules using the `--%suitepath` annotation allows you to logically organize your project's flat structure of packages into functional groups.
14811595

14821596
An additional advantage of such grouping is the fact that every element level of the grouping can be an actual unit test package containing a common module level setup for all of the submodules. So in addition to the packages mentioned above you could have the following package.
14831597
```sql
@@ -1493,9 +1607,10 @@ create or replace package payments as
14931607

14941608
end payments;
14951609
```
1496-
A `%suitepath` can be provided in three ways:
1610+
1611+
When executing tests, `path` for executing tests can be provided in three ways:
14971612
* schema - execute all tests in the schema
1498-
* [schema]:suite1[.suite2][.suite3]...[.procedure] - execute all tests in all suites from suite1[.suite2][.suite3]...[.procedure] path. If schema is not provided, then the current schema is used. Example: `:all.rooms_tests`
1613+
* [schema]:suite1[.suite2][.suite3]...[.procedure] - execute all tests by `suitepath` in all suites on path suite1[.suite2][.suite3]...[.procedure]. If schema is not provided, then the current schema is used. Example: `:all.rooms_tests`
14991614
* [schema.]package[.procedure] - execute all tests in the specified test package. The whole hierarchy of suites in the schema is built before all before/after hooks or part suites for the provided suite package are executed as well. Example: `tests.test_contact.test_last_name_validator` or simply `test_contact.test_last_name_validator` if `tests` is the current schema.
15001615

15011616

0 commit comments

Comments
 (0)