Skip to content

Commit 94b7ea1

Browse files
pessepesse
committed
Merge remote-tracking branch 'upstream/develop' into feature/reporters_object_model_fix
2 parents b4ece73 + 53ade01 commit 94b7ea1

19 files changed

+348
-201
lines changed

‎CONTRIBUTING.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,8 @@ To get started with development, follow the below steps.
5656

5757
_If you're using Windows, run the shell scripts using `GIT bash` - Windows-based bash command line._
5858

59+
**Caution**: If you are using jetBrains DataGrip, don't use the ingegrated console to run the shell scripts (not even with GIT bash configured). It might disconnect from oracle randomly during script-run.
60+
5961
### Clone your fork of utPLSQL git repository
6062

6163
```bash

‎docs/userguide/annotations.md‎

Lines changed: 165 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -4,63 +4,101 @@ Annotations are used to configure tests and suites in a declarative way similar
44
No configuration files or tables are needed. The annotation names are based on popular testing frameworks such as jUnit.
55
The framework runner searches for all the suitable annotated packages, automatically configures suites, forms the suite hierarchy, executes it and reports results in specified formats.
66

7-
Annotations are interpreted only in the package specification and are case-insensitive. It is recommended however, to use lower-case annotations as described in this documentation.
7+
Annotations are interpreted only in the package specification and are case-insensitive. We strongly recommend using lower-case annotations as described in this documentation.
88

9-
There are two places where annotations may appear:
9+
There are two locations where annotations can be placed:
10+
- Package level annotations can be placed at the very top of the package specification (`--%suite`, `--%suitepath` etc.)
11+
- Procedure level annotations can be placed right before a procedure (`--%test`, `--%beforeall`, `--%beforeeach` etc.)
12+
13+
If procedure level annotation is not placed right before procedure, it is not considered an annotation for procedure.
14+
15+
Example of invalid procedure level annotations
16+
```sql
17+
create or replace package test_pkg is
18+
19+
--%suite(Name of suite)
20+
21+
--%test
22+
-- this single-line comment makes the TEST annotation no longer associated with the procedure
23+
procedure first_test;
24+
25+
--%test
26+
--procedure some_test; /* This TEST annotation is not associated with any procedure*/
27+
28+
--%test(Name of another test)
29+
procedure another_test;
30+
31+
--%test
32+
/**
33+
* this multi-line comment makes the TEST annotation no longer associated with the procedure
34+
*/
35+
procedure yet_another_test;
36+
end test_pkg;
37+
```
38+
Procedure annotations are defined right before the procedure they reference, no empty lines are allowed, no comment lines can exist between annotation and the procedure.
1039

11-
- at the beginning of the package specification (`%suite`, `%suitepath` etc.)
12-
- right before a procedure (`%test`, `%beforeall`, `%beforeeach` etc.)
1340

1441
Package level annotations need to be separated by at least one empty line from the underlying procedure annotations.
1542

16-
Procedure annotations are defined right before the procedure they reference, no empty lines are allowed.
43+
Example of invalid package level annotation.
44+
```sql
45+
create or replace package test_pkg is
46+
--%suite(Name of suite)
47+
--%test
48+
procedure first_test;
49+
end test_pkg;
50+
```
1751

18-
If a package specification contains the `%suite` annotation, it is treated as a test package and is processed by the framework.
52+
If a package specification contains the `--%suite` annotation, it is treated as a test package and is processed by the framework.
1953

20-
Some annotations accept parameters like `%suite`, `%test` and `%displayname`. The parameters for annotations need to be placed in brackets. Values for parameters should be provided without any quotation marks.
54+
Some annotations accept parameters like `--%suite`, `--%test` and `--%displayname`. The parameters for annotations need to be placed in brackets.
55+
Values for parameters should be provided without any quotation marks.
56+
If the parameters are placed without brackets or with incomplete brackets, they will be ignored.
57+
Example: `--%suite(The name of suite without closing bracket`
2158

2259
# <a name="example"></a>Example of an annotated test package
2360

2461
```sql
2562
create or replace package test_pkg is
2663

27-
-- %suite(Name of suite)
28-
-- %suitepath(all.globaltests)
64+
--%suite(Name of suite)
65+
--%suitepath(all.globaltests)
2966

30-
-- %beforeall
67+
--%beforeall
3168
procedure global_setup;
3269

33-
-- %afterall
70+
--%afterall
3471
procedure global_cleanup;
3572

3673
/* Such comments are allowed */
3774

38-
-- %test
39-
-- %displayname(Name of a test)
75+
--%test
76+
--%displayname(Name of a test)
77+
--%throws(-20145,-20146,-20189,-20563)
4078
procedure some_test;
4179

42-
-- %test(Name of another test)
43-
-- %beforetest(setup_another_test)
44-
-- %aftertest(cleanup_another_test)
80+
--%test(Name of another test)
81+
--%beforetest(setup_another_test)
82+
--%aftertest(cleanup_another_test)
4583
procedure another_test;
4684

47-
-- %test
48-
-- %displayname(Name of test)
49-
-- %disabled
85+
--%test
86+
--%displayname(Name of test)
87+
--%disabled
5088
procedure disabled_test;
5189

52-
-- %test(Name of test)
53-
-- %rollback(manual)
90+
--%test(Name of test)
91+
--%rollback(manual)
5492
procedure no_transaction_control_test;
5593

5694
procedure setup_another_test;
5795

5896
procedure cleanup_another_test;
5997

60-
-- %beforeeach
98+
--%beforeeach
6199
procedure test_setup;
62100

63-
-- %aftereach
101+
--%aftereach
64102
procedure test_cleanup;
65103

66104
end test_pkg;
@@ -74,6 +112,7 @@ end test_pkg;
74112
| `%suitepath(<path>)` | Package | Similar to java package. The annotation allows logical grouping of suites into hierarchies. |
75113
| `%displayname(<description>)` | Package/procedure | Human-readable and meaningful description of a suite/test. `%displayname(Name of the suite/test)`. The annotation is provided for flexibility and convenience only. It has exactly the same meaning as `<description>` in `test` and `suite` annotations. If description is provided using both `suite`/`test` and `displayname`, then the one defined as last takes precedence. |
76114
| `%test(<description>)` | Procedure | Denotes that the annotated procedure is a unit test procedure. Optional test description can by provided (see `displayname`). |
115+
| `%throws(<exception_number>[,<exception_number>[,...]])`| Procedure | Denotes that the annotated procedure must throw one of the exception numbers provided. If no valid numbers were provided as annotation parameters the annotation is ignored. Applicable to test procedures only. |
77116
| `%beforeall` | Procedure | Denotes that the annotated procedure should be executed once before all elements of the suite. |
78117
| `%afterall` | Procedure | Denotes that the annotated procedure should be executed once after all elements of the suite. |
79118
| `%beforeeach` | Procedure | Denotes that the annotated procedure should be executed before each `%test` procedure in the suite. |
@@ -103,17 +142,17 @@ The `%suitepath` annotation is used for such grouping. Even though test packages
103142
```sql
104143
create or replace package test_payment_recognition as
105144

106-
-- %suite(Payment recognition tests)
107-
-- %suitepath(payments)
145+
--%suite(Payment recognition tests)
146+
--%suitepath(payments)
108147

109-
-- %test(Recognize payment by policy number)
148+
--%test(Recognize payment by policy number)
110149
procedure test_recognize_by_num;
111150

112-
-- %test
113-
-- %displayname(Recognize payment by payment purpose)
151+
--%test
152+
--%displayname(Recognize payment by payment purpose)
114153
procedure test_recognize_by_purpose;
115154

116-
-- %test(Recognize payment by customer)
155+
--%test(Recognize payment by customer)
117156
procedure test_recognize_by_customer;
118157

119158
end test_payment_recognition;
@@ -123,14 +162,14 @@ And payments set off test package:
123162
```sql
124163
create or replace package test_payment_set_off as
125164

126-
-- %suite(Payment set off tests)
127-
-- %suitepath(payments)
165+
--%suite(Payment set off tests)
166+
--%suitepath(payments)
128167

129-
-- %test(Set off creation test)
168+
--%test(Set off creation test)
130169
procedure test_create_set_off;
131170

132-
-- %test
133-
-- %displayname(Set off annulation test)
171+
--%test
172+
--%displayname(Set off annulation test)
134173
procedure test_annulate_set_off;
135174

136175
end test_payment_set_off;
@@ -143,12 +182,12 @@ An additional advantage of such grouping is the fact that every element level of
143182
```sql
144183
create or replace package payments as
145184

146-
-- %suite(Payments)
185+
--%suite(Payments)
147186

148-
-- %beforeall
187+
--%beforeall
149188
procedure set_common_payments_data;
150189

151-
-- %afterall
190+
--%afterall
152191
procedure reset_common_paymnets_data;
153192

154193
end payments;
@@ -167,7 +206,7 @@ In general, your unit tests should not use transaction control as long as the co
167206
Keeping the transactions uncommitted allows your changes to be isolated and the execution of tests does not impact others who might be using a shared development database.
168207

169208
If you are in a situation where the code you are testing uses transaction control (common case with ETL code), then your tests probably should not use the default automatic transaction control.
170-
In that case use the annotation `-- %rollback(manual)` on the suite level to disable automatic transaction control for the entire suite.
209+
In that case use the annotation `--%rollback(manual)` on the suite level to disable automatic transaction control for the entire suite.
171210
If you are using nested suites, you need to make sure that the entire suite all the way to the root is using manual transaction control.
172211

173212
It is possible with utPLSQL to change the transaction control on individual suites or tests that are part of complex suite.
@@ -235,3 +274,91 @@ Example:
235274
```sql
236275
exec ut_runner.purge_cache('HR', 'PACKAGE');
237276
```
277+
278+
# Throws annotation
279+
280+
The `--%throws` annotation allows you to specify a list of exception numbers that can be expected from a test.
281+
282+
If `--%throws(-20001,-20002)` is specified and no exception is raised or the exception raised is not on the list of provided exception numbers, the test is marked as failed.
283+
284+
The framework ignores bad arguments. `--%throws(7894562, operaqk, -=1, -20496, pow74d, posdfk3)` will be interpreted as `--%throws(-20496)`.
285+
The annotation is ignored, when no valid arguments are provided `--%throws()`,`--%throws`, `--%throws(abe, 723pf)`.
286+
287+
Example:
288+
```sql
289+
create or replace package example_pgk as
290+
291+
--%suite(Example Throws Annotation)
292+
293+
--%test(Throws one of the listed exceptions)
294+
--%throws(-20145,-20146, -20189 ,-20563)
295+
procedure raised_one_listed_exception;
296+
297+
--%test(Throws different exception than expected)
298+
--%throws(-20144)
299+
procedure raised_different_exception;
300+
301+
--%test(Throws different exception than listed)
302+
--%throws(-20144,-00001,-20145)
303+
procedure raised_unlisted_exception;
304+
305+
--%test(Gives failure when an exception is expected and nothing is thrown)
306+
--%throws(-20459, -20136, -20145)
307+
procedure nothing_thrown;
308+
309+
end;
310+
/
311+
create or replace package body example_pgk is
312+
procedure raised_one_listed_exception is
313+
begin
314+
raise_application_error(-20189, 'Test error');
315+
end;
316+
317+
procedure raised_different_exception is
318+
begin
319+
raise_application_error(-20143, 'Test error');
320+
end;
321+
322+
procedure raised_unlisted_exception is
323+
begin
324+
raise_application_error(-20143, 'Test error');
325+
end;
326+
327+
procedure nothing_thrown is
328+
begin
329+
ut.expect(1).to_equal(1);
330+
end;
331+
end;
332+
/
333+
334+
exec ut.run('example_pgk');
335+
```
336+
337+
Running the test will give report:
338+
```
339+
Example Throws Annotation
340+
Throws one of the listed exceptions [.018 sec]
341+
Throws different exception than expected [.008 sec] (FAILED - 1)
342+
Throws different exception than listed [.007 sec] (FAILED - 2)
343+
Gives failure when an exception is expected and nothing is thrown [.002 sec] (FAILED - 3)
344+
345+
Failures:
346+
347+
1) raised_different_exception
348+
Actual: -20143 was expected to equal: -20144
349+
ORA-20143: Test error
350+
ORA-06512: at "UT3.EXAMPLE_PGK", line 9
351+
ORA-06512: at line 6
352+
353+
2) raised_unlisted_exception
354+
Actual: -20143 was expected to be one of: (-20144, -1, -20145)
355+
ORA-20143: Test error
356+
ORA-06512: at "UT3.EXAMPLE_PGK", line 14
357+
ORA-06512: at line 6
358+
359+
3) nothing_thrown
360+
Expected one of exceptions (-20459, -20136, -20145) but nothing was raised.
361+
362+
Finished in .038692 seconds
363+
4 tests, 3 failed, 0 errored, 0 disabled, 0 warning(s)
364+
```

‎docs/userguide/exception-reporting.md‎

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2,19 +2,19 @@
22

33
The utPLSQL is responsible for handling exceptions wherever they occur in the test run. utPLSQL is trapping most of the exceptions so that the test execution is not affected by individual tests or test packages throwing an exception.
44
The framework provides a full stacktrace for every exception that was thrown. The stacktrace is clean and does not include any utPLSQL library calls in it.
5-
To achieve rerunability, the ORA-04068, ORA-04061 exceptions are not handled and test execution will be interrupted if such exception is encountered. This is because of how Oracle behaves on those exceptions.
5+
To achieve rerunability, the package state invalidation exceptions (ORA-04068, ORA-04061) are not handled and test execution will be interrupted if such exceptions are encountered. This is because of how Oracle behaves on those exceptions.
66

77
Test execution can fail for different reasons. The failures on different exceptions are handled as follows:
8-
* A test package without body - each `%test` is reported as failed with exception, nothing is executed
9-
* A test package with _invalid body_ - each `%test` is reported as failed with exception, nothing is executed
8+
* A test package without body - each `--%test` is reported as failed with exception, nothing is executed
9+
* A test package with _invalid body_ - each `--%test` is reported as failed with exception, nothing is executed
1010
* A test package with _invalid spec_ - package is not considered a valid unit test package and is excluded from execution. When trying to run a test package with invalid spec explicitly, exception is raised. Only valid specifications are parsed for annotations
11-
* A test package that is raising an exception in `%beforeall` - each `%test` is reported as failed with exception, `%test`, `%beforeeach`, `%beforetest`, `%aftertest` and `%aftereach` are not executed. `%afterall` is executed to allow cleanup of whatever was done in `%beforeall`
12-
* A test package that is raising an exception in `%beforeeach` - each `%test` is reported as failed with exception, `%test`, `%beforetest` and `%aftertest` is not executed. The `%aftereach` and `%afterall` blocks are getting executed to allow cleanup of whatever was done in `%before...` blocks
13-
* A test package that is raising an exception in `%beforetest` - the `%test` is reported as failed with exception, `%test` is not executed. The `%aftertest`, `%aftereach` and `%afterall` blocks are getting executed to allow cleanup of whatever was done in `%before...` blocks
14-
* A test package that is raising an exception in `%test` - the `%test` is reported as failed with exception. The execution of other blocks continues normally
15-
* A test package that is raising an exception in `%aftertest` - the `%test` is reported as failed with exception. The execution of other blocks continues normally
16-
* A test package that is raising an exception in `%aftereach` - each `%test` is reported as failed with exception.
17-
* A test package that is raising an exception in `%afterall` - all blocks of the package are executed, as the `%afterall` is the last step of package execution. Exception in `%afterall` is not affecting test results. A warning with exception stacktrace is displayed in the summary
11+
* A test package that is raising an exception in `--%beforeall` - each `--%test` is reported as failed with exception, `--%test`, `--%beforeeach`, `--%beforetest`, `--%aftertest` and `--%aftereach` are not executed. `--%afterall` is executed to allow cleanup of whatever was done in `--%beforeall`
12+
* A test package that is raising an exception in `--%beforeeach` - each `--%test` is reported as failed with exception, `--%test`, `--%beforetest` and `--%aftertest` is not executed. The `--%aftereach` and `--%afterall` blocks are getting executed to allow cleanup of whatever was done in `--%before...` blocks
13+
* A test package that is raising an exception in `--%beforetest` - the `--%test` is reported as failed with exception, `--%test` is not executed. The `--%aftertest`, `--%aftereach` and `--%afterall` blocks are getting executed to allow cleanup of whatever was done in `--%before...` blocks
14+
* A test package that is raising an exception in `--%test` - the `--%test` is reported as failed with exception. The execution of other blocks continues normally
15+
* A test package that is raising an exception in `--%aftertest` - the `--%test` is reported as failed with exception. The execution of other blocks continues normally
16+
* A test package that is raising an exception in `--%aftereach` - each `--%test` is reported as failed with exception.
17+
* A test package that is raising an exception in `--%afterall` - all blocks of the package are executed, as the `--%afterall` is the last step of package execution. Exception in `--%afterall` is not affecting test results. A warning with exception stacktrace is displayed in the summary
1818

1919

2020
Example of reporting with exception thrown in `%beforetest`:

0 commit comments

Comments
 (0)