Skip to content

Commit bd3291b

Browse files
jgebaljgebal
authored
Merge pull request #878 from utPLSQL/feature/random_order
Added support for random order of test execution.
2 parents b734f86 + 1c0f892 commit bd3291b

File tree

19 files changed

+448
-119
lines changed

19 files changed

+448
-119
lines changed

‎development/cleanup.sh‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ git rev-parse && cd "$(git rev-parse --show-cdup)"
77

88
"${SQLCLI}" sys/${ORACLE_PWD}@//${CONNECTION_STR} AS SYSDBA <<-SQL
99
set echo on
10+
set serveroutput on
1011
begin
1112
for x in (
1213
select * from dba_objects
@@ -25,12 +26,12 @@ drop user ${UT3_USER} cascade;
2526
2627
begin
2728
for i in (
28-
select decode(owner,'PUBLIC','drop public synonym "','drop synonym "'||owner||'"."')|| synonym_name ||'"' drop_orphaned_synonym from dba_synonyms a
29+
select decode(owner,'PUBLIC','drop public synonym "','drop synonym "'||owner||'"."')|| synonym_name ||'"' drop_orphaned_synonym, owner||'.'||synonym_name syn from dba_synonyms a
2930
where not exists (select 1 from dba_objects b where (a.table_name=b.object_name and a.table_owner=b.owner or b.owner='SYS' and a.table_owner=b.object_name) )
3031
and a.table_owner not in ('SYS','SYSTEM')
3132
) loop
32-
dbms_output.put_line(i.drop_orphaned_synonym);
3333
execute immediate i.drop_orphaned_synonym;
34+
dbms_output.put_line('synonym '||i.syn||' dropped');
3435
end loop;
3536
end;
3637
/

‎docs/userguide/running-unit-tests.md‎

Lines changed: 90 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,7 @@ The examples below illustrate different ways and options to invoke `ut.run` proc
4747

4848
```sql
4949
alter session set current_schema=hr;
50+
set serveroutput on
5051
begin
5152
ut.run();
5253
end;
@@ -55,6 +56,7 @@ Executes all tests in current schema (_HR_).
5556

5657

5758
```sql
59+
set serveroutput on
5860
begin
5961
ut.run('HR');
6062
end;
@@ -63,6 +65,7 @@ Executes all tests in specified schema (_HR_).
6365

6466

6567
```sql
68+
set serveroutput on
6669
begin
6770
ut.run('hr:com.my_org.my_project');
6871
end;
@@ -73,6 +76,7 @@ Check the [annotations documentation](annotations.md) to find out about suitepat
7376

7477

7578
```sql
79+
set serveroutput on
7680
begin
7781
ut.run('hr.test_apply_bonus');
7882
end;
@@ -81,6 +85,7 @@ Executes all tests from package _hr.test_apply_bonus_.
8185

8286

8387
```sql
88+
set serveroutput on
8489
begin
8590
ut.run('hr.test_apply_bonus.bonus_cannot_be_negative');
8691
end;
@@ -89,13 +94,15 @@ Executes single test procedure _hr.test_apply_bonus.bonus_cannot_be_negative_.
8994

9095

9196
```sql
97+
set serveroutput on
9298
begin
9399
ut.run(ut_varchar2_list('hr.test_apply_bonus','cust'));
94100
end;
95101
```
96102
Executes all tests from package _hr.test_apply_bonus_ and all tests from schema _cust_.
97103

98104
```sql
105+
set serveroutput on
99106
begin
100107
ut.run(ut_varchar2_list('hr.test_apply_bonus,cust)');
101108
end;
@@ -104,6 +111,7 @@ end;
104111
Executes all tests from package _hr.test_apply_bonus_ and all tests from schema _cust_.
105112

106113
```sql
114+
set serveroutput on
107115
begin
108116
ut.run('hr.test_apply_bonus,cust');
109117
end;
@@ -124,6 +132,7 @@ The `ut.run` procedures and functions accept `a_reporter` attribute that defines
124132
You can execute any set of tests with any of the predefined reporters.
125133

126134
```sql
135+
set serveroutput on
127136
begin
128137
ut.run('hr.test_apply_bonus', ut_junit_reporter());
129138
end;
@@ -133,33 +142,17 @@ Executes all tests from package _HR.TEST_APPLY_BONUS_ and provide outputs to DBM
133142

134143
For details on build-in reporters look at [reporters documentation](reporters.md).
135144

136-
## Keeping uncommited data after test-run
137-
138-
utPLSQL by default runs tests in autonomous transaction and performs automatic rollback to assure that tests do not impact one-another and do not have impact on the current session in your IDE.
139-
140-
If you would like to keep your uncommited data persisted after running tests, you can do so by using `a_force_manual_rollback` flag.
141-
Setting this flag to true has following side-effects:
142-
143-
- test execution is done in current transaction - if while running tests commit or rollback is issued your current session data will get commited too.
144-
- automatic rollback is forced to be disabled in test-run even if it was explicitly enabled by using annotation `--%rollback(manual)
145-
146-
Example invocation:
147-
```sql
148-
begin
149-
ut.run('hr.test_apply_bonus', a_force_manual_rollback => true);
150-
end;
151-
```
152-
153-
154-
This option is not anvailable when running tests using `ut.run` as a table function.
155-
156145
## ut.run functions
157146

158147
The `ut.run` functions provide exactly the same functionality as the `ut.run` procedures.
159148
You may use the same sets of parameters with both functions and procedures.
160149
The only difference is the output of the results.
161150
Functions provide output as a pipelined stream and therefore need to be executed as select statements.
162151

152+
**Note:**
153+
>When running tests with `ut.run` functions, whole test run is executed as autonomous transaction.
154+
At the end of the run, the transaction is automatically rolled-back and all uncommitted changes are reverted.
155+
163156
Example.
164157
```sql
165158
select * from table(ut.run('hr.test_apply_bonus', ut_junit_reporter()));
@@ -180,7 +173,83 @@ The concept is pretty simple.
180173
- as a separate thread, start `ut_runner.run` and pass reporters with previously defined output_ids.
181174
- for each reporter start a separate thread and read outputs from the `ut_output_buffer.get_lines` table function by providing the output_id defined in the main thread.
182175

183-
# Reports characterset encoding
176+
# Order of test execution
177+
178+
## Default order
179+
180+
When unit tests are executed without random order, they are ordered by:
181+
- schema name
182+
- suite path or test package name if `--%suitepath` was not specified for that package
183+
- `--%test` line number in package
184+
185+
## Random order
186+
187+
You can force a test run to execute tests in random order by providing one of options to `ut.run`:
188+
- `a_random_test_order` - true/false for procedures and 1/0 for functions
189+
- `a_random_test_order_seed` - positive number in range of 1 .. 1 000 000 000
190+
191+
When tests are executed with random order, randomization is applied to single level of suitepath hierarchy tree.
192+
This is needed to maintain visibility and accessibility of common setup/cleanup `beforeall`/`afterall` in tests.
193+
194+
Example:
195+
```sql
196+
set serveroutput on
197+
begin
198+
ut.run('hr.test_apply_bonus', a_random_test_order => true);
199+
end;
200+
```
201+
202+
```sql
203+
select * from table(ut.run('hr.test_apply_bonus', a_random_test_order => 1));
204+
```
205+
206+
When running with random order, the default report (`ut_documentation_reporter`) will include information about the random test run seed.
207+
Example output:
208+
```
209+
...
210+
Finished in .12982 seconds
211+
35 tests, 0 failed, 0 errored, 1 disabled, 0 warning(s)
212+
Tests were executed with random order seed '302980531'.
213+
```
214+
215+
If you want to re-run tests using previously generated seed, you may do so by running them with parameter `a_random_test_order_seed`
216+
Example:
217+
```sql
218+
set serveroutput on
219+
begin
220+
ut.run('hr.test_apply_bonus', a_random_test_order_seed => 302980531);
221+
end;
222+
```
223+
224+
```sql
225+
select * from table(ut.run('hr.test_apply_bonus', a_random_test_order_seed => 302980531));
226+
```
227+
228+
**Note**
229+
>Random order seed must be a positive number within range of 1 .. 1 000 000 000.
230+
231+
# Keeping uncommitted data after test-run
232+
233+
utPLSQL by default runs tests in autonomous transaction and performs automatic rollback to assure that tests do not impact one-another and do not have impact on the current session in your IDE.
234+
235+
If you would like to keep your uncommitted data persisted after running tests, you can do so by using `a_force_manual_rollback` flag.
236+
Setting this flag to true has following side-effects:
237+
238+
- test execution is done in current transaction - if while running tests commit or rollback is issued your current session data will get commited too.
239+
- automatic rollback is forced to be disabled in test-run even if it was explicitly enabled by using annotation `--%rollback(manual)
240+
241+
Example invocation:
242+
```sql
243+
set serveroutput on
244+
begin
245+
ut.run('hr.test_apply_bonus', a_force_manual_rollback => true);
246+
end;
247+
```
248+
249+
**Note:**
250+
>This option is not available when running tests using `ut.run` as a table function.
251+
252+
# Reports character-set encoding
184253

185254
To get properly encoded reports, when running utPLSQL with HTML/XML reports on data containing national characters you need to provide your client character set when calling `ut.run` functions and procedures.
186255

0 commit comments

Comments
 (0)