Skip to content

Commit fa64751

Browse files
jgebaljgebal
committed
Added ability to exclude items by tags.
Resolves #983
1 parent c035df7 commit fa64751

File tree

7 files changed

+371
-208
lines changed

7 files changed

+371
-208
lines changed

‎docs/userguide/annotations.md‎

Lines changed: 41 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -1225,11 +1225,11 @@ Finished in .035261 seconds
12251225

12261226
### Tags
12271227

1228-
Tag is a label attached to the test or a suite path. It is used for identification and execution a group of tests / suites that share same tag.
1228+
Tag is a label attached to the test or a suite. It is used for identification and execution a group of tests / suites that share same tag.
12291229

1230-
It allows us to group a tests / suites using a various categorization and place a test / suite in multiple buckets. Same tests can be group with other tests based on the functionality , frequency, type of output etc.
1230+
It allows for grouping of tests / suites using various categorization and place tests / suites in multiple buckets. Same tests can be grouped with other tests based on the functionality , frequency, type of output etc.
12311231

1232-
e.q.
1232+
e.g.
12331233

12341234
```sql
12351235
--%tags(batch,daily,csv)
@@ -1238,29 +1238,31 @@ e.q.
12381238
or
12391239

12401240
```sql
1241-
--%tags(api,online,json)
1241+
--%tags(online,json)
1242+
--%tags(api)
12421243
```
12431244

1245+
Tags are defined as a comma separated list within the `--%tags` annotation.
12441246

1247+
When executing a test run with tag filter applied, framework will find all tests associated with given tags and execute them.
1248+
Framework applies `OR` logic to all specified tags so any test / suite that matches at least one tag will be included in the test run.
12451249

1246-
Tags are defined as a coma separated list. When executing a test run with tag filter applied, framework will find all tests associated with given tags and execute them. Framework applies `OR` logic when resolving a tags so any tests / suites that match at least one tag will be included in the test run.
1250+
When a suite/context is tagged all of its children will automatically inherit a tag and get executed along with the parent. Parent suite tests are not executed, but a suitepath hierarchy is kept.
12471251

1248-
When a suite gets tagged all of its children will automatically inherit a tag and get executed along the parent. Parent suit tests are not executed. but a suitepath hierarchy is kept.
1249-
1250-
Sample tag package.
12511252

1253+
Sample test suite package with tags.
12521254
```sql
12531255
create or replace package ut_sample_test IS
12541256

12551257
--%suite(Sample Test Suite)
1256-
--%tag(suite1)
1258+
--%tags(api)
12571259

12581260
--%test(Compare Ref Cursors)
1259-
--%tag(test1,sample)
1261+
--%tags(complex,fast)
12601262
procedure ut_refcursors1;
12611263

12621264
--%test(Run equality test)
1263-
--%tag(test2,sample)
1265+
--%tags(simple,fast)
12641266
procedure ut_test;
12651267

12661268
end ut_sample_test;
@@ -1290,28 +1292,42 @@ end ut_sample_test;
12901292
Execution of the test is done by using a parameter `a_tags`
12911293

12921294
```sql
1293-
select * from table(ut.run(a_path => 'ut_sample_test',a_tags => 'suite1'));
1294-
select * from table(ut.run(a_tags => 'test1,test2'));
1295-
select * from table(ut.run(a_tags => 'sample'));
1295+
select * from table(ut.run(a_path => 'ut_sample_test',a_tags => 'api'));
1296+
```
1297+
The above call will execute all tests from `ut_sample_test` package as the whole suite is tagged with `api`
12961298

1297-
begin
1298-
ut.run(a_path => 'ut_sample_test',a_tags => 'suite1');
1299-
end;
1300-
/
1299+
```sql
1300+
select * from table(ut.run(a_tags => 'complex'));
1301+
```
1302+
The above call will execute only the `ut_sample_test.ut_refcursors1` test, as only the test `ut_refcursors1` is tagged with `complex`
13011303

1302-
exec ut.run('ut_sample_test', a_tags => 'sample');
1304+
```sql
1305+
select * from table(ut.run(a_tags => 'fast'));
13031306
```
1307+
The above call will execute both `ut_sample_test.ut_refcursors1` and `ut_sample_test.ut_test` tests, as both tests are tagged with `fast`
1308+
1309+
#### Excluding tests/suites by tags
13041310

1311+
It is possible to exclude parts of test suites with tags.
1312+
In order to do so, prefix the tag name to exclude with a `-` (dash) sign when invoking the test run.
13051313

1314+
Examples (based on above sample test suite)
13061315

1307-
Tags should adhere to following rules:
1316+
```sql
1317+
select * from table(ut.run(a_tags => 'api,fast,-complex'));
1318+
```
1319+
The above call will execute all suites/contexts/tests that are marked with any of tags `api` or `fast` except those suites/contexts/tests that are marked as `complex`
1320+
Given the above example package `ut_sample_test`, only `ut_sample_test.ut_test` will be executed.
13081321

1309-
- tags are case sensitive
1310-
- tags cannot be an empty string
1311-
- tags cannot contain spaces e.g. to create a multi-word `tag` please use underscores,dashes, dots etc. e.g. `test_of_batch`
1312-
- tags with empty spaces will be ignored during execution
1313-
- tags can contain special characters
1322+
**Note:**
1323+
Tags must follow the below naming convention:
13141324

1325+
- tag is case sensitive
1326+
- tag can contain special characters like `$#/\?-!` etc.
1327+
- tag cannot be an empty string
1328+
- tag cannot start with a dash e.g. `-some-stuff` is **not** a valid tag
1329+
- tag cannot contain spaces e.g. `test of batch`. To create a multi-word tag use underscores or dashes e.g. `test_of_batch`, `test-of-batch`
1330+
- leading and trailing spaces are ignored in tag name e.g. `--%tags( tag1 , tag2 )` becomes `tag1` and `tag2` tag names
13151331

13161332

13171333
### Suitepath

‎source/core/ut_suite_builder.pkb‎

Lines changed: 18 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -321,7 +321,8 @@ create or replace package body ut_suite_builder is
321321
a_procedure_name t_object_name := null
322322
) is
323323
l_annotation_pos binary_integer;
324-
l_tag_list ut_varchar2_list := ut_varchar2_list();
324+
l_tags_list ut_varchar2_list := ut_varchar2_list();
325+
l_tag_items ut_varchar2_list;
325326
begin
326327
l_annotation_pos := a_tags_ann_text.first;
327328
while l_annotation_pos is not null loop
@@ -331,14 +332,25 @@ create or replace package body ut_suite_builder is
331332
|| get_object_reference( a_suite, a_procedure_name, l_annotation_pos )
332333
);
333334
else
334-
l_tag_list := l_tag_list multiset union distinct ut_utils.trim_list_elements(
335-
ut_utils.string_to_table(a_tags_ann_text(l_annotation_pos),',')
336-
);
335+
l_tag_items := ut_utils.trim_list_elements(ut_utils.string_to_table(a_tags_ann_text(l_annotation_pos),','));
336+
if l_tag_items is not empty then
337+
for i in 1 .. l_tag_items.count loop
338+
if regexp_like(l_tag_items(i),'^[^-](\S)+$') then
339+
l_tags_list.extend();
340+
l_tags_list(l_tags_list.last) := l_tag_items(i);
341+
else
342+
a_suite.put_warning(
343+
'Invalid value "'||l_tag_items(i)||'" for "--%tags" annotation. See documentation for details on valid tag values. Annotation value ignored.'
344+
|| get_object_reference( a_suite, a_procedure_name, l_annotation_pos )
345+
);
346+
end if;
347+
end loop;
348+
end if;
337349
end if;
338350
l_annotation_pos := a_tags_ann_text.next(l_annotation_pos);
339351
end loop;
340-
--remove empty strings from table list e.g. tag1,,tag2 and conver to rows
341-
a_list := ut_utils.convert_collection( ut_utils.filter_list(l_tag_list,ut_utils.gc_word_no_space) );
352+
--remove empty strings from table list e.g. tag1,,tag2 and convert to rows
353+
a_list := ut_utils.convert_collection( ut_utils.filter_list(set(l_tags_list),ut_utils.gc_word_no_space) );
342354
end;
343355

344356
procedure set_seq_no(

‎source/core/ut_suite_cache_manager.pkb‎

Lines changed: 29 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -30,8 +30,8 @@ create or replace package body ut_suite_cache_manager is
3030
and ( {:path:}
3131
and {:object_name:}
3232
and {:procedure_name:}
33+
)
3334
)
34-
)
3535
),
3636
{:tags:}
3737
suitepaths as (
@@ -106,8 +106,8 @@ create or replace package body ut_suite_cache_manager is
106106
function get_path_sql(a_path in varchar2) return varchar2 is
107107
begin
108108
return case when a_path is not null then q'[
109-
:l_path||'.' like c.path || '.%' /*all children and self*/
110-
or ( c.path||'.' like :l_path || '.%' --all parents
109+
:l_path||'.' like c.path || '.%' /*all parents and self*/
110+
or ( c.path||'.' like :l_path || '.%' /*all children and self*/
111111
]'
112112
else ' :l_path is null and ( :l_path is null ' end;
113113
end;
@@ -129,22 +129,40 @@ create or replace package body ut_suite_cache_manager is
129129
function get_tags_sql(a_tags_count in integer) return varchar2 is
130130
begin
131131
return case when a_tags_count > 0 then
132-
q'[filter_tags as (
132+
q'[included_tags as (
133133
select c.obj.path as path
134134
from suite_items c
135-
where c.obj.tags multiset intersect :a_tag_list is not empty
136-
),
135+
where exists (
136+
select * from table(c.obj.tags)
137+
intersect
138+
select * from table(:a_tag_list) where column_value not like '-%'
139+
)
140+
or 0 = (select count(*) from table(:a_tag_list) where column_value not like '-%')
141+
),
142+
excluded_tags as (
143+
select c.obj.path as path
144+
from suite_items c
145+
where exists (
146+
select * from table(c.obj.tags)
147+
intersect
148+
select ltrim(column_value,'-') from table(:a_tag_list) where column_value like '-%'
149+
)
150+
),
137151
suite_items_tags as (
138152
select c.*
139153
from suite_items c
140154
where exists (
141-
select 1 from filter_tags t
142-
where t.path||'.' like c.obj.path || '.%' /*all children and self*/
143-
or c.obj.path||'.' like t.path || '.%' --all parents
155+
select 1 from included_tags t
156+
where t.path||'.' like c.obj.path || '.%' /*all parents and self*/
157+
or c.obj.path||'.' like t.path || '.%' /*all children and self*/
158+
)
159+
and not exists (
160+
select 1 from excluded_tags t
161+
where c.obj.path||'.' like t.path || '.%' /*all children and self*/
144162
)
145163
),]'
146164
else
147-
q'[dummy as (select 'x' from dual where :a_tag_list is null ),]'
165+
q'[dummy as (select 'x' from dual where :a_tag_list is null and :a_tag_list is null and :a_tag_list is null),]'
148166
end;
149167
end;
150168

@@ -216,7 +234,7 @@ create or replace package body ut_suite_cache_manager is
216234

217235
execute immediate l_sql
218236
bulk collect into l_results
219-
using upper(l_object_owner), l_path, l_path, upper(a_object_name), upper(a_procedure_name), l_tags, a_random_seed;
237+
using upper(l_object_owner), l_path, l_path, upper(a_object_name), upper(a_procedure_name), l_tags, l_tags, l_tags, a_random_seed;
220238
return l_results;
221239
end;
222240

‎test/ut3_tester/core/test_suite_builder.pkb‎

Lines changed: 61 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1431,7 +1431,67 @@ create or replace package body test_suite_builder is
14311431
'%</UT_LOGICAL_SUITE>%'
14321432
);
14331433

1434-
end;
1434+
end;
1435+
1436+
procedure test_spaces_in_tag is
1437+
l_actual clob;
1438+
l_annotations ut3.ut_annotations;
1439+
begin
1440+
--Arrange
1441+
l_annotations := ut3.ut_annotations(
1442+
ut3.ut_annotation(2, 'suite','testsuite', null),
1443+
ut3.ut_annotation(3, 'tags',' good_tag , bad tag , good-tag ', null),
1444+
ut3.ut_annotation(8, 'test','Some test', 'test_procedure'),
1445+
ut3.ut_annotation(9, 'tags',' good_tag , bad tag , good-tag ', 'test_procedure')
1446+
);
1447+
--Act
1448+
l_actual := invoke_builder_for_annotations(l_annotations, 'SOME_PACKAGE');
1449+
--Assert
1450+
ut.expect(l_actual).to_be_like(
1451+
'%<TAGS><VARCHAR2>good_tag</VARCHAR2><VARCHAR2>good-tag</VARCHAR2></TAGS>%'||
1452+
'<TAGS><VARCHAR2>good_tag</VARCHAR2><VARCHAR2>good-tag</VARCHAR2></TAGS>%'
1453+
);
1454+
ut.expect(l_actual).to_be_like(
1455+
'%<WARNINGS><VARCHAR2>Invalid value &quot;bad tag&quot; for &quot;--%tags&quot; annotation.'||
1456+
' See documentation for details on valid tag values. Annotation value ignored.
1457+
at package &quot;UT3_TESTER.SOME_PACKAGE&quot;, line 3</VARCHAR2><VARCHAR2>%'
1458+
);
1459+
ut.expect(l_actual).to_be_like(
1460+
'%<VARCHAR2>Invalid value &quot;bad tag&quot; for &quot;--%tags&quot; annotation.'||
1461+
' See documentation for details on valid tag values. Annotation value ignored.
1462+
at package &quot;UT3_TESTER.SOME_PACKAGE.TEST_PROCEDURE&quot;, line 9</VARCHAR2></WARNINGS>%'
1463+
);
1464+
end;
1465+
1466+
procedure test_minus_in_tag is
1467+
l_actual clob;
1468+
l_annotations ut3.ut_annotations;
1469+
begin
1470+
--Arrange
1471+
l_annotations := ut3.ut_annotations(
1472+
ut3.ut_annotation(2, 'suite','testsuite', null),
1473+
ut3.ut_annotation(3, 'tags',' good_tag , -invalid_tag , good-tag ', null),
1474+
ut3.ut_annotation(8, 'test','Some test', 'test_procedure'),
1475+
ut3.ut_annotation(9, 'tags',' good_tag , -invalid_tag , good-tag ', 'test_procedure')
1476+
);
1477+
--Act
1478+
l_actual := invoke_builder_for_annotations(l_annotations, 'SOME_PACKAGE');
1479+
--Assert
1480+
ut.expect(l_actual).to_be_like(
1481+
'%<TAGS><VARCHAR2>good_tag</VARCHAR2><VARCHAR2>good-tag</VARCHAR2></TAGS>%'||
1482+
'<TAGS><VARCHAR2>good_tag</VARCHAR2><VARCHAR2>good-tag</VARCHAR2></TAGS>%'
1483+
);
1484+
ut.expect(l_actual).to_be_like(
1485+
'%<WARNINGS><VARCHAR2>Invalid value &quot;-invalid_tag&quot; for &quot;--%tags&quot; annotation.'||
1486+
' See documentation for details on valid tag values. Annotation value ignored.
1487+
at package &quot;UT3_TESTER.SOME_PACKAGE&quot;, line 3</VARCHAR2><VARCHAR2>%'
1488+
);
1489+
ut.expect(l_actual).to_be_like(
1490+
'%<VARCHAR2>Invalid value &quot;-invalid_tag&quot; for &quot;--%tags&quot; annotation.'||
1491+
' See documentation for details on valid tag values. Annotation value ignored.
1492+
at package &quot;UT3_TESTER.SOME_PACKAGE.TEST_PROCEDURE&quot;, line 9</VARCHAR2></WARNINGS>%'
1493+
);
1494+
end;
14351495

14361496
end test_suite_builder;
14371497
/

0 commit comments

Comments
 (0)