Various PR fixe

utPLSQL · jgebal · Apr 28, 2023 · Mar 28, 2023 · Mar 28, 2023 · Mar 29, 2023
commit 077fdb11316cda8b99b9ccc909a1c9cb3de57267
diff --git a/docs/userguide/annotations.md b/docs/userguide/annotations.md
@@ -1616,7 +1616,7 @@ or
 
 Tags are defined as a comma separated list within the `--%tags` annotation. 
 
-When a suite/context is tagged, all of its children will automatically inherit the tag and get executed along with the parent, unless they are excluded by tag expression.
+When a suite/context is tagged, all of its children will automatically inherit the tag and get executed along with the parent, unless they are excluded explicitly at runtime with a negated tag expression.
 Parent suite tests are not executed, but a suitepath hierarchy is kept.
 
 Sample test suite package with tags.
@@ -1657,52 +1657,6 @@ create or replace package body ut_sample_test is
 end ut_sample_test;
 /
 ```
-
-#### Tag Expressions
-
-Tag expressions are boolean expressions with the operators !, & and |. In addition, ( and ) can be used to adjust for operator precedence.
-
-| Operator | Meaning |
-| -------- | --------|
-| !        | not     |
-| &        | and     |
-| \|        | or      |
-
-If you are tagging your tests across multiple dimensions, tag expressions help you to select which tests to execute. When tagging by test type (e.g., micro, integration, end-to-end) and feature (e.g., product, catalog, shipping), the following tag expressions can be useful.
-
-
-| Tag Expression | Selection |
-| -------- | --------|
-| product        | all tests for product     |
-| catalog \| shipping        | all tests for catalog plus all tests for shipping     |
-| catalog & shipping       | all tests for the intersection between catalog and shipping     |
-| product & !end-to-end | all tests for product, but not the end-to-end tests |
-| (micro \| integration) & (product \| shipping) | all micro or integration tests for product or shipping |
-
-
-Execution of the test is done by using the parameter `a_tags` with tag expressions
-
-
-```sql linenums="1"
-select * from table(ut.run(a_tags => 'fast|!complex'));
-```
-The above call will execute all tests from `ut_sample_test` package as the whole suite is tagged with `api` because a suite meet expression condition.
-
-```sql linenums="1"
-select * from table(ut.run(a_path => 'ut_sample_test',a_tags => 'api'));
-```
-The above call will execute all tests from `ut_sample_test` package as the whole suite is tagged with `api`
-
-```sql linenums="1"
-select * from table(ut.run(a_tags => 'complex'));
-```
-The above call will execute only the `ut_sample_test.ut_refcursors1` test, as only the test `ut_refcursors1` is tagged with `complex`
-
-```sql linenums="1"
-select * from table(ut.run(a_tags => 'fast'));
-```
-The above call will execute both `ut_sample_test.ut_refcursors1` and `ut_sample_test.ut_test` tests, as both tests are tagged with `fast`
-
 #### Tag naming convention
 
 Tags must follow the below naming convention:
@@ -1715,45 +1669,11 @@ Tags must follow the below naming convention:
   - vertical bar (|)
   - exclamation point (!)
 - tag cannot be null or blank
-- tag cannot contain whitespace
 - tag cannot start with a dash, e.g. `-some-stuff` is **not** a valid tag
 - tag cannot contain spaces, e.g. `test of batch`. To create a multi-word tag use underscores or dashes, e.g. `test_of_batch`, `test-of-batch`
 - leading and trailing spaces are ignored in tag name, e.g. `--%tags(  tag1  ,   tag2  )` becomes `tag1` and `tag2` tag names
 
 
-#### Excluding tests/suites by tags
-
-It is possible to exclude parts of test suites with tags.
-In order to do so, prefix the tag name to exclude with a `!` (exclamation) sign when invoking the test run which is equivalent of `-` (dash) in legacy notation.
-Examples (based on above sample test suite)
-
-```sql linenums="1"
-select * from table(ut.run(a_tags => '(api|fast)&!complex'));
-```
-
-which is equivalent of legacy calling:
-
-```sql linenums="1"
-select * from table(ut.run(a_tags => 'api,fast,-complex'));
-```
-
-or 
-
-```sql linenums="1"
-select * from table(ut.run(a_tags => '(api|fast)&(!complex&!test1)'));
-```
-
-which is equivalent of legacy calling:
-
-```sql linenums="1"
-select * from table(ut.run(a_tags => 'api,fast,-complex,-test1'));
-```
-
-The above call will execute all suites/contexts/tests that are marked with any of tags `api` or `fast` except those suites/contexts/tests that are marked as `complex`.  
-Given the above example package `ut_sample_test`, only `ut_sample_test.ut_test` will be executed.  
-
-
-
 ### Suitepath
 
 It is very likely that the application for which you are going to introduce tests consists of many different packages, procedures and functions.

diff --git a/docs/userguide/running-unit-tests.md b/docs/userguide/running-unit-tests.md
@@ -319,23 +319,85 @@ select * from table(ut.run('hr.test_apply_bonus', a_random_test_order_seed => 30
 
 In addition to the path, you can filter the tests to be run by specifying tags. Tags are defined in the test / context / suite with the `--%tags`-annotation ([Read more](annotations.md#tags)).  
 Multiple tags are separated by comma. 
-The framework applies `OR` logic to all specified tags so any test / suite that matches at least one tag will be included in the test run.
+
+
+### Tag Expressions
+
+Tag expressions are boolean expressions with the operators !, & and |. In addition, ( and ) can be used to adjust for operator precedence.
+
+| Operator | Meaning |
+| -------- | --------|
+| !        | not     |
+| &        | and     |
+| \|        | or      |
+
+If you are tagging your tests across multiple dimensions, tag expressions help you to select which tests to execute. When tagging by test type (e.g., micro, integration, end-to-end) and feature (e.g., product, catalog, shipping), the following tag expressions can be useful.
+
+
+| Tag Expression | Selection |
+| -------- | --------|
+| product        | all tests for product     |
+| catalog \| shipping        | all tests for catalog plus all tests for shipping     |
+| catalog & shipping       | all tests for the intersection between catalog and shipping     |
+| product & !end-to-end | all tests for product, but not the end-to-end tests |
+| (micro \| integration) & (product \| shipping) | all micro or integration tests for product or shipping |
+
+
+Execution of the test is done by using the parameter `a_tags` with tag expressions
+
 
 ```sql linenums="1"
-begin
-  ut.run('hr.test_apply_bonus', a_tags => 'test1,test2');
-end;
+select * from table(ut.run(a_tags => 'fast|!complex'));
+```
+The above call will execute all tests from `ut_sample_test` package as the whole suite is tagged with `api` because a suite meet expression condition.
+
+```sql linenums="1"
+select * from table(ut.run(a_path => 'ut_sample_test',a_tags => 'api'));
+```
+The above call will execute all tests from `ut_sample_test` package as the whole suite is tagged with `api`
+
+```sql linenums="1"
+select * from table(ut.run(a_tags => 'complex'));
 ```
+The above call will execute only the `ut_sample_test.ut_refcursors1` test, as only the test `ut_refcursors1` is tagged with `complex`
+
 ```sql linenums="1"
-select * from table(ut.run('hr.test_apply_bonus', a_tags => 'suite1'))
+select * from table(ut.run(a_tags => 'fast'));
 ```
+The above call will execute both `ut_sample_test.ut_refcursors1` and `ut_sample_test.ut_test` tests, as both tests are tagged with `fast`
+
+### Excluding tests/suites by tags
 
-You can also exclude specific tags by adding a `-` (dash) in front of the tag
+It is possible to exclude parts of test suites with tags.
+In order to do so, prefix the tag name to exclude with a `!` (exclamation) sign when invoking the test run which is equivalent of `-` (dash) in legacy notation.
+Examples (based on above sample test suite)
 
 ```sql linenums="1"
-select * from table(ut.run('hr.test_apply_bonus', a_tags => '-suite1'))
+select * from table(ut.run(a_tags => '(api|fast)&!complex'));
 ```
 
+which is equivalent of legacy calling:
+
+```sql linenums="1"
+select * from table(ut.run(a_tags => 'api,fast,-complex'));
+```
+
+or 
+
+```sql linenums="1"
+select * from table(ut.run(a_tags => '(api|fast)&(!complex&!test1)'));
+```
+
+which is equivalent of legacy calling:
+
+```sql linenums="1"
+select * from table(ut.run(a_tags => 'api,fast,-complex,-test1'));
+```
+
+The above call will execute all suites/contexts/tests that are marked with any of tags `api` or `fast` except those suites/contexts/tests that are marked as `complex`.  
+Given the above example package `ut_sample_test`, only `ut_sample_test.ut_test` will be executed.  
+
+
 ## Keeping uncommitted data after test-run
 
 utPLSQL by default runs tests in autonomous transaction and performs automatic rollback to assure that tests do not impact one-another and do not have impact on the current session in your IDE.

diff --git a/source/core/ut_suite_tag_filter.pkb b/source/core/ut_suite_tag_filter.pkb
@@ -87,19 +87,18 @@ create or replace package body ut_suite_tag_filter is
   /*
     https://stackoverflow.com/questions/29634992/shunting-yard-validate-expression
   */
-  function shunt_logical_expression(a_tags in varchar2) return ut_varchar2_list is
+  function shunt_logical_expression(a_tags in ut_varchar2_list) return ut_varchar2_list is
     l_operator_stack ut_stack := ut_stack();
-    l_input_tokens ut_varchar2_list := tokenize_tags_string(a_tags);
     l_rnp_tokens ut_varchar2_list := ut_varchar2_list();
     l_token varchar2(32767);
     l_expect_operand boolean := true;
     l_expect_operator boolean := false;
     l_idx pls_integer;
   begin    
-    l_idx := l_input_tokens.first;
+    l_idx := a_tags.first;
     --Exuecute modified shunting algorithm
     WHILE (l_idx is not null) loop
-      l_token := l_input_tokens(l_idx);
+      l_token := a_tags(l_idx);
       if (l_token member of gc_operators and l_token member of gc_binary_operators) then
         if not(l_expect_operator) then 
           raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); 
@@ -108,21 +107,21 @@ create or replace package body ut_suite_tag_filter is
           l_rnp_tokens.extend;
           l_rnp_tokens(l_rnp_tokens.last) := l_operator_stack.pop;
         end loop;
-        l_operator_stack.push(l_input_tokens(l_idx));
+        l_operator_stack.push(a_tags(l_idx));
         l_expect_operand := true;
         l_expect_operator:= false;
       elsif (l_token member of gc_operators and l_token member of gc_unary_operators) then  
         if not(l_expect_operand) then 
           raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); 
         end if;        
-        l_operator_stack.push(l_input_tokens(l_idx));
+        l_operator_stack.push(a_tags(l_idx));
         l_expect_operand := true;
         l_expect_operator:= false;   
       elsif l_token = '(' then
         if not(l_expect_operand) then 
           raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); 
         end if;        
-        l_operator_stack.push(l_input_tokens(l_idx));
+        l_operator_stack.push(a_tags(l_idx));
         l_expect_operand := true;
         l_expect_operator:= false;      
       elsif l_token = ')' then
@@ -146,7 +145,7 @@ create or replace package body ut_suite_tag_filter is
         l_expect_operand := false;
       end if;
 
-      l_idx := l_input_tokens.next(l_idx);
+      l_idx := a_tags.next(l_idx);
     end loop;
 
     while l_operator_stack.peek is not null loop
@@ -198,9 +197,10 @@ create or replace package body ut_suite_tag_filter is
   function create_where_filter(a_tags varchar2
   ) return varchar2 is
     l_tags varchar2(4000);
+    l_tokenized_tags ut_varchar2_list;
   begin
     l_tags := replace(replace_legacy_tag_notation(a_tags),' ');
-    l_tags := conv_postfix_to_infix_sql(shunt_logical_expression(l_tags));
+    l_tags := conv_postfix_to_infix_sql(shunt_logical_expression(tokenize_tags_string(l_tags)));
     l_tags := replace(l_tags, '|',' or ');
     l_tags := replace(l_tags ,'&',' and ');
     l_tags := replace(l_tags ,'!','not');

diff --git a/source/core/ut_suite_tag_filter.pks b/source/core/ut_suite_tag_filter.pks
@@ -31,7 +31,7 @@ create or replace package ut_suite_tag_filter authid definer is
   * and return a list of elements in Reverse Polish Notation ( postfix )
   * As part of execution it will validate expression.
   */
-  function shunt_logical_expression(a_tags in varchar2) return ut_varchar2_list;
+  function shunt_logical_expression(a_tags in ut_varchar2_list) return ut_varchar2_list;
 
   /*
   * Function that converts postfix notation into infix and creating a string of sql filter 

diff --git a/test/ut3_tester/core/test_ut_suite_tag_filter.pkb b/test/ut3_tester/core/test_ut_suite_tag_filter.pkb
@@ -3,24 +3,65 @@ create or replace package body test_ut_suite_tag_filter is
   procedure test_conversion_to_rpn is
     l_postfix ut3_develop.ut_varchar2_list;
     l_postfix_string varchar2(4000);
+    l_input_token ut3_develop.ut_varchar2_list;
   begin
-    l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression('A');
+    l_input_token := ut3_develop.ut_suite_tag_filter.tokenize_tags_string('A');
+    l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression(l_input_token);
     l_postfix_string := ut3_develop.ut_utils.table_to_clob(l_postfix,'');
     ut.expect(l_postfix_string).to_equal('A');
 
-    l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression('A|B');
+    l_input_token := ut3_develop.ut_suite_tag_filter.tokenize_tags_string('A|B');
+    l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression(l_input_token);
     l_postfix_string := ut3_develop.ut_utils.table_to_clob(l_postfix,'');
     ut.expect(l_postfix_string).to_equal('AB|');
 
-    l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression('(a|b)|c&d');
+    l_input_token := ut3_develop.ut_suite_tag_filter.tokenize_tags_string('(a|b)|c&d');
+    l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression(l_input_token);
     l_postfix_string := ut3_develop.ut_utils.table_to_clob(l_postfix,'');
     ut.expect(l_postfix_string).to_equal('ab|cd&|');     
 
-    l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression('!a|b');
+    l_input_token := ut3_develop.ut_suite_tag_filter.tokenize_tags_string('!a|b');
+    l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression(l_input_token);
     l_postfix_string := ut3_develop.ut_utils.table_to_clob(l_postfix,'');
     ut.expect(l_postfix_string).to_equal('a!b|');         
   end;
 
+  procedure test_conversion_opr_by_opr is
+    l_postfix ut3_develop.ut_varchar2_list;
+    l_input_token ut3_develop.ut_varchar2_list;
+  begin
+    l_input_token := ut3_develop.ut_varchar2_list('A','B');
+    l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression(l_input_token);
+    ut.fail('Expected exception but nothing was raised');
+  end;
+
+  procedure test_conversion_oprd_by_opd is
+    l_postfix ut3_develop.ut_varchar2_list;
+    l_input_token ut3_develop.ut_varchar2_list;
+  begin
+    l_input_token := ut3_develop.ut_varchar2_list('|','|');
+    l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression(l_input_token);
+    ut.fail('Expected exception but nothing was raised');
+  end;
+
+  procedure test_conversion_lb_by_oper is
+    l_postfix ut3_develop.ut_varchar2_list;
+    l_input_token ut3_develop.ut_varchar2_list;
+  begin
+    l_input_token := ut3_develop.ut_varchar2_list('(','A','|','B',')','(');
+    l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression(l_input_token);
+    ut.fail('Expected exception but nothing was raised');
+  end;
+
+  procedure test_conversion_rb_by_oprd is
+    l_postfix ut3_develop.ut_varchar2_list;
+    l_input_token ut3_develop.ut_varchar2_list;
+  begin
+    l_input_token := ut3_develop.ut_varchar2_list(')','A');
+    l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression(l_input_token);
+    ut.fail('Expected exception but nothing was raised');
+  end;
+
   procedure conv_from_rpn_to_sql_filter is
     l_postfix_rpn ut3_develop.ut_varchar2_list;
     l_infix_string varchar2(4000);

diff --git a/test/ut3_tester/core/test_ut_suite_tag_filter.pks b/test/ut3_tester/core/test_ut_suite_tag_filter.pks
@@ -3,9 +3,29 @@ create or replace package test_ut_suite_tag_filter is
   --%suite(ut_suite_tag_filter)
   --%suitepath(utplsql.ut3_tester.core)
 
+  --%context( Conversion to Reverse Polish Notation)
+
   --%test( Test conversion of expression into Reverse Polish Notation)
   procedure test_conversion_to_rpn;
 
+  --%test( Operator is followed by operator)
+  --%throws(ut3_develop.ut_utils.gc_invalid_tag_expression)
+  procedure test_conversion_opr_by_opr;  
+
+  --%test( Operand is followed by operand)
+  --%throws(ut3_develop.ut_utils.gc_invalid_tag_expression)
+  procedure test_conversion_oprd_by_opd;
+
+  --%test( Left Bracket is followed by operator)
+  --%throws(ut3_develop.ut_utils.gc_invalid_tag_expression)
+  procedure test_conversion_lb_by_oper;
+
+  --%test( Right Bracket is followed by operand)
+  --%throws(ut3_develop.ut_utils.gc_invalid_tag_expression)
+  procedure test_conversion_rb_by_oprd;
+
+  --%endcontext
+
   --%test( Test conversion of expression from Reverse Polish Notation into custom where filter for SQL)
   procedure conv_from_rpn_to_sql_filter;