diff --git a/docs/userguide/coverage.md b/docs/userguide/coverage.md index 664fb78cb..664c972d1 100644 --- a/docs/userguide/coverage.md +++ b/docs/userguide/coverage.md @@ -1,7 +1,7 @@ ![version](https://img.shields.io/badge/version-v3.1.13.3908--develop-blue.svg) # Coverage -utPLSQL comes with a built-in coverage reporting engine. The code coverage reporting uses DBMS_PROFILER package provided with Oracle database. +utPLSQL comes with a built-in coverage reporting engine. The code coverage reporting uses package DBMS_PROFILER (and DBMS_PLSQL_CODE_COVERAGE on Oracle database version 12.2 and above) provided with Oracle database. Code coverage is gathered for the following source types: * package bodies * type bodies @@ -15,7 +15,7 @@ Code coverage is gathered for the following source types: To obtain information about code coverage for unit tests, run utPLSQL with one of built-in code coverage reporters. The following code coverage reporters are supplied with utPLSQL: -* `ut_coverage_html_reporter` - generates a HTML coverage report providing summary and detailed information on code coverage. The HTML reporter is based on the open-source [simplecov-html](https://github.com/colszowka/simplecov-html) reporter for Ruby. It includes source code of the code that was covered (if possible) +* `ut_coverage_html_reporter` - generates a HTML coverage report providing summary and detailed information on code coverage. The HTML reporter is based on the open-source [simplecov-html](https://github.com/colszowka/simplecov-html) reporter for Ruby. It includes source code of the code that was covered (if the code is accessible for test user) * `ut_coveralls_reporter` - generates a [Coveralls compatible JSON](https://coveralls.zendesk.com/hc/en-us/articles/201774865-API-Introduction) coverage report providing detailed information on code coverage with line numbers. This coverage report is designed to be consumed by cloud services like [Coveralls](https://coveralls.io) * `ut_coverage_sonar_reporter` - generates a [Sonar Compatible XML](https://docs.sonarqube.org/latest/analysis/generic-test/) coverage report providing detailed information on code coverage with line numbers. This coverage report is designed to be consumed by services like [SonarQube](https://www.sonarqube.org/) and [SonarCloud](https://about.sonarcloud.io/) * `ut_coverage_cobertura_reporter` - generates a basic Cobertura coverage (http://cobertura.sourceforge.net/xml/coverage-04.dtd) report providing detailed information on code coverage with line numbers. This coverage report is designed to be consumed by services like TFS and Jenkins. Check this link for an example of XML generated by Java: https://raw.githubusercontent.com/jenkinsci/cobertura-plugin/master/src/test/resources/hudson/plugins/cobertura/coverage-with-data.xml @@ -30,20 +30,21 @@ In order to be able to gather coverage information, the user executing unit test If you have `execute` privilege on the code that is being tested, but do not have `create any procedure` system privilege, then the code that is being tested will be reported as not covered (coverage = 0%). If you have `execute` privilege only on the unit tests, but do not have `execute` privilege on the code that is being tested, the code will not be reported by coverage - as if it did not exist in the database. -If the code that is being tested is complied as NATIVE, the code coverage will not be reported as well. +If the code that is being tested is compiled as NATIVE, the code coverage will not be reported as well. -## Running unit tests with coverage -Using the code coverage functionality is as easy as using any other [reporter](reporters.md) for the utPLSQL test-run. You just run your tests from your preferred SQL tool and save the reporter results to a file. -All you need to do is pass the constructor of the reporter to the `ut.run` procedure call. +## Manually running unit tests with coverage +Using the code coverage functionality is as easy as using any other [reporter](reporters.md) for the utPLSQL test-run. Run your tests from your preferred SQL tool and save the reporter results to a file. +All you need to do, is pass the constructor of the reporter to the `ut.run` procedure call. Example: ```sql +set serveroutput on begin ut.run(ut_coverage_html_reporter()); end; / ``` -The above command executes all unit tests in the **current schema**, gathers information about code coverage and outputs the HTML report as text into DBMS_OUTPUT. +The above command executes all unit tests in the **current schema**, gathers information about code coverage for all sources in that schema and outputs the HTML report as text into DBMS_OUTPUT. The `ut_coverage_html_reporter` will produce an interactive HTML report. You can see a sample of code coverage for the utPLSQL project [here](https://utplsql.github.io/utPLSQL-coverage-html/) The report provides summary information with a list of source code that should be covered. @@ -65,6 +66,7 @@ Due to security model of `dbms_plsql_code_coverage` package, utPLSQL grants acce The access and synonyms will be public when using the headless installation. This approach avoids complexity of forcing every user of utPLSQL framework to create tables on their own. Sample output: + ![Package Coverage Summary](../images/extended_coverage_html_summary.png) ![Line Coverage Details](../images/extended_coverage_html_line.png) @@ -78,14 +80,14 @@ There are two distinct ways to gather code coverage: - Coverage on project files Those two options are mutually exclusive and cannot be mixed. -By default, when using one of coverage reporters, coverage is gathered on schema(s). +By default, when using one of coverage reporters, coverage is gathered on schema(s). +The database schema(s) containing the tests that were executed during the run will be reported on by coverage reporter. The parameters used to execute tests determine if utPLSQL will be using one approach or the other. +If parameter `a_source_file_mappings` or `a_source_files` is provided, then coverage is gathered on project files provided, otherwise coverage is gathered on schemas. -The database schema(s) containing the tests that were executed during the run will be reported on by coverage reporter. **Note** - > Regardless of the options provided, all unit test packages are excluded from the coverage report. Coverage reports provide information only about the **tested** code. The default behavior of coverage reporting can be altered using invocation parameters. @@ -128,14 +130,112 @@ exec ut.run('unit_test_schema', ut_coverage_html_reporter(), a_coverage_schemes #### Filtering objects in coverage reports -There are two options that can be used to narrow down the scope of coverage report: -- `a_include_objects` - list of `[object_owner.].object_name` to be included in the coverage report -- `a_exclude_objects` - list of `[object_owner.].object_name` to be excluded from the coverage report +Multiple parameters can be used to define the scope of coverage report. +- `a_source_file_mappings ( ut_file_mappings )` - map of filenames to database objects. It is used for file-based coverage - see below. +- `a_include_schema_expr ( varchar(4000) )` - string of regex expression of schemas to be included in the coverage report. Case-insensitive. +- `a_include_object_expr ( varchar(4000) )` - string of regex expression of objects ( without schema name ) to be included in the coverage report. Case-insensitive. +- `a_exclude_schema_expr ( varchar(4000) )` - string of regex expression of schemas to be excluded from the coverage report. Case-insensitive. +- `a_exclude_object_expr ( varchar(4000) )` - string of regex expression of objects ( without schema name ) to be excluded from the coverage report. Case-insensitive. +- `a_coverage_schemes ( ut_varchar2_list )` - List of database schema names to gather coverage on. +- `a_include_objects ( ut_varchar2_list )` - list of `[object_owner.]object_name` to be included in the coverage report. +- `a_exclude_objects ( ut_varchar2_list )` - list of `[object_owner.]object_name` to be excluded from the coverage report. + +You may specify both _include_ and _exclude_ options to gain more control over what needs to be included / excluded from the coverage report. + +**Important notes** +The order of priority is for evaluation of include/exclude filter parameters is as follows. +- if `a_source_file_mappings` is defined then all include/exclude parameters are ignored (see section below for usage of `a_source_file_mappings` parameter ) +- else if `a_include_schema_expr` or `a_include_object_expr` parameter is specified then parameters `a_coverage_schemes` and `a_include_objects` are ignored +- else if `a_include_objects` is specified then the coverage is gathered only on specified database objects. + - if `a_coverage_schemes` is specified then those schemas are used for objects in `a_include_objects` without schema name + - if `a_coverage_schemes` is not specified then schema from paths (`a_paths`) parameter are used for objects in `a_include_objects` without schema name +- else if, only the `a_coverage_schemes` is specified then the coverage is gathered only on specified database schemas +- else if no coverage specific parameters are provided coverage is gathered on all schemas specified in paths passed to run procedure +- else if no paths were specified, the coverage is gathered on current schema of the session running the tests + +The exclude parameters are not mutually-exclusive and can be mixed together. All of exclude parameters are always applied. + +Example: Limiting coverage by schema regex. +```sql +begin + ut.run(ut_varchar2_list('user_1','user_2'), ut_coverage_html_reporter(), + a_include_schema_expr => '^ut3_develop' + ); +end; +``` +Will result in showing coverage for all schemas that match regular expression `^ut3_develop` -You may specify both _include_ and _exclude_ object lists to gain more control over what needs to be included / excluded from the coverage report. +Example: Limiting coverage by schema regex with parameter `a_include_objects` ignored. +```sql +begin + ut.run(ut_varchar2_list('user_1','user_2'), ut_coverage_html_reporter(), + a_include_schema_expr => '^ut3_develop', a_include_objects => ut_varchar2_list( 'ut3_tester_helper.regex_dummy_cov' ) + ); +end; +``` +Will result in showing coverage for all schemas that match regular expression `^ut3_develop`. -The object owner is optional in the object list. -If you do not provide an object owner, the _include_/_exclude_ list will be considered for every schema used for coverage gathering (as described above). +Example: Limiting coverage by object regex. +```sql +begin + ut.run(ut_varchar2_list('user_1','user_2'), ut_coverage_html_reporter(), + a_include_object_expr => 'regex123' + ); +end; +``` +Will result in showing coverage for all objects that name match regular expression `regex123`. + +Example: Limiting coverage by object regex with parameter `a_include_objects` ignored. +```sql +begin + ut.run(ut_varchar2_list('user_1','user_2'), ut_coverage_html_reporter(), + a_include_object_expr => 'utl', a_include_objects => ut_varchar2_list( 'user_2.utils_package' ) + ); +end; +``` +Will result in showing coverage for all objects that name match regular expression `utl`. + +Example: Limiting coverage by excluding schema with regex. +```sql +begin + ut.run(ut_varchar2_list('user_1','user_2'), ut_coverage_html_reporter(), + a_exclude_schema_expr => 'er_1$' + ); +end; +``` +Will result in showing coverage for objects in all schema except schemas that are matching regular expression `er_1$` + +Example: Limiting coverage by excluding schema with regex and excluding specific object. +```sql +begin + ut.run(ut_varchar2_list('user_1','user_2'), ut_coverage_html_reporter(), + a_exclude_schema_expr => 'er_1$', a_exclude_objects => ut_varchar2_list( 'user_2.utils_package' ) + ); +end; +``` +Will result in showing coverage for objects in all schemas except schemas that are matching regular expression `er_1$` +Will also exclude object `user_2.utils_package` from coverage report + +Example: Limiting coverage by excluding objects with regex. +```sql +begin + ut.run(ut_varchar2_list('user_1','user_2'), ut_coverage_html_reporter(), + a_exclude_object_expr => 'utl' + ); +end; +``` +Will result in showing coverage for all objects that name is not matching regular expression `utl`. + +Example: Limiting coverage by excluding objects with regex with parameter `a_exclude_objects` ignored. +```sql +begin + ut.run(ut_varchar2_list('user_1','user_2'), ut_coverage_html_reporter(), + a_exclude_object_expr => 'utl', a_exclude_objects => ut_varchar2_list( 'user_2.utils_package' ) + ); +end; +``` +Will result in showing coverage for all objects that name is not matching regular expression `utl`. +Will also exclude object `user_2.utils_package` from coverage report Example: Limiting coverage by object name, for tested code located in the same schema as the unit tests. @@ -187,7 +287,8 @@ Executes test `test_award_bonus` in schema `unit_test_schema`. Coverage will be **Note** > When running coverage on schema objects, all source code of package bodies, functions, procedures, type bodies and triggers that were not executed will be reported as having 0% code coverage and all source code lines will show as uncovered. -> This is different than the behavior when gathering coverage on project files. +> This is different from the behavior when gathering coverage on project files. + ### Project based Coverage @@ -660,7 +761,7 @@ begin end; ``` -## Reporting coverage outside of utPLSQL +## Reporting coverage outside utPLSQL and in parallel sessions utPSLQL allows fo standalone reporting code coverage across multiple database sessions. This functionality enables coverage reporting for external testing tools. @@ -703,7 +804,7 @@ exec ut_runner.coverage_stop(); ```sql --SESSION 1 or SESSION2 2 or SESSION 3 --- run after calls in SESSION 1 & 2 are finshed +-- run after calls in SESSION 1 & 2 are finished -- retrieve coverage report in HTML format coverage_run_id value select * from table( @@ -717,7 +818,7 @@ select * ```sql --SESSION 1 or SESSION2 2 or SESSION 3 --- run after calls in SESSION 1 & 2 are finshed +-- run after calls in SESSION 1 & 2 are finished declare l_results_cursor sys_refcursor; begin @@ -753,6 +854,10 @@ ut_coverage_options( exclude_objects ut_varchar2_rows := null, include_objects ut_varchar2_rows := null, file_mappings ut_file_mappings := null + include_schema_expr varchar2(4000) := null, + include_object_expr varchar2(4000) := null, + exclude_schema_expr varchar2(4000) := null, + exclude_object_expr varchar2(4000) := null ); ``` @@ -768,6 +873,10 @@ The `a_coverage_options` parameter is used to control the scope and formatting o - `exclude_objects` - optional - list of object names to exclude from report - data-type `UT_VARCHAR2_ROWS` - `include_objects` - optional - list of object names to gather coverage on - data-type `UT_VARCHAR2_ROWS` - `file_mappings` - optional - list of schema names to gather coverage on - data-type `UT_FILE_MAPPINGS` +- `include_schema_expr` - optional - regular expression to match against schema name to include in coverage - data-type `VARCHAR2(4000)` +- `include_object_expr` - optional - regular expression to match against object name to include in coverage - data-type `VARCHAR2(4000)` +- `exclude_schema_expr` - optional - regular expression to match against schema name to exclude in coverage - data-type `VARCHAR2(4000)` +- `exclude_object_expr` - optional - regular expression to match against object name to exclude in coverage - data-type `VARCHAR2(4000)` `coverage_run_id` parameter identifies a common coverage run. The valid value type for that parameter is RAW(32). It is recommended to use `sys_guid()` to generate a common, unique identifier for a specific coverage run. @@ -775,4 +884,4 @@ If the identifier is not unique, previous runs of coverage that used the same `c For details on the meaning of `schema_names`, `exclude_objects`, `include_objects`, `file_mappings` see sections above. Note that data-types of include/exclude/schema lists are different when calling `ut.run` vs. calling `get_report/get_report_cursor`. - + diff --git a/source/api/ut.pkb b/source/api/ut.pkb index ee3d85342..5c91e54d4 100644 --- a/source/api/ut.pkb +++ b/source/api/ut.pkb @@ -131,7 +131,11 @@ create or replace package body ut is a_client_character_set varchar2, a_random_test_order integer, a_random_test_order_seed positive, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ) is pragma autonomous_transaction; begin @@ -150,7 +154,11 @@ create or replace package body ut is false, ut_utils.int_to_boolean(a_random_test_order), a_random_test_order_seed, - a_tags + a_tags, + a_include_schema_expr, + a_include_object_expr, + a_exclude_schema_expr, + a_exclude_object_expr ); rollback; end; @@ -167,10 +175,15 @@ create or replace package body ut is a_client_character_set varchar2, a_random_test_order integer, a_random_test_order_seed positive, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ) is pragma autonomous_transaction; begin + a_reporter := coalesce(a_reporter,ut_documentation_reporter()); ut_runner.run( a_paths, @@ -186,7 +199,11 @@ create or replace package body ut is false, ut_utils.int_to_boolean(a_random_test_order), a_random_test_order_seed, - a_tags + a_tags, + a_include_schema_expr, + a_include_object_expr, + a_exclude_schema_expr, + a_exclude_object_expr ); rollback; end; @@ -227,7 +244,11 @@ create or replace package body ut is a_client_character_set varchar2 := null, a_random_test_order integer := 0, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ) return ut_varchar2_rows pipelined is l_reporter ut_reporter_base := a_reporter; l_results sys_refcursor; @@ -244,7 +265,11 @@ create or replace package body ut is a_client_character_set, a_random_test_order, a_random_test_order_seed, - a_tags + a_tags, + a_include_schema_expr, + a_include_object_expr, + a_exclude_schema_expr, + a_exclude_object_expr ); if l_reporter is of (ut_output_reporter_base) then l_results := treat(l_reporter as ut_output_reporter_base).get_lines_cursor(); @@ -266,7 +291,11 @@ create or replace package body ut is a_client_character_set varchar2 := null, a_random_test_order integer := 0, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ) return ut_varchar2_rows pipelined is l_reporter ut_reporter_base := a_reporter; l_results sys_refcursor; @@ -283,7 +312,11 @@ create or replace package body ut is a_client_character_set, a_random_test_order, a_random_test_order_seed, - a_tags + a_tags, + a_include_schema_expr, + a_include_object_expr, + a_exclude_schema_expr, + a_exclude_object_expr ); if l_reporter is of (ut_output_reporter_base) then l_results := treat(l_reporter as ut_output_reporter_base).get_lines_cursor(); @@ -306,7 +339,11 @@ create or replace package body ut is a_client_character_set varchar2 := null, a_random_test_order integer := 0, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ) return ut_varchar2_rows pipelined is l_reporter ut_reporter_base := a_reporter; l_results sys_refcursor; @@ -323,7 +360,11 @@ create or replace package body ut is a_client_character_set, a_random_test_order, a_random_test_order_seed, - a_tags + a_tags, + a_include_schema_expr, + a_include_object_expr, + a_exclude_schema_expr, + a_exclude_object_expr ); if l_reporter is of (ut_output_reporter_base) then l_results := treat(l_reporter as ut_output_reporter_base).get_lines_cursor(); @@ -346,7 +387,11 @@ create or replace package body ut is a_client_character_set varchar2 := null, a_random_test_order integer := 0, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ) return ut_varchar2_rows pipelined is l_reporter ut_reporter_base := a_reporter; l_results sys_refcursor; @@ -363,7 +408,11 @@ create or replace package body ut is a_client_character_set, a_random_test_order, a_random_test_order_seed, - a_tags + a_tags, + a_include_schema_expr, + a_include_object_expr, + a_exclude_schema_expr, + a_exclude_object_expr ); if l_reporter is of (ut_output_reporter_base) then l_results := treat(l_reporter as ut_output_reporter_base).get_lines_cursor(); @@ -386,7 +435,11 @@ create or replace package body ut is a_client_character_set varchar2 := null, a_random_test_order integer := 0, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ) return ut_varchar2_rows pipelined is l_reporter ut_reporter_base := a_reporter; l_results sys_refcursor; @@ -403,7 +456,11 @@ create or replace package body ut is a_client_character_set, a_random_test_order, a_random_test_order_seed, - a_tags + a_tags, + a_include_schema_expr, + a_include_object_expr, + a_exclude_schema_expr, + a_exclude_object_expr ); if l_reporter is of (ut_output_reporter_base) then l_results := treat(l_reporter as ut_output_reporter_base).get_lines_cursor(); @@ -426,7 +483,11 @@ create or replace package body ut is a_client_character_set varchar2 := null, a_random_test_order integer := 0, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ) return ut_varchar2_rows pipelined is l_reporter ut_reporter_base := a_reporter; l_results sys_refcursor; @@ -443,7 +504,11 @@ create or replace package body ut is a_client_character_set, a_random_test_order, a_random_test_order_seed, - a_tags + a_tags, + a_include_schema_expr, + a_include_object_expr, + a_exclude_schema_expr, + a_exclude_object_expr ); if l_reporter is of (ut_output_reporter_base) then l_results := treat(l_reporter as ut_output_reporter_base).get_lines_cursor(); @@ -467,7 +532,11 @@ create or replace package body ut is a_force_manual_rollback boolean := false, a_random_test_order boolean := false, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ) is l_reporter ut_reporter_base := a_reporter; begin @@ -487,7 +556,11 @@ create or replace package body ut is a_force_manual_rollback, a_random_test_order, a_random_test_order_seed, - a_tags + a_tags, + a_include_schema_expr, + a_include_object_expr, + a_exclude_schema_expr, + a_exclude_object_expr ); else run_autonomous( @@ -502,7 +575,11 @@ create or replace package body ut is a_client_character_set, ut_utils.boolean_to_int(a_random_test_order), a_random_test_order_seed, - a_tags + a_tags, + a_include_schema_expr, + a_include_object_expr, + a_exclude_schema_expr, + a_exclude_object_expr ); end if; if l_reporter is of (ut_output_reporter_base) then @@ -524,7 +601,11 @@ create or replace package body ut is a_force_manual_rollback boolean := false, a_random_test_order boolean := false, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ) is l_reporter ut_reporter_base := a_reporter; begin @@ -541,7 +622,11 @@ create or replace package body ut is a_force_manual_rollback, a_random_test_order, a_random_test_order_seed, - a_tags + a_tags, + a_include_schema_expr, + a_include_object_expr, + a_exclude_schema_expr, + a_exclude_object_expr ); end; @@ -557,7 +642,11 @@ create or replace package body ut is a_force_manual_rollback boolean := false, a_random_test_order boolean := false, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ) is begin ut.run( @@ -573,7 +662,11 @@ create or replace package body ut is a_force_manual_rollback, a_random_test_order, a_random_test_order_seed, - a_tags + a_tags, + a_include_schema_expr, + a_include_object_expr, + a_exclude_schema_expr, + a_exclude_object_expr ); end; @@ -589,7 +682,11 @@ create or replace package body ut is a_force_manual_rollback boolean := false, a_random_test_order boolean := false, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ) is begin ut.run( @@ -605,7 +702,11 @@ create or replace package body ut is a_force_manual_rollback, a_random_test_order, a_random_test_order_seed, - a_tags + a_tags, + a_include_schema_expr, + a_include_object_expr, + a_exclude_schema_expr, + a_exclude_object_expr ); end; @@ -622,7 +723,11 @@ create or replace package body ut is a_force_manual_rollback boolean := false, a_random_test_order boolean := false, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ) is begin ut.run( @@ -638,7 +743,11 @@ create or replace package body ut is a_force_manual_rollback, a_random_test_order, a_random_test_order_seed, - a_tags + a_tags, + a_include_schema_expr, + a_include_object_expr, + a_exclude_schema_expr, + a_exclude_object_expr ); end; @@ -655,7 +764,11 @@ create or replace package body ut is a_force_manual_rollback boolean := false, a_random_test_order boolean := false, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ) is begin ut.run( @@ -671,7 +784,11 @@ create or replace package body ut is a_force_manual_rollback, a_random_test_order, a_random_test_order_seed, - a_tags + a_tags, + a_include_schema_expr, + a_include_object_expr, + a_exclude_schema_expr, + a_exclude_object_expr ); end; diff --git a/source/api/ut.pks b/source/api/ut.pks index df5a8d8e5..f72c82a0c 100644 --- a/source/api/ut.pks +++ b/source/api/ut.pks @@ -62,7 +62,11 @@ create or replace package ut authid current_user as a_client_character_set varchar2 := null, a_random_test_order integer := 0, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ) return ut_varchar2_rows pipelined; function run( @@ -76,7 +80,11 @@ create or replace package ut authid current_user as a_client_character_set varchar2 := null, a_random_test_order integer := 0, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ) return ut_varchar2_rows pipelined; function run( @@ -91,7 +99,11 @@ create or replace package ut authid current_user as a_client_character_set varchar2 := null, a_random_test_order integer := 0, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ) return ut_varchar2_rows pipelined; function run( @@ -106,7 +118,11 @@ create or replace package ut authid current_user as a_client_character_set varchar2 := null, a_random_test_order integer := 0, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ) return ut_varchar2_rows pipelined; function run( @@ -121,7 +137,11 @@ create or replace package ut authid current_user as a_client_character_set varchar2 := null, a_random_test_order integer := 0, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ) return ut_varchar2_rows pipelined; function run( @@ -136,7 +156,11 @@ create or replace package ut authid current_user as a_client_character_set varchar2 := null, a_random_test_order integer := 0, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ) return ut_varchar2_rows pipelined; procedure run( @@ -151,7 +175,11 @@ create or replace package ut authid current_user as a_force_manual_rollback boolean := false, a_random_test_order boolean := false, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ); procedure run( @@ -166,7 +194,11 @@ create or replace package ut authid current_user as a_force_manual_rollback boolean := false, a_random_test_order boolean := false, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ); procedure run( @@ -182,7 +214,11 @@ create or replace package ut authid current_user as a_force_manual_rollback boolean := false, a_random_test_order boolean := false, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ); procedure run( @@ -198,7 +234,11 @@ create or replace package ut authid current_user as a_force_manual_rollback boolean := false, a_random_test_order boolean := false, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ); procedure run( @@ -214,7 +254,11 @@ create or replace package ut authid current_user as a_force_manual_rollback boolean := false, a_random_test_order boolean := false, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ); procedure run( @@ -230,7 +274,11 @@ create or replace package ut authid current_user as a_force_manual_rollback boolean := false, a_random_test_order boolean := false, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ); /** diff --git a/source/api/ut_runner.pkb b/source/api/ut_runner.pkb index d2bd74b91..cbf1971e7 100644 --- a/source/api/ut_runner.pkb +++ b/source/api/ut_runner.pkb @@ -69,7 +69,11 @@ create or replace package body ut_runner is a_force_manual_rollback boolean := false, a_random_test_order boolean := false, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ) is l_run ut_run; l_coverage_schema_names ut_varchar2_rows; @@ -120,15 +124,18 @@ create or replace package body ut_runner is ut_utils.trim_list_elements(ut_utils.filter_list(ut_utils.string_to_table(a_tags,','),ut_utils.gc_word_no_space)) ); end if; - l_run := ut_run( a_run_paths => l_paths, a_coverage_options => ut_coverage_options( - coverage_run_id => sys_guid(), + coverage_run_id => ut_coverage.get_coverage_run_id(), schema_names => l_coverage_schema_names, exclude_objects => ut_utils.convert_collection(a_exclude_objects), include_objects => ut_utils.convert_collection(a_include_objects), - file_mappings => set(a_source_file_mappings) + file_mappings => set(a_source_file_mappings), + include_schema_expr => a_include_schema_expr, + include_object_expr => a_include_object_expr, + exclude_schema_expr => a_exclude_schema_expr, + exclude_object_expr => a_exclude_object_expr ), a_test_file_mappings => set(a_test_file_mappings), a_client_character_set => a_client_character_set, @@ -142,7 +149,6 @@ create or replace package body ut_runner is end if; l_run.do_execute(); - finish_run(l_run, a_force_manual_rollback); exception when others then diff --git a/source/api/ut_runner.pks b/source/api/ut_runner.pks index 75aa12f8f..3170dedee 100644 --- a/source/api/ut_runner.pks +++ b/source/api/ut_runner.pks @@ -71,7 +71,11 @@ create or replace package ut_runner authid current_user is a_force_manual_rollback boolean := false, a_random_test_order boolean := false, a_random_test_order_seed positive := null, - a_tags varchar2 := null + a_tags varchar2 := null, + a_include_schema_expr varchar2 := null, + a_include_object_expr varchar2 := null, + a_exclude_schema_expr varchar2 := null, + a_exclude_object_expr varchar2 := null ); /** diff --git a/source/core/coverage/ut_coverage.pkb b/source/core/coverage/ut_coverage.pkb index 8bdcec4eb..f5eccc893 100644 --- a/source/core/coverage/ut_coverage.pkb +++ b/source/core/coverage/ut_coverage.pkb @@ -18,6 +18,7 @@ create or replace package body ut_coverage is g_develop_mode boolean not null := false; g_is_started boolean not null := false; + g_coverage_run_id raw(32); procedure set_develop_mode(a_develop_mode in boolean) is begin @@ -35,6 +36,7 @@ create or replace package body ut_coverage is l_join_mappings varchar2(32767); l_filters varchar2(32767); l_mappings_cardinality integer := 0; + l_regex_exc_filters varchar2(32767); begin l_result := q'[ with @@ -54,6 +56,12 @@ create or replace package body ut_coverage is from {sources_view} s {join_file_mappings} where s.type in ('PACKAGE BODY', 'TYPE BODY', 'PROCEDURE', 'FUNCTION', 'TRIGGER') {filters} + {regex_exc_filters} + and not exists ( + select /*+ cardinality(el {excuded_objects_cardinality})*/ 1 + from table(:l_excluded_objects) el + where s.owner = el.owner and s.name = el.name + ) ), coverage_sources as ( select full_name, owner, name, type, line, text, @@ -85,7 +93,7 @@ create or replace package body ut_coverage is ) and line > 0 ]'; - + if a_coverage_options.file_mappings is not empty then l_mappings_cardinality := ut_utils.scale_cardinality(cardinality(a_coverage_options.file_mappings)); l_full_name := 'f.file_name'; @@ -94,6 +102,18 @@ create or replace package body ut_coverage is on s.name = f.object_name and s.type = f.object_type and s.owner = f.object_owner'; + elsif coalesce(a_coverage_options.include_schema_expr,a_coverage_options.include_object_expr) is not null then + l_full_name := q'[lower(s.type||' '||s.owner||'.'||s.name)]'; + if a_coverage_options.include_schema_expr is not null then + l_filters := q'[and regexp_like(s.owner,:a_include_schema_expr,'i')]'; + else + l_filters := 'and :a_include_schema_expr is null'; + end if; + if a_coverage_options.include_object_expr is not null then + l_filters := l_filters|| q'[ and regexp_like(s.name,:a_include_object_expr,'i')]'; + else + l_filters := l_filters|| 'and :a_include_object_expr is null'; + end if; else l_full_name := q'[lower(s.type||' '||s.owner||'.'||s.name)]'; l_filters := case @@ -111,37 +131,67 @@ create or replace package body ut_coverage is end; end if; + + if a_coverage_options.exclude_schema_expr is not null then + l_regex_exc_filters := q'[ and not regexp_like(s.owner,:a_exclude_schema_expr,'i')]'; + else + l_regex_exc_filters := ' and :a_exclude_schema_expr is null '; + end if; + + if a_coverage_options.exclude_object_expr is not null then + l_regex_exc_filters := l_regex_exc_filters||q'[ and not regexp_like(s.name,:a_exclude_obj_expr,'i')]'; + else + l_regex_exc_filters := l_regex_exc_filters||'and :a_exclude_obj_expr is null '; + end if; + + + l_result := replace(l_result, '{sources_view}', ut_metadata.get_source_view_name()); l_result := replace(l_result, '{l_full_name}', l_full_name); l_result := replace(l_result, '{join_file_mappings}', l_join_mappings); l_result := replace(l_result, '{filters}', l_filters); l_result := replace(l_result, '{mappings_cardinality}', l_mappings_cardinality); l_result := replace(l_result, '{skipped_objects_cardinality}', ut_utils.scale_cardinality(cardinality(a_skip_objects))); + l_result := replace(l_result, '{excuded_objects_cardinality}', ut_utils.scale_cardinality(cardinality(coalesce(a_coverage_options.exclude_objects, ut_object_names())))); + l_result := replace(l_result, '{regex_exc_filters}', l_regex_exc_filters); return l_result; end; function get_cov_sources_cursor(a_coverage_options in ut_coverage_options) return sys_refcursor is - l_cursor sys_refcursor; - l_skip_objects ut_object_names; - l_sql varchar2(32767); + l_cursor sys_refcursor; + l_skip_objects ut_object_names; + l_excluded_objects ut_object_names; + l_sql varchar2(32767); begin if not is_develop_mode() then --skip all the utplsql framework objects and all the unit test packages that could potentially be reported by coverage. - l_skip_objects := ut_utils.get_utplsql_objects_list() multiset union all coalesce(a_coverage_options.exclude_objects, ut_object_names()); + l_skip_objects := coalesce(ut_utils.get_utplsql_objects_list(),ut_object_names()); end if; + --Regex exclusion override the standard exclusion objects. + if a_coverage_options.exclude_schema_expr is null and a_coverage_options.exclude_object_expr is null then + l_excluded_objects := coalesce(a_coverage_options.exclude_objects, ut_object_names()); + end if; + l_sql := get_cov_sources_sql(a_coverage_options, l_skip_objects); ut_event_manager.trigger_event(ut_event_manager.gc_debug, ut_key_anyvalues().put('l_sql',l_sql) ); if a_coverage_options.file_mappings is not empty then - open l_cursor for l_sql using a_coverage_options.file_mappings, l_skip_objects; + open l_cursor for l_sql using a_coverage_options.file_mappings, a_coverage_options.exclude_schema_expr, + a_coverage_options.exclude_object_expr, l_excluded_objects, l_skip_objects; + elsif coalesce(a_coverage_options.include_schema_expr,a_coverage_options.include_object_expr) is not null then + open l_cursor for l_sql using a_coverage_options.include_schema_expr, a_coverage_options.include_object_expr, + a_coverage_options.exclude_schema_expr, a_coverage_options.exclude_object_expr, + l_excluded_objects, l_skip_objects; elsif a_coverage_options.include_objects is not empty then - open l_cursor for l_sql using a_coverage_options.include_objects, l_skip_objects; + open l_cursor for l_sql using a_coverage_options.include_objects, a_coverage_options.exclude_schema_expr, + a_coverage_options.exclude_object_expr, l_excluded_objects, l_skip_objects; else - open l_cursor for l_sql using a_coverage_options.schema_names, l_skip_objects; + open l_cursor for l_sql using a_coverage_options.schema_names, a_coverage_options.exclude_schema_expr, + a_coverage_options.exclude_object_expr, l_excluded_objects, l_skip_objects; end if; return l_cursor; end; @@ -179,6 +229,7 @@ create or replace package body ut_coverage is l_block_coverage_id integer; begin if not is_develop_mode() and not g_is_started then + g_coverage_run_id := a_coverage_run_id; l_line_coverage_id := ut_coverage_helper_profiler.coverage_start( l_run_comment ); l_block_coverage_id := ut_coverage_helper_block.coverage_start( l_run_comment ); g_is_started := true; @@ -202,9 +253,9 @@ create or replace package body ut_coverage is begin if not is_develop_mode() then g_is_started := false; + g_coverage_run_id := null; ut_coverage_helper_block.coverage_stop(); ut_coverage_helper_profiler.coverage_stop(); - g_is_started := false; end if; end; @@ -262,7 +313,15 @@ create or replace package body ut_coverage is $end return l_result_profiler_enrich; - end get_coverage_data; + end get_coverage_data; + + function get_coverage_run_id return raw is + begin + if g_coverage_run_id is null then + g_coverage_run_id := sys_guid(); + end if; + return g_coverage_run_id; + end; end; / diff --git a/source/core/coverage/ut_coverage.pks b/source/core/coverage/ut_coverage.pks index 210cb13f6..21f3b1f9e 100644 --- a/source/core/coverage/ut_coverage.pks +++ b/source/core/coverage/ut_coverage.pks @@ -74,5 +74,7 @@ create or replace package ut_coverage authid current_user is function get_coverage_data(a_coverage_options ut_coverage_options) return t_coverage; + function get_coverage_run_id return raw; + end; / diff --git a/source/core/types/ut_coverage_options.tpb b/source/core/types/ut_coverage_options.tpb index 308c75553..a091802af 100644 --- a/source/core/types/ut_coverage_options.tpb +++ b/source/core/types/ut_coverage_options.tpb @@ -22,7 +22,11 @@ create or replace type body ut_coverage_options as schema_names ut_varchar2_rows := null, exclude_objects ut_varchar2_rows := null, include_objects ut_varchar2_rows := null, - file_mappings ut_file_mappings := null + file_mappings ut_file_mappings := null, + include_schema_expr varchar2 := null, + include_object_expr varchar2 := null, + exclude_schema_expr varchar2 := null, + exclude_object_expr varchar2 := null ) return self as result is function to_ut_object_list(a_names ut_varchar2_rows, a_schema_names ut_varchar2_rows) return ut_object_names is l_result ut_object_names; @@ -54,10 +58,15 @@ create or replace type body ut_coverage_options as if exclude_objects is not empty then self.exclude_objects := to_ut_object_list(exclude_objects, self.schema_names); end if; --- self.exclude_objects := self.exclude_objects multiset union all ut_suite_manager.get_schema_ut_packages(schema_names); self.include_objects := to_ut_object_list(include_objects, self.schema_names); + self.include_schema_expr := include_schema_expr; + self.include_object_expr := include_object_expr; + self.exclude_schema_expr := exclude_schema_expr; + self.exclude_object_expr := exclude_object_expr; + + return; end; end; diff --git a/source/core/types/ut_coverage_options.tps b/source/core/types/ut_coverage_options.tps index 1be518c5e..a8e3d5129 100644 --- a/source/core/types/ut_coverage_options.tps +++ b/source/core/types/ut_coverage_options.tps @@ -21,13 +21,21 @@ create or replace type ut_coverage_options force as object ( exclude_objects ut_object_names, include_objects ut_object_names, file_mappings ut_file_mappings, + include_schema_expr varchar2(4000), + include_object_expr varchar2(4000), + exclude_schema_expr varchar2(4000), + exclude_object_expr varchar2(4000), constructor function ut_coverage_options( self in out nocopy ut_coverage_options, coverage_run_id raw, schema_names ut_varchar2_rows := null, exclude_objects ut_varchar2_rows := null, include_objects ut_varchar2_rows := null, - file_mappings ut_file_mappings := null + file_mappings ut_file_mappings := null, + include_schema_expr varchar2 := null, + include_object_expr varchar2 := null, + exclude_schema_expr varchar2 := null, + exclude_object_expr varchar2 := null ) return self as result ) / diff --git a/test/ut3_tester_helper/coverage_helper.pkb b/test/ut3_tester_helper/coverage_helper.pkb index 137c7e953..fdfa32364 100644 --- a/test/ut3_tester_helper/coverage_helper.pkb +++ b/test/ut3_tester_helper/coverage_helper.pkb @@ -158,6 +158,107 @@ create or replace package body coverage_helper is end; + procedure create_regex_dummy_for_schema(p_schema in varchar2) is + pragma autonomous_transaction; + begin + execute immediate q'[create or replace package ]'||p_schema||q'[.regex_dummy_cov is + procedure do_stuff(i_input in number); + end;]'; + + execute immediate q'[create or replace package body ]'||p_schema||q'[.regex_dummy_cov is + procedure do_stuff(i_input in number) is + begin + if i_input = 2 then dbms_output.put_line('should not get here'); elsif i_input = 1 then dbms_output.put_line('should get here'); + else + dbms_output.put_line('should not get here'); + end if; + end; + end;]'; + + execute immediate q'[create or replace package ]'||p_schema||q'[.test_regex_dummy_cov is + --%suite(dummy coverage test) + --%suitepath(coverage_testing) + + --%test + procedure test_do_stuff; + + --%test + procedure zero_coverage; + end;]'; + + execute immediate q'[create or replace package body ]'||p_schema||q'[.test_regex_dummy_cov is + procedure test_do_stuff is + begin + regex_dummy_cov.do_stuff(1); + ut.expect(1).to_equal(1); + end; + procedure zero_coverage is + begin + null; + end; + end;]'; + end; + + procedure create_regex_dummy_obj is + pragma autonomous_transaction; + begin + execute immediate q'[create or replace package ut3_develop.regex123_dummy_cov is + procedure do_stuff(i_input in number); + end;]'; + + execute immediate q'[create or replace package body ut3_develop.regex123_dummy_cov is + procedure do_stuff(i_input in number) is + begin + if i_input = 2 then dbms_output.put_line('should not get here'); elsif i_input = 1 then dbms_output.put_line('should get here'); + else + dbms_output.put_line('should not get here'); + end if; + end; + end;]'; + + execute immediate q'[create or replace package ut3_develop.test_regex123_dummy_cov is + --%suite(dummy coverage test) + --%suitepath(coverage_testing) + + --%test + procedure test_do_stuff; + + --%test + procedure zero_coverage; + end;]'; + + execute immediate q'[create or replace package body ut3_develop.test_regex123_dummy_cov is + procedure test_do_stuff is + begin + regex123_dummy_cov.do_stuff(1); + ut.expect(1).to_equal(1); + end; + procedure zero_coverage is + begin + null; + end; + end;]'; + end; + + procedure create_regex_dummy_cov is + begin + create_regex_dummy_for_schema('ut3_develop'); + create_regex_dummy_for_schema('ut3_tester_helper'); + create_regex_dummy_obj; + end; + + procedure drop_regex_dummy_cov is + pragma autonomous_transaction; + begin + begin execute immediate q'[drop package ut3_develop.regex_dummy_cov]'; exception when others then null; end; + begin execute immediate q'[drop package ut3_develop.test_regex_dummy_cov]'; exception when others then null; end; + begin execute immediate q'[drop package ut3_tester_helper.regex_dummy_cov]'; exception when others then null; end; + begin execute immediate q'[drop package ut3_tester_helper.test_regex_dummy_cov]'; exception when others then null; end; + begin execute immediate q'[drop package ut3_develop.regex123_dummy_cov]'; exception when others then null; end; + begin execute immediate q'[drop package ut3_develop.test_regex123_dummy_cov]'; exception when others then null; end; + end; + + procedure drop_cov_with_dbms_stats is pragma autonomous_transaction; begin @@ -240,7 +341,7 @@ create or replace package body coverage_helper is e_exists exception; pragma exception_init ( e_exists, -955 ); begin - execute immediate 'create table test_results (text varchar2(4000))'; + execute immediate 'create table test_results (id integer, text varchar2(4000))'; exception when e_exists then null; @@ -267,9 +368,10 @@ create or replace package body coverage_helper is declare l_results ut3_develop.ut_varchar2_list; begin - select * + select text bulk collect into l_results - from test_results; + from test_results + order by id; delete from test_results; commit; :clob_results := ut3_tester_helper.main_helper.table_to_clob(l_results); @@ -280,14 +382,122 @@ create or replace package body coverage_helper is return l_result_clob; end; + procedure copy_coverage_data_to_ut3(a_coverage_run_id raw) is + pragma autonomous_transaction; + l_current_coverage_run_id raw(32) := hextoraw(sys_context('UT3_INFO','COVERAGE_RUN_ID')); + begin + insert into ut3.ut_coverage_runs(coverage_run_id, line_coverage_id, block_coverage_id) + select l_current_coverage_run_id, -line_coverage_id, -block_coverage_id + from ut3_develop.ut_coverage_runs + where coverage_run_id = a_coverage_run_id; + + insert into ut3.plsql_profiler_runs(runid, related_run, run_owner, run_date, run_comment, run_total_time, run_system_info, run_comment1, spare1) + select -runid, related_run, run_owner, run_date, run_comment, run_total_time, run_system_info, run_comment1, spare1 + from ut3_develop.plsql_profiler_runs c + join ut3_develop.ut_coverage_runs r + on r.line_coverage_id = c.runid + where r.coverage_run_id = a_coverage_run_id; + + insert into ut3.plsql_profiler_units(runid, unit_number, unit_type, unit_owner, unit_name, unit_timestamp, total_time, spare1, spare2) + select -runid, unit_number, unit_type, unit_owner, unit_name, unit_timestamp, total_time, spare1, spare2 + from ut3_develop.plsql_profiler_units c + join ut3_develop.ut_coverage_runs r + on r.line_coverage_id = c.runid + where r.coverage_run_id = a_coverage_run_id; + + insert into ut3.plsql_profiler_data(runid, unit_number, line#, total_occur, total_time, min_time, max_time, spare1, spare2, spare3, spare4) + select -runid, unit_number, line#, total_occur, total_time, min_time, max_time, spare1, spare2, spare3, spare4 + from ut3_develop.plsql_profiler_data c + join ut3_develop.ut_coverage_runs r + on r.line_coverage_id = c.runid + where r.coverage_run_id = a_coverage_run_id; + + insert into ut3.dbmspcc_runs(run_id, run_comment, run_owner, run_timestamp) + select -run_id, run_comment, run_owner, run_timestamp + from ut3_develop.dbmspcc_runs c + join ut3_develop.ut_coverage_runs r + on r.block_coverage_id = c.run_id + where r.coverage_run_id = a_coverage_run_id; + + insert into ut3.dbmspcc_units(run_id, object_id, owner, name, type, last_ddl_time) + select -run_id, object_id, owner, name, type, last_ddl_time + from ut3_develop.dbmspcc_units c + join ut3_develop.ut_coverage_runs r + on r.block_coverage_id = c.run_id + where r.coverage_run_id = a_coverage_run_id; + + insert into ut3.dbmspcc_blocks(run_id, object_id, block, line, col, covered, not_feasible) + select -run_id, object_id, block, line, col, covered, not_feasible + from ut3_develop.dbmspcc_blocks c + join ut3_develop.ut_coverage_runs r + on r.block_coverage_id = c.run_id + where r.coverage_run_id = a_coverage_run_id; + + commit; + end; + + function gather_coverage_on_coverage( a_cov_options varchar2) return clob is + pragma autonomous_transaction; + l_plsql_block varchar2(32767); + l_result_clob clob; + l_coverage_id raw(32) := sys_guid(); + begin + l_plsql_block := q'[ + declare + l_coverage_options ut3_develop.ut_coverage_options; + l_coverage_run_id raw(32) := ']'||rawtohex(l_coverage_id)||q'['; + l_result ut3_develop.ut_coverage.t_coverage; + begin + ut3_develop.ut_runner.coverage_start(l_coverage_run_id); + ut3_develop.ut_coverage.set_develop_mode(a_develop_mode => true); + l_coverage_options := {a_cov_options}; + l_result := ut3_develop.ut_coverage.get_coverage_data(l_coverage_options); + ut3_develop.ut_coverage.set_develop_mode(a_develop_mode => false); + ut3_develop.ut_runner.coverage_stop(); + insert into test_results select rownum, owner||'.'||name from ut3_develop.ut_coverage_sources_tmp; + commit; + end;]'; + l_plsql_block := replace(l_plsql_block,'{a_cov_options}',a_cov_options); + run_job_and_wait_for_finish( l_plsql_block ); + execute immediate q'[ + declare + l_results ut3_develop.ut_varchar2_list; + begin + select text + bulk collect into l_results + from test_results + order by id; + delete from test_results; + commit; + :clob_results := ut3_tester_helper.main_helper.table_to_clob(l_results); + end; + ]' + using out l_result_clob; + copy_coverage_data_to_ut3(l_coverage_id); + return l_result_clob; + end; + function run_tests_as_job( a_run_command varchar2 ) return clob is l_plsql_block varchar2(32767); l_result_clob clob; - pragma autonomous_transaction; + l_coverage_id raw(32) := sys_guid(); begin - l_plsql_block := 'begin insert into test_results select * from table( {a_run_command} ); commit; end;'; + l_plsql_block := q'[ + begin + ut3_develop.ut_runner.coverage_start(']'||rawtohex(l_coverage_id)||q'['); + ut3_develop.ut_coverage.set_develop_mode(a_develop_mode => true); + --gather coverage on the command executed + begin {a_run_command}; end; + ut3_develop.ut_coverage.set_develop_mode(a_develop_mode => false); + ut3_develop.ut_runner.coverage_stop(); + --get the actual results of the command gathering the coverage + insert into test_results select rownum as id, x.* from table( {a_run_command} ) x; + commit; + end;]'; l_plsql_block := replace(l_plsql_block,'{a_run_command}',a_run_command); - return run_code_as_job( l_plsql_block ); + l_result_clob := run_code_as_job( l_plsql_block ); + copy_coverage_data_to_ut3(l_coverage_id); + return l_result_clob; end; procedure create_dup_object_name is diff --git a/test/ut3_tester_helper/coverage_helper.pks b/test/ut3_tester_helper/coverage_helper.pks index 1db09a01b..8d2c1c150 100644 --- a/test/ut3_tester_helper/coverage_helper.pks +++ b/test/ut3_tester_helper/coverage_helper.pks @@ -14,12 +14,16 @@ create or replace package coverage_helper is procedure create_dummy_coverage_1; procedure drop_dummy_coverage_1; + procedure create_regex_dummy_cov; + procedure drop_regex_dummy_cov; + procedure create_cov_with_dbms_stats; procedure drop_cov_with_dbms_stats; procedure run_standalone_coverage(a_coverage_run_id raw, a_input integer); procedure run_coverage_job(a_coverage_run_id raw, a_input integer); + function gather_coverage_on_coverage( a_cov_options varchar2) return clob; function run_tests_as_job( a_run_command varchar2 ) return clob; function run_code_as_job( a_plsql_block varchar2 ) return clob; procedure create_test_results_table; diff --git a/test/ut3_user/expectations/binary/test_to_be_within.pkb b/test/ut3_user/expectations/binary/test_to_be_within.pkb index d490a8280..032b684a5 100644 --- a/test/ut3_user/expectations/binary/test_to_be_within.pkb +++ b/test/ut3_user/expectations/binary/test_to_be_within.pkb @@ -199,7 +199,7 @@ create or replace package body test_to_be_within is begin --Arrange --Act - ut3_develop.ut.expect(sysdate).to_be_within(INTERVAL '1' MONTH).of_(sysdate+ 45); + ut3_develop.ut.expect(sysdate).to_be_within(INTERVAL '1' MONTH).of_(sysdate+ 46); --Assert l_expected_message := q'[Actual: % (date) was expected to be within 1 month of % (date)]'; l_actual_message := ut3_tester_helper.main_helper.get_failed_expectations(1); diff --git a/test/ut3_user/reporters/test_coverage/test_extended_coverage.pkb b/test/ut3_user/reporters/test_coverage/test_extended_coverage.pkb index a8d104e28..3c8781bc4 100644 --- a/test/ut3_user/reporters/test_coverage/test_extended_coverage.pkb +++ b/test/ut3_user/reporters/test_coverage/test_extended_coverage.pkb @@ -106,5 +106,210 @@ create or replace package body test_extended_coverage is ut.expect(l_actual).to_be_like(l_expected); end; + procedure coverage_regex_include_schema is + l_expected clob; + l_not_expected clob; + l_actual clob; + begin + --Arrange + l_expected := '%' || + '%%'; + l_not_expected := '%' || + '%%'; + --Act + l_actual := + ut3_tester_helper.coverage_helper.run_tests_as_job( + q'[ + ut3_develop.ut.run( + a_paths => ut3_develop.ut_varchar2_list('ut3_develop.test_regex_dummy_cov', 'ut3_tester_helper.test_regex_dummy_cov'), + a_reporter=> ut3_develop.ut_coverage_sonar_reporter( ), + a_include_schema_expr => '^ut3_develop', + a_include_objects => ut3_develop.ut_varchar2_list( 'ut3_tester_helper.regex_dummy_cov' ) + ) + ]' + ); + --Assert + ut.expect(l_actual).to_be_like(l_expected); + ut.expect(l_actual).not_to_be_like(l_not_expected); + end; + + procedure coverage_regex_include_object is + l_expected clob; + l_not_expected clob; + l_actual clob; + begin + --Arrange + l_expected := '%' || + '%%'; + l_not_expected := '%' || + '%%'; + --Act + l_actual := + ut3_tester_helper.coverage_helper.run_tests_as_job( + q'[ + ut3_develop.ut.run( + a_paths => ut3_develop.ut_varchar2_list('ut3_develop.test_regex_dummy_cov', 'ut3_develop.test_regex123_dummy_cov'), + a_reporter=> ut3_develop.ut_coverage_sonar_reporter( ), + a_include_object_expr => 'regex123', + a_include_objects => ut3_develop.ut_varchar2_list( 'ut3_develop.regex_dummy_cov' ) + ) + ]' + ); + --Assert + ut.expect(l_actual).to_be_like(l_expected); + ut.expect(l_actual).not_to_be_like(l_not_expected); + end; + + procedure coverage_regex_exclude_schema is + l_expected clob; + l_not_expected clob; + l_actual clob; + begin + --Arrange + l_expected := '%' || + '%%'; + l_not_expected := '%' || + '%%'; + --Act + l_actual := + ut3_tester_helper.coverage_helper.run_tests_as_job( + q'[ + ut3_develop.ut.run( + a_paths => ut3_develop.ut_varchar2_list('ut3_develop.test_regex_dummy_cov', 'ut3_tester_helper.test_regex_dummy_cov'), + a_reporter=> ut3_develop.ut_coverage_sonar_reporter( ), + a_exclude_schema_expr => '^ut3_tester', + a_exclude_objects => ut3_develop.ut_varchar2_list( 'ut3_develop.regex_dummy_cov' ) + ) + ]' + ); + --Assert + ut.expect(l_actual).to_be_like(l_expected); + ut.expect(l_actual).not_to_be_like(l_not_expected); + end; + + procedure coverage_regex_exclude_object is + l_expected clob; + l_not_expected clob; + l_actual clob; + begin + --Arrange + l_expected := '%' || + '%%'; + l_not_expected := '%' || + '%%'; + --Act + l_actual := + ut3_tester_helper.coverage_helper.run_tests_as_job( + q'[ + ut3_develop.ut.run( + a_paths => ut3_develop.ut_varchar2_list('ut3_develop.test_regex_dummy_cov', 'ut3_develop.test_regex123_dummy_cov'), + a_reporter=> ut3_develop.ut_coverage_sonar_reporter( ), + a_exclude_object_expr => 'regex123', + a_exclude_objects => ut3_develop.ut_varchar2_list( 'ut3_develop.regex_dummy_cov' ) + ) + ]' + ); + --Assert + ut.expect(l_actual).to_be_like(l_expected); + ut.expect(l_actual).not_to_be_like(l_not_expected); + end; + + procedure cove_rgx_inc_schema_norun is + l_expected clob; + l_not_expected clob; + l_actual clob; + begin + --Arrange + l_expected := '%UT3_DEVELOP.REGEX_DUMMY_COV%'; + l_not_expected := '%UT3_TESTER_HELPER.REGEX123_DUMMY_COV%'; + --Act + l_actual := ut3_tester_helper.coverage_helper.gather_coverage_on_coverage( + q'[ut3_develop.ut_coverage_options( + coverage_run_id => l_coverage_run_id, + include_schema_expr => '^ut3_develop' + )]'); + --Assert + ut.expect(l_actual).to_be_like(l_expected); + ut.expect(l_actual).not_to_be_like(l_not_expected); + end; + + procedure cove_rgx_inc_object_norun is + l_expected clob; + l_not_expected clob; + l_actual clob; + begin + --Arrange + l_expected := '%UT3_DEVELOP.REGEX123_DUMMY_COV%'; + l_not_expected := '%UT3_DEVELOP.REGEX_DUMMY_COV%'; + --Act + l_actual := ut3_tester_helper.coverage_helper.gather_coverage_on_coverage( + q'[ut3_develop.ut_coverage_options( + coverage_run_id => l_coverage_run_id, + include_object_expr => 'regex123' + )]'); + --Assert + ut.expect(l_actual).to_be_like(l_expected); + ut.expect(l_actual).not_to_be_like(l_not_expected); + end; + + procedure cove_rgx_exc_schema_norun is + l_expected clob; + l_not_expected clob; + l_actual clob; + begin + --Arrange + l_expected := '%UT3_DEVELOP.REGEX_DUMMY_COV%'; + l_not_expected := '%UT3_TESTER_HELPER.REGEX_DUMMY_COV%'; + --Act + l_actual := ut3_tester_helper.coverage_helper.gather_coverage_on_coverage( + q'[ut3_develop.ut_coverage_options( + coverage_run_id => l_coverage_run_id, + schema_names => ut3_develop.ut_varchar2_rows( 'UT3_DEVELOP','UT3_TESTER_HELPER' ), + exclude_schema_expr => '^ut3_tester', + exclude_objects => ut3_develop.ut_varchar2_rows( 'ut3_develop.regex_dummy_cov' ) + )]'); + --Assert + ut.expect(l_actual).to_be_like(l_expected); + ut.expect(l_actual).not_to_be_like(l_not_expected); + end; + + procedure cove_rgx_exc_object_norun is + l_expected clob; + l_not_expected clob; + l_actual clob; + begin + --Arrange + l_expected := '%UT3_DEVELOP.REGEX_DUMMY_COV%'; + l_not_expected := '%UT3_DEVELOP.REGEX123_DUMMY_COV%'; + --Act + l_actual := ut3_tester_helper.coverage_helper.gather_coverage_on_coverage( + q'[ut3_develop.ut_coverage_options( + coverage_run_id => l_coverage_run_id, + schema_names => ut3_develop.ut_varchar2_rows( 'UT3_DEVELOP' ), + exclude_object_expr => 'regex123', + exclude_objects => ut3_develop.ut_varchar2_rows( 'ut3_develop.regex_dummy_cov' ) + )]'); + --Assert + ut.expect(l_actual).to_be_like(l_expected); + ut.expect(l_actual).not_to_be_like(l_not_expected); + end; + + procedure cov_file_mapping_no_utrun is + l_expected clob; + l_not_expected clob; + l_actual clob; + begin + --Arrange + l_expected := '%UT3_DEVELOP.REGEX_DUMMY_COV%'; + --Act + l_actual := ut3_tester_helper.coverage_helper.gather_coverage_on_coverage( + q'[ut3_develop.ut_coverage_options( + coverage_run_id => l_coverage_run_id, + file_mappings => ut3_develop.ut_file_mappings( ut3_develop.ut_file_mapping('C:\tests\helpers\core.pkb','UT3_DEVELOP','REGEX_DUMMY_COV','PACKAGE BODY')) + )]'); + --Assert + ut.expect(l_actual).to_be_like(l_expected); + end; + end; / diff --git a/test/ut3_user/reporters/test_coverage/test_extended_coverage.pks b/test/ut3_user/reporters/test_coverage/test_extended_coverage.pks index c575c240c..82b19fbed 100644 --- a/test/ut3_user/reporters/test_coverage/test_extended_coverage.pks +++ b/test/ut3_user/reporters/test_coverage/test_extended_coverage.pks @@ -17,6 +17,50 @@ create or replace package test_extended_coverage is --%tags(#1097,#1094) --%test(Extended coverage does not fail the test run then tested code calls DBMS_STATS) procedure coverage_with_dbms_stats; - + + --%beforetest(ut3_tester_helper.coverage_helper.create_regex_dummy_cov) + --%aftertest(ut3_tester_helper.coverage_helper.drop_regex_dummy_cov) + --%test(Collect coverage for objects with schema regex include with ut_run) + procedure coverage_regex_include_schema; + + --%beforetest(ut3_tester_helper.coverage_helper.create_regex_dummy_cov) + --%aftertest(ut3_tester_helper.coverage_helper.drop_regex_dummy_cov) + --%test(Collect coverage for objects with object regex include with ut_run) + procedure coverage_regex_include_object; + + --%beforetest(ut3_tester_helper.coverage_helper.create_regex_dummy_cov) + --%aftertest(ut3_tester_helper.coverage_helper.drop_regex_dummy_cov) + --%test(Collect coverage for objects with schema regex exclude with ut_run) + procedure coverage_regex_exclude_schema; + + --%beforetest(ut3_tester_helper.coverage_helper.create_regex_dummy_cov) + --%aftertest(ut3_tester_helper.coverage_helper.drop_regex_dummy_cov) + --%test(Collect coverage for objects with object regex exclude with ut_run) + procedure coverage_regex_exclude_object; + + --%beforetest(ut3_tester_helper.coverage_helper.create_regex_dummy_cov) + --%aftertest(ut3_tester_helper.coverage_helper.drop_regex_dummy_cov) + --%test(Collect coverage for objects with schema regex include without ut_run) + procedure cove_rgx_inc_schema_norun; + + --%beforetest(ut3_tester_helper.coverage_helper.create_regex_dummy_cov) + --%aftertest(ut3_tester_helper.coverage_helper.drop_regex_dummy_cov) + --%test(Collect coverage for objects with object regex include without ut_run) + procedure cove_rgx_inc_object_norun; + + --%beforetest(ut3_tester_helper.coverage_helper.create_regex_dummy_cov) + --%aftertest(ut3_tester_helper.coverage_helper.drop_regex_dummy_cov) + --%test(Collect coverage for objects with schema regex exclude without ut_run) + procedure cove_rgx_exc_schema_norun; + + --%beforetest(ut3_tester_helper.coverage_helper.create_regex_dummy_cov) + --%aftertest(ut3_tester_helper.coverage_helper.drop_regex_dummy_cov) + --%test(Collect coverage for objects with object regex exclude without ut_run) + procedure cove_rgx_exc_object_norun; + + --%beforetest(ut3_tester_helper.coverage_helper.create_regex_dummy_cov) + --%aftertest(ut3_tester_helper.coverage_helper.drop_regex_dummy_cov) + --%test(Collect coverage for file mapping without ut_run) + procedure cov_file_mapping_no_utrun; end; / diff --git a/test/ut3_user/reporters/test_coverage/test_proftab_coverage.pkb b/test/ut3_user/reporters/test_coverage/test_proftab_coverage.pkb index 7256fc202..7cad0a67e 100644 --- a/test/ut3_user/reporters/test_coverage/test_proftab_coverage.pkb +++ b/test/ut3_user/reporters/test_coverage/test_proftab_coverage.pkb @@ -137,14 +137,14 @@ create or replace package body test_proftab_coverage is --Act insert into test_results - select * + select rownum, x.* from table( ut3_develop.ut.run( a_path => 'ut3_develop:coverage_testing', a_reporter=> ut3_develop.ut_coverage_sonar_reporter( ), a_include_objects => ut3_develop.ut_varchar2_list( 'ut3_develop.{p}' ) ) - ); + ) x; commit; end; ]';