Updated exception-reporting.

utPLSQL · jgebal · Feb 24, 2017 · Feb 4, 2017 · Feb 4, 2017 · Feb 5, 2017
commit ee89c42d64528a201037896a06378bee22f25c4e
diff --git a/docs/userguide/exception-reporting.md b/docs/userguide/exception-reporting.md
@@ -1,65 +1,42 @@
 # Exception handling and reporting
 
-The utPLSQL is responsibly for handling exceptions wherever they occure in the test run. Exceptions are pororted as follows:
+The utPLSQL is responsible for handling exceptions wherever they occur in the test run. utPLSQL is trapping most of the exceptions so that the test execution is not affected by individual tests or test packages throwing an exception.
+The framework provides a full stacktrace for every exception that was thrown. The stacktrace is clean and does not include any utPLSQL library calls in it.
+To achieve rerunability, the ORA-04068, ORA-04061 exceptions are not handled and test execution will be interrupted if such exception is encountered. This is because of how Oracle behaves on those exceptions.
 
-* A test package without body faced - each `%test` is reported as failed
-* A test package with _invalid body_ - each `%test` is reported as failed
-* A test package with _invalid spec_ - *`%test`s are skipped*. Only valid specifications are parsed for annotations.
-* A test package that is raising an exception in beforeall - each `%test` is reported as failed
-* A test package that is raising an exception in afterall - `%test` are reported normally, warnings are displayed in the summary
+Test execution can fail for different reasons. The failures on different exceptions are handled as follows:
+* A test package without body - each `%test` is reported as failed with exception, nothing is executed
+* A test package with _invalid body_ - each `%test` is reported as failed with exception, nothing is executed
+* A test package with _invalid spec_ - package is not considered a valid unit test package and is excluded from execution. When trying to run a test package with invalid spec explicitly, exception is raised. Only valid specifications are parsed for annotations 
+* A test package that is raising an exception in `%beforeall` - each `%test` is reported as failed with exception, `%test`, `%beforeeach`, `%beforetest`, `%aftertest` and `%aftereach` are not executed. `%afterall` is executed to allow cleanup of whatever was done in `%beforeall`
+* A test package that is raising an exception in `%beforeeach` - each `%test` is reported as failed with exception, `%test`, `%beforetest` and `%aftertest` is not executed. The `%aftereach` and `%afterall` blocks are getting executed to allow cleanup of whatever was done in `%before...` blocks
+* A test package that is raising an exception in `%beforetest` - the `%test` is reported as failed  with exception, `%test` is not executed. The `%aftertest`, `%aftereach` and `%afterall` blocks are getting executed to allow cleanup of whatever was done in `%before...` blocks
+* A test package that is raising an exception in `%test` - the `%test` is reported as failed with exception. The execution of other blocks continues normally
+* A test package that is raising an exception in `%aftertest` - the `%test` is reported as failed with exception. The execution of other blocks continues normally
+* A test package that is raising an exception in `%aftereach` - all blocks of the package are executed, as ehe `%aftereach` is a closing block for an individual test. Exception in `%aftereach` is not affecting test results. For every failed execution of `%aftereach` a warning with exception stacktrace is displayed in the summary
+* A test package that is raising an exception in `%afterall` - all blocks of  the package are executed, as the `%afterall` is the last step of package execution. Exception in `%afterall` is not affecting test results. A warning with exception stacktrace is displayed in the summary
 
-Example:
-```
-Remove rooms by name
-  Removes a room without content in it
-  Does not remove room when it has content
-  Raises exception when null room name given
-
-Warnings:
-
-  1) test_remove_rooms_by_name - Afterall procedure failed: 
-       ORA-20001: Test exception
-       ORA-06512: at "UT3.TEST_REMOVE_ROOMS_BY_NAME", line 35
-       ORA-06512: at line 6
-
-Finished in ,044902 seconds
-3 tests, 0 failed, 0 errored, 0 ignored. 1 warning(s)
-```
 
-* A test package that is raising an exception in `%beforeeach` - each `%test` is reported as failed
-* A test package that is raising an exception in `%aftereach` - `%test`s are reported normally, warnings are displayed in the summary
-
-Example:
-```
+Example of reporting with exception thrown in `%beforetest`:
+````
 Remove rooms by name
-  Removes a room without content in it
+  Removes a room without content in it (FAILED - 1)
   Does not remove room when it has content
   Raises exception when null room name given
 
-Warnings:
-
-  1) test_remove_rooms_by_name - Aftereach procedure failed:
-       ORA-20001: Test exception
-       ORA-06512: at "UT3.TEST_REMOVE_ROOMS_BY_NAME", line 31
-       ORA-06512: at line 6
-
-  2) test_remove_rooms_by_name - Aftereach procedure failed:
-       ORA-20001: Test exception
-       ORA-06512: at "UT3.TEST_REMOVE_ROOMS_BY_NAME", line 31
-       ORA-06512: at line 6
-
-  3) test_remove_rooms_by_name - Aftereach procedure failed:
-       ORA-20001: Test exception
-       ORA-06512: at "UT3.TEST_REMOVE_ROOMS_BY_NAME", line 31
-       ORA-06512: at line 6
+Failures:
 
-Finished in ,05071 seconds
-3 tests, 0 failed, 0 errored, 0 ignored. 3 warning(s)
-```
-
-* A test package that is raising an exception in test - the `%test` is reported as failed
+  1) remove_empty_room
+
+        error: ORA-20001: Test exception
+               ORA-06512: at "UT3.TEST_REMOVE_ROOMS_BY_NAME", line 39
+               ORA-06512: at line 6
+
+Finished in ,039346 seconds
+3 tests, 0 failed, 1 errored, 0 ignored.
+````
 
-Exampple:
+Example of reporting with exception thrown in `%test`:
 ```
 Remove rooms by name
   Removes a room without content in it (FAILED - 1)
@@ -78,10 +55,8 @@ Finished in ,035726 seconds
 3 tests, 0 failed, 1 errored, 0 ignored.
 ```
 
-* A test package that is raising an exception in `%beforetest` - the `%test` is reported as failed
-
-Example:
-````
+Example of reporting with exception thrown in `%aftertest`:
+```
 Remove rooms by name
   Removes a room without content in it (FAILED - 1)
   Does not remove room when it has content
@@ -92,30 +67,55 @@ Failures:
   1) remove_empty_room
 
         error: ORA-20001: Test exception
-               ORA-06512: at "UT3.TEST_REMOVE_ROOMS_BY_NAME", line 39
+               ORA-06512: at "UT3.TEST_REMOVE_ROOMS_BY_NAME", line 42
                ORA-06512: at line 6
 
-Finished in ,039346 seconds
+Finished in ,045523 seconds
 3 tests, 0 failed, 1 errored, 0 ignored.
-````
+```
 
-* A test package that is raising an exception in `%aftertest` - the `%test` is reported as failed
+Example of reporting with exception thrown in `%aftereach`:
+```
+Remove rooms by name
+  Removes a room without content in it
+  Does not remove room when it has content
+  Raises exception when null room name given
+
+Warnings:
+
+  1) test_remove_rooms_by_name - Aftereach procedure failed:
+       ORA-20001: Test exception
+       ORA-06512: at "UT3.TEST_REMOVE_ROOMS_BY_NAME", line 31
+       ORA-06512: at line 6
+
+  2) test_remove_rooms_by_name - Aftereach procedure failed:
+       ORA-20001: Test exception
+       ORA-06512: at "UT3.TEST_REMOVE_ROOMS_BY_NAME", line 31
+       ORA-06512: at line 6
+
+  3) test_remove_rooms_by_name - Aftereach procedure failed:
+       ORA-20001: Test exception
+       ORA-06512: at "UT3.TEST_REMOVE_ROOMS_BY_NAME", line 31
+       ORA-06512: at line 6
+
+Finished in ,05071 seconds
+3 tests, 0 failed, 0 errored, 0 ignored. 3 warning(s)
+```
 
-Example:
+Example of reporting with exception thrown in `%afterall`:
 ```
 Remove rooms by name
-  Removes a room without content in it (FAILED - 1)
+  Removes a room without content in it
   Does not remove room when it has content
   Raises exception when null room name given
 
-Failures:
+Warnings:
 
-  1) remove_empty_room
-
-        error: ORA-20001: Test exception
-               ORA-06512: at "UT3.TEST_REMOVE_ROOMS_BY_NAME", line 42
-               ORA-06512: at line 6
-
-Finished in ,045523 seconds
-3 tests, 0 failed, 1 errored, 0 ignored.
+  1) test_remove_rooms_by_name - Afterall procedure failed: 
+       ORA-20001: Test exception
+       ORA-06512: at "UT3.TEST_REMOVE_ROOMS_BY_NAME", line 35
+       ORA-06512: at line 6
+
+Finished in ,044902 seconds
+3 tests, 0 failed, 0 errored, 0 ignored. 1 warning(s)
 ```