UG: Documented new --pre(run|rebot)modifier options (robotframework#1976).

pekkaklarck · pekkaklarck · commit b26a664c21e4 · 2015-06-04T23:10:41.000+03:00
Also enhanced documentation related to imporing libraries that was
referenced by the new docs.
diff --git a/doc/api/code_examples/select_every_xth_test.py b/doc/api/code_examples/select_every_xth_test.py
@@ -2,6 +2,7 @@
 
 
 class SelectEveryXthTest(SuiteVisitor):
+    """Visitor that keeps only every Xth test in the visited suite structure."""
 
     def __init__(self, x, start=0):
         self.x = int(x)
diff --git a/doc/userguide/src/Appendices/AvailableSettings.rst b/doc/userguide/src/Appendices/AvailableSettings.rst
@@ -20,7 +20,7 @@ importing libraries, resources, and variables.
    +-----------------+--------------------------------------------------------+
    |       Name      |                         Description                    |
    +=================+========================================================+
-   | Library         | Used for `taking test libraries into use`_.            |
+   | Library         | Used for `importing libraries`_.                       |
    +-----------------+--------------------------------------------------------+
    | Resource        | Used for `taking resource files into use`_.            |
    +-----------------+--------------------------------------------------------+
diff --git a/doc/userguide/src/Appendices/CommandLineOptions.rst b/doc/userguide/src/Appendices/CommandLineOptions.rst
@@ -68,6 +68,8 @@ Command line options for test execution
   --exitonerror           `Stops test execution <Stopping on parsing or execution error_>`__
                           if any error occurs when parsing test data, importing libraries, and so on.
   --skipteardownonexit    `Skips teardowns`_ is test execution is prematurely stopped.
+  --prerunmodifier <name:args>    Activate `programmatic modification of test data`_.
+  --prerebotmodifier <name:args>  Activate `programmatic modification of results`_.
   --randomize <all|suites|tests|none>  `Randomizes`_ test execution order.
   -W, --monitorwidth <chars>  `Sets the width`_ of the console output.
   -C, --monitorcolors <on|off|force>  `Specifies are colors`_ used on the console.
@@ -127,6 +129,8 @@ Command line options for post-processing outputs
                           in test cases. Error codes are returned normally.
   --processemptysuite     Processes output files even if files contain
                           `empty test suites`_.
+  --prerebotmodifier <name:args>  Activate `programmatic modification of results`_.
+  -P, --pythonpath <path>   Additional locations to add to the `module search path`_.
   -E, --escape <what:with>  `Escapes characters`_ that are problematic in the console.
   -A, --argumentfile <path>   A text file to `read more arguments`_ from.
   -h, --help              Prints `usage instructions`_.
diff --git a/doc/userguide/src/CreatingTestData/CreatingTestCases.rst b/doc/userguide/src/CreatingTestData/CreatingTestCases.rst
@@ -338,7 +338,7 @@ Where named arguments are supported
 '''''''''''''''''''''''''''''''''''
 
 As already explained, the named argument syntax works with keywords. In
-addition to that, it also works when `taking test libraries into use`_.
+addition to that, it also works when `importing libraries`_.
 
 Naming arguments is supported by `user keywords`_ and by most `test libraries`_.
 The only exception are Java based libraries that use the `static library API`_.
diff --git a/doc/userguide/src/CreatingTestData/UsingTestLibraries.rst b/doc/userguide/src/CreatingTestData/UsingTestLibraries.rst
@@ -13,14 +13,14 @@ section.
    :depth: 2
    :local:
 
-Taking test libraries into use
-------------------------------
+Importing libraries
+-------------------
 
-Instructions for taking test libraries into use are given in the
-subsections below.
+Test libraries are typically imported using the :setting:`Library` setting,
+but it is also possible to use the :name:`Import Library` keyword.
 
-Using Library setting
-~~~~~~~~~~~~~~~~~~~~~
+Using `Library` setting
+~~~~~~~~~~~~~~~~~~~~~~~
 
 Test libraries are normally imported using the :setting:`Library`
 setting in the Setting table and having the library name in the
@@ -56,8 +56,8 @@ cases, all the keywords in the imported library are available in that
 file. With resource files, those keywords are also available in other
 files using them.
 
-Using Import Library keyword
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Using `Import Library` keyword
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Another possibility to take a test library into use is using the
 keyword :name:`Import Library` from the BuiltIn_ library. This keyword
@@ -79,8 +79,16 @@ make it available.
    \            KW From MyLibrary  \           \           \
    ===========  =================  ==========  ==========  ==========
 
-Library search path
-~~~~~~~~~~~~~~~~~~~
+Specifying library to import
+----------------------------
+
+Libraries to import can be specified either by using the library name
+or the path to the library. These approaches work the same way regardless
+is the library imported using the :setting:`Library` setting or the
+:name:`Import Library` keyword.
+
+Using library name
+~~~~~~~~~~~~~~~~~~
 
 The most common way to specify a test library to import is using its
 name, like it has been done in all the examples in this section. In
diff --git a/doc/userguide/src/ExecutingTestCases/ConfiguringExecution.rst b/doc/userguide/src/ExecutingTestCases/ConfiguringExecution.rst
@@ -281,7 +281,7 @@ extensions are found is a requirement for successful test execution. If
 you need to customize it using approaches explained below, it is often
 a good idea to create a custom `start-up script`_.
 
-__ `Taking test libraries into use`_
+__ `Specifying library to import`_
 __ `Setting listeners`_
 
 Locations automatically in module search path
@@ -444,6 +444,57 @@ Examples::
 
 __ `Free test suite metadata`_
 
+Programmatic modification of test data
+--------------------------------------
+
+If the provided built-in features to modify test data before execution
+are not enough, Robot Framework 2.9 and newer provide a possible to do
+custom modifications programmatically. This is accomplished by creating
+a model modifier and activating it using the :option:`--prerunmodifier`
+option.
+
+Model modifiers should be implemented as visitors that can traverse through
+the executable test suite structure and modify it as needed. The visitor
+interface is explained as part of the `Robot Framework API documentation`__,
+and the example below ought to give an idea of how it can be used and how
+powerful this functionality is.
+
+.. sourcecode:: python
+
+   ../api/code_examples/select_every_xth_test.py
+
+When a model modifier is taken into use on the command line using the
+:option:`--prerunmodifier` option, it can be specified either as a name of
+the modifier class or a path to the modifier file. If the modifier is given
+as a class name, the module containing the class must be in the `module search
+path`_, and if the module name is different than the class name, the given
+name must include both like `module.ModifierClass`. If the modifier is given
+as a path, the class name must be same as the file name. For most parts this
+works exactly like when `specifying a test library to import`__.
+
+If a modifier requires arguments, like the example above does, they can be
+specified after the modifier name or path using either a colon (`:`) or a
+semicolon (`;`) as a separator. If both are used in the value, the one first
+is considered the actual separator.
+
+For example, if the above model modifier would be in a file
+:file:`SelectEveryXthTest.py`, it could be used like this::
+
+    # Specify the modifier as a path. Run every second test.
+    pybot --prerunmodifier path/to/SelectEveryXthTest.py:2 tests.robot
+
+    # Specify the modifier as a name. Run every third test, starting from the second.
+    # SelectEveryXthTest.py must be in the module search path.
+    pybot --prerunmodifier SelectEveryXthTest:3:1 tests.robot
+
+If more than one model modifier is needed, they can be specified by using
+the :option:`--prerunmodifier` option multiple times. If similar modifying
+is needed before creating results, `programmatic modification of results`_
+can be enabled using the :option:`--prerebotmodifier` option.
+
+__ https://robot-framework.readthedocs.org/en/latest/autodoc/robot.model.html#module-robot.model.visitor
+__ `Specifying library to import`_
+
 Controlling console output
 --------------------------
 
@@ -509,7 +560,8 @@ case-insensitive values:
 Setting listeners
 -----------------
 
-So-called listeners_ can be used for monitoring the test
-execution. They are taken into use with the command line option
-:option:`--listener`, and the specified listeners must be in the `module
-search path`_ similarly as test libraries.
+Listeners_ can be used to monitor the test execution. When they are taken into
+use from the command line, they are specified using the :option:`--listener`
+command line option. The value can either be a path to a listener or
+a listener name. See the `Using listener interface`_ section for more details
+about importing listeners and using them in general.
diff --git a/doc/userguide/src/ExecutingTestCases/OutputFiles.rst b/doc/userguide/src/ExecutingTestCases/OutputFiles.rst
@@ -575,6 +575,51 @@ Examples::
    rebot --starttime 20080611-175920 --endtime 20080611-180242 *.xml
    rebot --starttime 20110302-1317 --endtime 20110302-11418 myoutput.xml
 
+Programmatic modification of results
+------------------------------------
+
+If the provided built-in features to modify results are are not enough,
+Robot Framework 2.9 and newer provide a possible to do custom modifications
+programmatically. This is accomplished by creating a model modifier and
+activating it using the :option:`--prerebotmodifier` option.
+
+This functionality works nearly exactly like `programmatic modification of
+test data`_ that can be enabled with the :option:`--prerunmodifier` option.
+The only difference is that the modified model is Robot Framework's
+result model and not the executable test suite model. For example, the
+following modifier marks all passed tests that have taken more time than
+allowed as failed:
+
+.. sourcecode:: python
+
+    from robot.api import SuiteVisitor
+
+
+    class ExecutionTimeChecker(SuiteVisitor):
+
+        def __init__(self, max_seconds):
+            self.max_milliseconds = float(max_seconds) * 1000
+
+        def visit_test(self, test):
+            if test.status == 'PASS' and test.elapsedtime > self.max_milliseconds:
+                test.status = 'FAIL'
+                test.message = 'Test execution took too long.'
+
+If the above modifier would be in file :file:`ExecutionTimeChecker.py`, it
+could be used, for example, like this::
+
+    # Specify modifier as a path when running tests. Maximum time is 42 seconds.
+    pybot --prerebotmodifier path/to/ExecutionTimeChecker.py:42 tests.robot
+
+    # Specify modifier as a name when using Rebot. Maximum time is 3.14 seconds.
+    # ExecutionTimeChecker.py must be in the module search path.
+    rebot --prerebotmodifier ExecutionTimeChecker:3.14 output.xml
+
+If more than one model modifier is needed, they can be specified by using
+the :option:`--prerebotmodifier` option multiple times. When executing tests,
+it is possible to use :option:`--prerunmodifier` and
+:option:`--prerebotmodifier` options together.
+
 System log
 ----------
 
diff --git a/doc/userguide/src/ExtendingRobotFramework/CreatingTestLibraries.rst b/doc/userguide/src/ExtendingRobotFramework/CreatingTestLibraries.rst
@@ -1705,7 +1705,7 @@ Since libraries are normal programming code, they can be packaged
 using normal packaging tools. With Python, good options include
 distutils_, contained by Python's standard library, and the newer
 setuptools_. A benefit of these tools is that library modules are
-installed into a location that is automatically in the `library
+installed into a location that is automatically in the `module
 search path`_.
 
 When using Java, it is natural to package libraries into a JAR
diff --git a/src/robot/model/visitor.py b/src/robot/model/visitor.py
@@ -52,9 +52,9 @@
 Examples
 --------
 
-The following example modifies the test suite structure so that it keeps only
-every Xth test. It could be used, for example, with Robot Framework's
-``--prerunmodifier`` option to modify test data before execution.
+The following example visitor modifies the test suite structure it visits.
+It could be used, for example, with Robot Framework's ``--prerunmodifier``
+option to modify test data before execution.
 
 .. literalinclude:: /../../doc/api/code_examples/select_every_xth_test.py
 
