TensorForce has the ability to record summary data for use with TensorBoard as well STDIO and file export. This is accomplished through dictionary parameter called "summary_spec" passed to the agent on initialization.
"summary_spec" supports the following optional dictionary entries:
+--------------+------------------------------------------------------------+
| Key | Value |
+==============+============================================================+
| directory | (str) Path to storage for TensorBoard summary data |
+--------------+------------------------------------------------------------+
| steps | (int) Frequency in steps between storage of summary data |
+--------------+------------------------------------------------------------+
| seconds | (int) Frequency in seconds to store summary data |
+--------------+------------------------------------------------------------+
| labels | (list) Requested Export, See "*LABELS*" section |
+--------------+------------------------------------------------------------+
| meta\_dict | (dict) For used with label "configuration" |
+--------------+------------------------------------------------------------+
+------------------------+---------------------------------------------------------+
| Entry | Data produced |
+========================+=========================================================+
| losses | Training total-loss and "loss-without-regularization" |
+------------------------+---------------------------------------------------------+
| total-loss | Final calculated loss value |
+------------------------+---------------------------------------------------------+
| variables | Network variables |
+------------------------+---------------------------------------------------------+
| inputs | Equivalent to: ['states', 'actions', 'rewards'] |
+------------------------+---------------------------------------------------------+
| states | Histogram of input state space |
+------------------------+---------------------------------------------------------+
| actions | Histogram of input action space |
+------------------------+---------------------------------------------------------+
| rewards | Histogram of input reward space |
+------------------------+---------------------------------------------------------+
| gradients | Histogram and scalar gradients |
+------------------------+---------------------------------------------------------+
| gradients\_histogram | Variable gradients as histograms |
+------------------------+---------------------------------------------------------+
| gradients\_scalar | Variable Mean/Variance of gradients as scalar |
+------------------------+---------------------------------------------------------+
| regularization | Regularization values |
+------------------------+---------------------------------------------------------+
| **configuration** | See *Configuration Export* for more detail |
+------------------------+---------------------------------------------------------+
| configuration | Export configuration to "TEXT" tab in TensorBoard |
+------------------------+---------------------------------------------------------+
| print\_configuration | Prints configuration to STDOUT |
+------------------------+---------------------------------------------------------+
from tensorforce.agents import PPOAgent
# Create a Proximal Policy Optimization agent
agent = PPOAgent(
states_spec=...,
actions_spec=...,
network_spec=...,
summary_spec=dict(directory="./board/",
steps=50,
labels=['configuration',
'gradients_scalar',
'regularization',
'inputs',
'losses',
'variables']
),
...
)Adding the "configuration" label will create a "TEXT" tab in TensorBoard that contains all the parameters passed to the Agent. By using the additional "summary_spec" dictionary key "meta_dict", custom keys and values can be added to the data export. The user may want to pass "Description", "Experiement #", "InputDataSet", etc.
If a key is already in use within TensorForce an error will be raised to notify you to change the key value. To use the custom feature, create a dictionary with keys to export:
from tensorforce.agents import PPOAgent
metaparams['MyDescription'] = "This experiment covers the first test ...."
metaparams['My2D'] = np.ones((9,9)) # 9x9 matrix of 1.0's
metaparams['My1D'] = np.ones((9)) # Column of 9 1.0's
# Create a Proximal Policy Optimization agent
agent = PPOAgent(
states_spec=...,
actions_spec=...,
network_spec=...,
summary_spec=dict(directory="./board/",
steps=50,
meta_dict=metaparams, #Add custom keys to export
labels=['configuration',
'gradients_scalar',
'regularization',
'inputs',
'losses',
'variables']
),
...
)Use the "print_configuration" label to export the configuration data to the command line's STDOUT.