Executable:

o2-dpl-raw-proxy

A proxy workflow to connect to a channel from DataDistribution. The incoming data is supposed to follow the O2 data model with header-payload pair messages. The header message must contain DataHeader and the internal DPL DataProcessingHeader. Only data which match the configured outputs will be forwarded.

Since payload can be split into several messages, the proxy will create a new message to merge all parts belonging to one payload entity.

o2-dpl-raw-proxy --dataspec "0:FLP/RAWDATA" --channel-config "name=readout-proxy,type=pair,method=bind,address=ipc:///tmp/stf-builder-dpl-pipe-0,transport=shmem,rateLogging=1"

Options:

  --dataspec arg (=A:FLP/RAWDATA;B:FLP/DISTSUBTIMEFRAME/0)
                                        selection string for the data to be 
                                        proxied
  --throwOnUnmatched                    throw if unmatched input data is found

Data is selected by using a configuration string of the format tag1:origin/description/subspec;tag2:..., the tags are arbitrary and only for internal reasons and are ignored. Example:

--dataspec 'A:FLP/RAWDATA;B:FLP/DISTSUBTIMEFRAME/0'

The input channel has name readout-proxy and is configured with option --channel-config.

By default, data blocks not matching the configured filter/output spec will be silently ignored. Use --throwOnUnmatched to apply a strict checking in order to indicate unmatched data.

The proxy workflow only defines the proxy process, workflows can be combined by piping them together om the command line

o2-dpl-raw-proxy --dataspec "0:FLP/RAWDATA" --channel-config "..." | the-subscribing-workflow

• stress test with data not following the O2 data model
• forwarding of shared memory messages
• vector handling of split payload parts in DPL
• make the channel config more convenient, do not require the full string and use defaults, e.g. for the name (which can not be changed anyhow)
• check if the channel config string can be parsed omitting tags, right now this is only possible for the first entry

Class RawParser implements a parser for O2 raw data which is organized in pages of a fixed size, each page starts with the RAWDataHeader. The page size may actually be smaller than the maximum size, depending on the header fields.

The parser class works on a contiguous sequence of raw pages in a raw buffer. Multiple versions of RAWDataHeader are supported transparently and selected depending on the version field of the header.

// option 1: parse method
RawParser parser(buffer, size);
auto processor = [&count](auto data, size_t length) {
  std::cout << "Processing block of length " << length << std::endl;
};
parser.parse(processor);

// option 2: iterator
RawParser parser(buffer, size);
for (auto it = parser.begin(), end = parser.end(); it != end; ++it, ++count) {
  std::cout << "Iterating block of length " << it.length() << std::endl;
  auto dataptr = it.data();
}

Executable:

o2-dpl-raw-parser

Parser workflow defines one process with configurable inputs and a loop over the input data which sets up the RawParser class and executes some basic parsing and statistic printout.

o2-dpl-raw-parser --input-spec "A:FLP/RAWDATA"

Options:

  --input-spec arg (=A:FLP/RAWDATA)

The workflow can be connected to the o2-dpl-raw-proxy workflow using pipe on command line.

Executable:

o2-dpl-output-proxy

A proxy workflow to connect to workflow output and forward it to out-of-band channels, meaning channels outside DPL.

o2-dpl-output-proxy --dataspec "channelname:TPC/TRACKS" --channel-config name=channelname,...

Output channel(s): to be defined with the FairMQ device channel configuration option, e.g.

--channel-config name=downstream,type=push,method=bind,address=icp://localhost_4200,rateLogging=60,transport=shmem

Input to the proxy is defined by the DPL data spec syntax

binding1:origin1/description1/subspecification1;binding2:origin2/descritption2;...

The binding label can be used internally to access the input, here it is used to match to a configured output channel. That binding can be the same for multiple data specs.

Options:

  --dataspec       specs           Data specs of data to be proxied
  --channel-config config          FairMQ device channel configuration for the output channel
  --proxy-name     name            name of the proxy processor, used also as default output channel name
  --default-transport arg (=shmem) default transport: shmem, zeromq
  --default-port arg (=4200)       default port number

Note: the 'default-*' options are only for building the the default of the channel configuration. The default channel name is build from the name of the proxy device. Having a default channel configuration allows to skip the '--channel-config' option on the command line.

documentation to be filled