Skip to content

Optimizations to the streaming of MCTruthContainer - #2151

Merged
matthiasrichter merged 2 commits into
AliceO2Group:devfrom
matthiasrichter:dev-mclabelcontainer
Sep 6, 2019
Merged

Optimizations to the streaming of MCTruthContainer#2151
matthiasrichter merged 2 commits into
AliceO2Group:devfrom
matthiasrichter:dev-mclabelcontainer

Conversation

@matthiasrichter

@matthiasrichter matthiasrichter commented Jul 1, 2019

Copy link
Copy Markdown
Collaborator

The MCLabelContainer has two vectors of messageable type which do not need to be root serialized.
For the moment, flattening always requires a copy, but the data can be directly copied
to the message resource. The prototype can be extended later to use multiple parts as allowed by the
DataHeader.

Restore can be done using spectator memory resources for the two regions of the input buffer, but this kind of memory resource needs to be implemented. For the moment we have a copy also here.

The DPL I/O API will be extended to use the custom methods for flatten/restore. In the end its not much different then the BOOST serialization concept, but we have the option to optimize this by using polymorphic memory resources and allocators.

TODO before merging:

  • find names for flatten/restore methods -> its good enough
  • check the necessary changes in the DPL I/O API (we might want to sort out DPL: streamline DataAllocator::make #1970 before) - DPL IO API has been updated and a custom implementation for MCTruthContainer will be implemented soon
  • alternatively check if the two vectors can be sent in individual messages, o2::pmr::MessageResouce could be used directly -> flattening can be done directly to buffer with underlying memory resource

This concept can be extended to a generic tool later.

Next developments which will not be part of this PR

  • add benchmark test for MCTruthContainer serialization
  • develop "spectator memory resouces", may be several of this kind working over the memory of of the o2::pmr::MessageResouce

@davidrohr

davidrohr commented Jul 1, 2019 via email

Copy link
Copy Markdown
Collaborator

@matthiasrichter

Copy link
Copy Markdown
Collaborator Author

Hi David, I'm thinking in a similar direction. For the moment I'm planning for a prototype to initiate the discussion. I had a look to the FlatObject interface, but not to all the details, not 100% sure if this is the best interface for this use case. I want to understand it, do you have a small example somewhere? I have more a templated approach in mind and some static code analysis.

@matthiasrichter
matthiasrichter requested review from ktf and sawenzel July 1, 2019 11:29
@davidrohr

Copy link
Copy Markdown
Collaborator

Hi @matthiasrichter : for examples, please check GPU/TPCFastTransformation/TPCFastTransform.h or or Detectors/Base/include/.../MatLayerCylSet.h

The FlatObject is actually not an interface for polymorphic allocation. Its main purpose is exposing the setActualBufferPtr and setFutureBufferPtr methods, as well as the getFlatBufferSize and getFlatBufferPtr, which can be used to ship it into preallocated memory on the GPU.

In addition, it can be root-serializable if one calls setActualBufferAddress after reading from file.

My idea is to complement (not replace) this with the polymorphic allocator, which should be rather trivial. Then, the flat objects could do both: being shipped in DPL with polymorphic allocators, and be moved to GPU with the current interface.

@sawenzel

sawenzel commented Sep 3, 2019

Copy link
Copy Markdown
Collaborator

Is this still being worked on or do we rely on other solutions?

Adding flatten_to and restore_from methods which allow to flatten to a container
instance and restore the vector content from a buffer, both by copy. This can
be used with a container with pmr allocator and underlying memory resource.

Flattening always requires a copy, but the data can be directly copied
to the message resource. Restore can be done using spectator memory
resources for the two regions of the input buffer, but this kind of
memory resource needs to be implemented.
Using custom streamer for MCTruthContainer<MCCompLabel> and flatten the two
vectors into one additional char vector member before serialization. The original
vectors are empty during serialization. When reading an object, the vectors are
restored from the char vector which has been streamed.

The reduced serialization time for one single char vector and two empty vectors outweighs
the cost of flattening and restoring. There is a significant speedup in the serialization.

As the next step, for transmitting messages the serialization can be skipped completely
by a custom DPL IO interface implementation for MCTruthContainer.

TODO:
- add move assignment from a source vector, by that passing an object which has access to
  different underlying memory resources, until that, the pmr::MemoryResource has been
  removed again
- add interface to access header and truth elements directly from the raw buffer, by that
  inflation can be postponed until new elements are added, with the effect that inflation
  can be avoided in most cases. This avoids copy for deserialized read-only objects.
@matthiasrichter

Copy link
Copy Markdown
Collaborator Author

I did many tests during the last week, the newly pushed update is a condensed result. Main results:

  • significant speedup of serialization of MCTruthContainer
  • significant reduction of the size of serialized MCTruthContainer for bit data sets, smaller by factor 3 to 4 now.
  • the inefficiencies of the ROOT serialization for large data sets is handled tranparently

Details:
In addition to the flatten and restore methods there is some optimization applied to the streaming. Using a custom streamer function, the two vectors are flattened into a third char vector, which is then serialized while the two original ones are left empty. This is much more efficient, the faster serialization of one char vector outweighs the overhead for flattening and restoring. A simple workflow reading TPC digits and MC labels, passing it through the clusterer and writing to raw data file is 1/3 faster with this approach. The size is significantly reduced by factor 3-4, the size corresponds now to what can be calculated from the vector sizes and their member sizes. A benchmark test needs to be added, though.

Unfortunately ROOT schema evolution does not work correctly if the two original vectors are marked transient, which would be in principle possible. The ,from a read pragma, generated conversion function does not access the members of the old object correctly. The workaround is thus to keep the original members and leave them empty for serialization.

More to come: During my tests I have also created a direct access to the flattened buffer, so that read-only access to a deserialized object can be even more efficient. Code has to get finalized though, leaving this for a later optimization.

Once we have a pmr memory resource working as spectator over message buffer, we can extract a vector and pass it directly to MCTruthContainer, making serialization completely unnecessary.

@matthiasrichter

Copy link
Copy Markdown
Collaborator Author

@shahor02 @davidrohr can you please check the tracking benchmarks with this update. Is there a procedure and some tests I can run myself to verify the changes?

@matthiasrichter

Copy link
Copy Markdown
Collaborator Author

@davidrohr we can do the conversion to FlatObject later, this is not a quick task and I prefer to push forward this PR in the current version to get the improvements quickly into the simulation end reconstruction.

@shahor02 shahor02 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @matthiasrichter

I run the its and tpc tracking with your branch and dev and the results are exactly the same, both for the reconstruction and for assigned MClables.

Cheers,
Ruben

@davidrohr davidrohr left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK with me (Ruben was faster with his test), the minor comment can be addressed when we integrate FlatObject.

#define ALICEO2_DATAFORMATS_MCTRUTH_H_

#include <TNamed.h>
#include <TNamed.h> // to have the ClassDef macros

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For later when we port to FlatObject and make it compatible to GPU, we should use GPUCommonRtypes.h for ClassDef macro, instead of the ROOT header.

Comment thread DataFormats/simulation/src/SimulationDataLinkDef.h
@matthiasrichter

Copy link
Copy Markdown
Collaborator Author

thanks for the feedback!

@matthiasrichter matthiasrichter changed the title [WIP] Adding custom flatten/restore functionality to MCLabelContainer Adding custom flatten/restore functionality to MCLabelContainer Sep 6, 2019
@matthiasrichter matthiasrichter changed the title Adding custom flatten/restore functionality to MCLabelContainer Optimizations to the streaming of MCTruthContainer Sep 6, 2019
@matthiasrichter

Copy link
Copy Markdown
Collaborator Author

then we are ready to go and I leave the open items as task list for future extensions.

@shahor02

shahor02 commented Sep 6, 2019

Copy link
Copy Markdown
Collaborator

@matthiasrichter concerning the custom streamers of MCTruthContainer<MCCompLabel>: note that most of detectors (essentially all except TPC and ITS) are defining their own labels derived from the MCCompLabel. So, currently only TPC and ITS will profit from this PR. I am wondering if you can define custom streamer w/o detalization?

@matthiasrichter

Copy link
Copy Markdown
Collaborator Author

Yes, I have this on the radar. We actually need to implement for every specialization. This is because ClassDefNV defines the Streamer function in the header. That's why I could not implement a generic method. Maybe it's ok because one anyhow has to have the specialization in the LinkDef with tag -.

Alternatively we can skip ClassDefNV macro and define the things from the macro direrectly in the header.

EmilGorm pushed a commit to EmilGorm/AliceO2 that referenced this pull request Apr 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

4 participants