Skip to content

User hook (JITable) for track references during stepping#10085

Merged
sawenzel merged 2 commits into
AliceO2Group:devfrom
sawenzel:swenzel/trackrefHook
Oct 15, 2022
Merged

User hook (JITable) for track references during stepping#10085
sawenzel merged 2 commits into
AliceO2Group:devfrom
sawenzel:swenzel/trackrefHook

Conversation

@sawenzel

Copy link
Copy Markdown
Collaborator

This commit implements a generic and customizable mechanism to define and execute a callback during MC stepping. The purpose of the callback is to create TrackReferences for certain conditions, but can be used for other purposes (monitoring etc). The callback just takes the instance to TVirtualMC

The default implementation creates a TrackReference on "IsStop" whenever the current track already left a TrackReference previously.

The feature needs to be switched on via configurable Parameter

SimCutParams.stepTrackRefHook=1

Furthermore, the user can provide own callback implementations via a macro file "MyHook.C". The implementation should follow the following pattern:

o2::steer::O2MCApplicationBase::TrackRefFcn trackRefHook() {                                                                               return [](TVirtualMC const* vmc) {                                                                                                         auto stack = (o2::data::Stack*)vmc->GetStack();                                                                                          if (vmc->IsTrackStop() && stack->currentTrackLeftTrackRef()) {                                                                              // we add a stopping TrackRef when the current track already                                                                             // registered previous TrackRefs                                                                                                         stack->addTrackReference(o2::TrackReference(*vmc, 0));                                                                                }                                                                                                                                      };                                                                                                                                     }

MyHook.C will be activated with

SimCutParams.mCutParams.stepTrackRefHookFile=PATHTO/MyHook.C

This commit implements a generic and customizable mechanism
to define and execute a callback during MC stepping. The purpose
of the callback is to create TrackReferences for certain conditions,
but can be used for other purposes (monitoring etc).
The callback just takes the instance to TVirtualMC

The default implementation creates a TrackReference on "IsStop"
whenever the current track already left a TrackReference previously.

The feature needs to be switched on via configurable Parameter

`SimCutParams.stepTrackRefHook=1`

Furthermore, the user can provide own callback implementations
via a macro file "MyHook.C". The implementation should follow the
following pattern:
```
o2::steer::O2MCApplicationBase::TrackRefFcn trackRefHook() {                                                                               return [](TVirtualMC const* vmc) {                                                                                                         auto stack = (o2::data::Stack*)vmc->GetStack();                                                                                          if (vmc->IsTrackStop() && stack->currentTrackLeftTrackRef()) {                                                                              // we add a stopping TrackRef when the current track already                                                                             // registered previous TrackRefs                                                                                                         stack->addTrackReference(o2::TrackReference(*vmc, 0));                                                                                }                                                                                                                                      };                                                                                                                                     }
```

MyHook.C will be activated with

`SimCutParams.mCutParams.stepTrackRefHookFile=PATHTO/MyHook.C`
@alibuild

Copy link
Copy Markdown
Collaborator

Error while checking build/O2/fullCI for 539e857 at 2022-10-14 21:17:

## sw/BUILD/o2checkcode-latest/log
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/Common/test/testGPUsortCUDA.cu:22:10: error: 'boost/test/unit_test.hpp' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUTracking/TRDTracking/GPUTRDTracker.cxx:37:10: error: 'omp.h' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUTracking/Base/GPUReconstruction.cxx:37:10: error: 'omp.h' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUTracking/Base/GPUReconstructionCPU.cxx:45:10: error: 'omp.h' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUTracking/display/GPUDisplay.cxx:36:10: error: 'omp.h' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUTracking/Base/cuda/GPUReconstructionCUDAGenRTC.cu:16:10: error: 'omp.h' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/cuda/../Shared/Utils.h:26:10: error: 'boost/program_options.hpp' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/cuda/../Shared/Utils.h:26:10: error: 'boost/program_options.hpp' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/Framework/Logger/include/Framework/Logger.h:14:10: error: 'fairlogger/Logger.h' file not found [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:520:12: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:1059:69: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:1175:5: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:1852:16: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:2276:18: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:4336:16: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/sm_20_atomic_functions.h:89:39: error: redefinition of 'atomicAdd' [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/Framework/Logger/include/Framework/Logger.h:14:10: error: 'fairlogger/Logger.h' file not found [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:520:12: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:1059:69: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:1175:5: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:1852:16: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:2276:18: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:4336:16: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/sm_20_atomic_functions.h:89:39: error: redefinition of 'atomicAdd' [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/../Shared/Utils.h:146:34: error: use of undeclared identifier 'int4'; did you mean 'int'? [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/benchmark.hip.cxx:199:35: error: use of undeclared identifier 'int4'; did you mean 'int'? [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/../Shared/Utils.h:146:34: error: use of undeclared identifier 'int4'; did you mean 'int'? [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:64:8: error: unknown type name '__host__' [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:64:27: error: expected ';' after top level declarator [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:64:33: error: overloaded 'operator+=' must have at least one parameter of class or enumeration type [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:64:44: error: unknown type name 'int4'; did you mean 'int'? [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:64:53: error: unknown type name 'int4'; did you mean 'int'? [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:66:4: error: member reference base type 'int' is not a structure or union [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:66:11: error: member reference base type 'int' is not a structure or union [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:67:4: error: member reference base type 'int' is not a structure or union [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:67:11: error: member reference base type 'int' is not a structure or union [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:68:4: error: member reference base type 'int' is not a structure or union [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:68:11: error: member reference base type 'int' is not a structure or union [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:69:4: error: member reference base type 'int' is not a structure or union [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:69:11: error: member reference base type 'int' is not a structure or union [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:85:1: error: unknown type name '__global__' [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:90:19: error: use of undeclared identifier 'blockIdx' [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:90:32: error: use of undeclared identifier 'blockDim' [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:90:45: error: use of undeclared identifier 'threadIdx' [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:90:78: error: use of undeclared identifier 'blockDim' [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/Detectors/EMCAL/calibration/include/EMCALCalibration/EMCALCalibExtractor.h:36:10: error: 'omp.h' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/Detectors/EMCAL/calibration/include/EMCALCalibration/EMCALCalibExtractor.h:36:10: error: 'omp.h' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/Detectors/EMCAL/calibration/include/EMCALCalibration/EMCALCalibExtractor.h:36:10: error: 'omp.h' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/Detectors/TOF/calibration/src/TOFChannelCalibrator.cxx:23:10: error: 'omp.h' file not found [clang-diagnostic-error]

Full log here.

@alibuild

Copy link
Copy Markdown
Collaborator

Error while checking build/O2/fullCI for f3de58e at 2022-10-15 02:35:

## sw/BUILD/o2checkcode-latest/log
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/Common/test/testGPUsortCUDA.cu:22:10: error: 'boost/test/unit_test.hpp' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUTracking/TRDTracking/GPUTRDTracker.cxx:37:10: error: 'omp.h' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUTracking/Base/GPUReconstruction.cxx:37:10: error: 'omp.h' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUTracking/Base/GPUReconstructionCPU.cxx:45:10: error: 'omp.h' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUTracking/display/GPUDisplay.cxx:36:10: error: 'omp.h' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUTracking/Base/cuda/GPUReconstructionCUDAGenRTC.cu:16:10: error: 'omp.h' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/cuda/../Shared/Utils.h:26:10: error: 'boost/program_options.hpp' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/cuda/../Shared/Utils.h:26:10: error: 'boost/program_options.hpp' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/Framework/Logger/include/Framework/Logger.h:14:10: error: 'fairlogger/Logger.h' file not found [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:520:12: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:1059:69: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:1175:5: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:1852:16: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:2276:18: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:4336:16: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/sm_20_atomic_functions.h:89:39: error: redefinition of 'atomicAdd' [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/Framework/Logger/include/Framework/Logger.h:14:10: error: 'fairlogger/Logger.h' file not found [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:520:12: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:1059:69: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:1175:5: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:1852:16: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:2276:18: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:4336:16: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/sm_20_atomic_functions.h:89:39: error: redefinition of 'atomicAdd' [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/../Shared/Utils.h:146:34: error: use of undeclared identifier 'int4'; did you mean 'int'? [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/benchmark.hip.cxx:199:35: error: use of undeclared identifier 'int4'; did you mean 'int'? [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/../Shared/Utils.h:146:34: error: use of undeclared identifier 'int4'; did you mean 'int'? [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:64:8: error: unknown type name '__host__' [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:64:27: error: expected ';' after top level declarator [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:64:33: error: overloaded 'operator+=' must have at least one parameter of class or enumeration type [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:64:44: error: unknown type name 'int4'; did you mean 'int'? [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:64:53: error: unknown type name 'int4'; did you mean 'int'? [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:66:4: error: member reference base type 'int' is not a structure or union [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:66:11: error: member reference base type 'int' is not a structure or union [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:67:4: error: member reference base type 'int' is not a structure or union [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:67:11: error: member reference base type 'int' is not a structure or union [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:68:4: error: member reference base type 'int' is not a structure or union [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:68:11: error: member reference base type 'int' is not a structure or union [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:69:4: error: member reference base type 'int' is not a structure or union [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:69:11: error: member reference base type 'int' is not a structure or union [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:85:1: error: unknown type name '__global__' [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:90:19: error: use of undeclared identifier 'blockIdx' [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:90:32: error: use of undeclared identifier 'blockDim' [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:90:45: error: use of undeclared identifier 'threadIdx' [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:90:78: error: use of undeclared identifier 'blockDim' [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/Detectors/EMCAL/calibration/include/EMCALCalibration/EMCALCalibExtractor.h:36:10: error: 'omp.h' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/Detectors/EMCAL/calibration/include/EMCALCalibration/EMCALCalibExtractor.h:36:10: error: 'omp.h' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/Detectors/EMCAL/calibration/include/EMCALCalibration/EMCALCalibExtractor.h:36:10: error: 'omp.h' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/10085-slc8_x86-64/0/Detectors/TOF/calibration/src/TOFChannelCalibrator.cxx:23:10: error: 'omp.h' file not found [clang-diagnostic-error]

Full log here.

@sawenzel sawenzel merged commit 5893629 into AliceO2Group:dev Oct 15, 2022
@sawenzel sawenzel deleted the swenzel/trackrefHook branch October 15, 2022 06:39
@miranov25

Copy link
Copy Markdown
Contributor

Reading the source code the default callback (if enabled) is one
https://github.com/sawenzel/AliceO2/blob/f3de58ee84d12be64f4e32f4627a77675aa7d550/Detectors/gconfig/src/StandardSteppingTrackRefHook.macro

For me it is good default

@miranov25

Copy link
Copy Markdown
Contributor

The standard hook is fine with me.
I suggest turning it on by default so that you only create references when particles have been created before. The increase in data volume is minimal in this case of the default hook. We will have much less isStop references than the number of particles.

I understand that this might not be the case once the hook changes, but it is definitely an expert action. Also, we will never create more TrackRefernces than there are particles in the stack.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants