Skip to content

Benchmark History Tracking in Continuous Integration #590

Open

Description

@ngober

Is your feature request related to a problem? Please describe.
Performance is a very important part of ChampSim. If the simulations are slower, research is more difficult. Recently, I've pushed in a few occations (two examples) to add benchmarks to our CI runs. Currently, this merely runs the benchmarks and puts the results in the terminal. Tracking their change over time is a very manual process and their benefits are likely to be missed.

Describe the solution you'd like
I would like a solution with at least the following two features:

  • Alerts if the performance of a PR drifts significantly (perhaps in a comment on that PR)
  • Tracks the performance of the benchmarks over time (perhaps in the docs)

The solution would need to interface with the existing body of Catch2 benchmarks and extend to additional benchmarks easily.

Describe alternatives you've considered
This GitHub Action seems to do many of these things, but it would probably need to be implemented carefully. Its documentation seems to assume starting from scratch with GitHub Pages.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions