Skip to content

various improvements to bisection of performance issues#8592

Draft
firewave wants to merge 6 commits into
cppcheck-opensource:mainfrom
firewave:bisect-hang-y
Draft

various improvements to bisection of performance issues#8592
firewave wants to merge 6 commits into
cppcheck-opensource:mainfrom
firewave:bisect-hang-y

Conversation

@firewave
Copy link
Copy Markdown
Collaborator

@firewave firewave commented May 28, 2026

these improvements help with bisecting performance issues in long/short-running cases

  • print timing information for each file
  • print the timeout we are running with
  • print the run time factor
  • round down the elapsed time
  • introduced a maximum run time factor (default: 1.3)
  • print the difference in run time

@firewave
Copy link
Copy Markdown
Collaborator Author

The timeout in the bisection is based on the run time of a good revision times two - matching the daca time regression report. But if the package runs very long the timeout might be too big.

Example (short-running):
If an analysis finishes very fast it might not exceed the timeout but might still show a sizable regression. For that case a run time factor threshold was introduced so be able catch differences which cannot be properly captured in an integer value.

Example (long-running):
If an analysis takes 30 seconds it requires 60 seconds to hit the timeout. But an increase in 5 or 10 seconds is already sizable. For that case a run time difference threshold has been introduced.

These changes also help with cases where the increase is incremental over various commits and not absolute caused by a single one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant