Repository navigation
Conversation
test/benchmark holds the same benchmarks as test/performance, on reactBenchmark() from @mui/internal-benchmark/page, run by the benchmark CLI from mui/mui-public#1898 (its preview build). test/performance stays for comparison as test:benchmark:vitest.
commit: |
Bundle size
PerformanceNo regressions in 36 benchmarks · ran 4m 24s · details Smoke · RadioGroup
Smoke · Slider
Smoke · ToggleGroup
Smoke · Toolbar
Medians; Δ is the confidence interval of the paired per-round difference (99%). Check out the code infra dashboard for more information about this PR. |
✅ Deploy Preview for base-ui ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
…as artifacts The .lintignore entries were appended onto its last line. The artifacts are TEMPORARY, while the benchmark setups are compared on repeated runs.
Shortens the Benchmark CLI check on pull requests; with 50 rounds and 3 minutes it took 19 minutes for the 15 benchmarks.
test/benchmark runs the same cases with the same metrics, so the Vitest suite in test/performance and its CI job go. The CLI job takes over the 'Benchmark tests' name.
…y component smoke/smoke.bench.tsx mounts each of the library's 36 components on its own, at a modest count; the runner shares a page load per round between a file's benchmarks (mui/mui-public#1898 at 621b803), so they cost little more than one. The separate mount benchmarks and the mixed surface it duplicates go; the Combobox, Menu and Select open and typing cases stay.
This reverts commit a885246.
The rows were built inside each render function, so every sample built them again inside the timed window. Also picks up mui/mui-public#1898 at feb4328, which stops the page build from warning about every 'use client' directive.
A round of the whole smoke file takes about 4 s in CI, so a minute fit about 15 rounds past the sample size, too few to settle at the 5% band.
…ase + adaptive rounds in the log
smokeBenchmark(name, count, renderRow) creates the rows once per page load and applies the shared sampling, so each case is just the component it mounts. Picks up mui/mui-public#1898 at f172e98.
…s the sample size; graceful stop on a signal
The open and typing benchmarks stay, for a fuller run against the last release to be set up separately.
Their CI precision is about ±10%; at the default 5% band, a run of unchanged code flagged one of them in 10. They are there to catch big regressions.
This reverts commit a0290ec.
The 30-round sample size moves from the CI command into SMOKE_OPTIONS, so a local run samples the same way; the 3-minute timeout was the default and goes. The comments say why without numbers that drift.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #5403
Ports the benchmarks to the
benchmarkCLI from mui/mui-public#1898 (installed from its pkg.pr.new build), the way mui/base-ui-charts#1186 and mui/material-ui#49329 do, and removes the Vitest benchmarks intest/performance.smoke/smoke.bench.tsxmounts each of the library's 36 components in a benchmark of its own; the runner shares one page load per round between a file's benchmarks, so the file costs little more than one. It replaces the separate mount benchmarks. The pull request job runs only the smoke file; the Combobox, Menu and Select open and typing cases stay, for a nightly run against the last release, to be set up separately.Comparison
We ran both setups 10 times in CI on all 15 benchmarks. A good setup should therefore report "no change" every time. Any regression it reports is a false alarm.
Both time the same thing: React's render and the paint after it. Vitest runs on a 2-CPU machine, the new tool on a larger one.
Precision is roughly the smallest slowdown a setup can reliably catch.
The same check on the smoke file as the pull request job runs it now (10% threshold, 3 minutes per run):
The smoke file gets 30 to 70 rounds within its time limit, so it is less precise than the separate files were; it is meant to catch big regressions. The one false alarm was a whole run off by +10% to +32% for Collapsible, which renders in a few milliseconds.
Library fixes, to try the benchmarks on
Three changes that make components render less on mount:
Slider.Valueupdates.Measured in CI by the smoke file against master: Slider −17% to −34% render time, Toggle Group −5% to −29%, Toolbar −3% to −26%; Radio Group and Scroll Area showed no clear change, and no other benchmark changed. The earlier separate benchmarks measured Slider −20% to −31%, Radio Group −10% to −26% and Toggle Group −20% to −32%.