Skip to content

Checker affinity tweaks - #64870

Open
Jake Bailey (jakebailey) wants to merge 5 commits into
microsoft:mainfrom
jakebailey:perf/checker-affinity
Open

Jake Bailey (jakebailey) wants to merge 5 commits into
microsoft:mainfrom
jakebailey:perf/checker-affinity

Conversation

@jakebailey

Copy link
Copy Markdown
Member

See the two commits.

--checkers 4:

                      │   b4.txt   │              a4.txt               │
                      │   sec/op   │   sec/op    vs base               │
    Tsc/bluesky         2.684 ± 4%   2.411 ± 2%  -10.17% (p=0.002 n=6)
    Tsc/sentry          10.45 ± 4%   10.32 ± 3%        ~ (p=0.180 n=6)
    Tsc/t3code-server   9.234 ± 4%   8.306 ± 2%  -10.04% (p=0.002 n=6)
    Tsc/mui-docs        7.426 ± 2%   6.714 ± 2%   -9.59% (p=0.002 n=6)
    geomean             6.623        6.103        -7.85%

                      │    b4.txt    │               a4.txt                │
                      │    rss-B     │    rss-B      vs base               │
    Tsc/bluesky         1.184Gi ± 3%   1.167Gi ± 2%        ~ (p=0.065 n=6)
    Tsc/sentry          3.526Gi ± 6%   3.393Gi ± 6%        ~ (p=0.132 n=6)
    Tsc/t3code-server   2.816Gi ± 6%   2.690Gi ± 8%        ~ (p=0.132 n=6)
    Tsc/mui-docs        2.246Gi ± 2%   2.015Gi ± 2%  -10.25% (p=0.002 n=6)
    geomean             2.267Gi        2.153Gi        -5.03%

                      │    b4.txt    │               a4.txt               │
                      │    heap-B    │    heap-B     vs base              │
    Tsc/bluesky         933.1Mi ± 0%   915.0Mi ± 0%  -1.94% (p=0.002 n=6)
    Tsc/sentry          2.605Gi ± 0%   2.479Gi ± 0%  -4.84% (p=0.002 n=6)
    Tsc/t3code-server   2.281Gi ± 0%   2.179Gi ± 0%  -4.50% (p=0.002 n=6)
    Tsc/mui-docs        1.717Gi ± 0%   1.562Gi ± 0%  -9.05% (p=0.002 n=6)
    geomean             1.746Gi        1.657Gi       -5.12%

--checkers 8:

                      │   b8.txt   │              a8.txt               │
                      │   sec/op   │   sec/op    vs base               │
    Tsc/bluesky         2.036 ± 4%   1.915 ± 2%   -5.97% (p=0.004 n=6)
    Tsc/sentry          8.352 ± 3%   7.233 ± 1%  -13.40% (p=0.002 n=6)
    Tsc/t3code-server   7.864 ± 3%   6.880 ± 5%  -12.52% (p=0.002 n=6)
    Tsc/mui-docs        6.367 ± 3%   5.856 ± 2%   -8.04% (p=0.002 n=6)
    geomean             5.402        4.860       -10.03%

                      │    b8.txt    │               a8.txt                │
                      │    rss-B     │    rss-B      vs base               │
    Tsc/bluesky         1.530Gi ± 2%   1.470Gi ± 2%   -3.93% (p=0.002 n=6)
    Tsc/sentry          4.214Gi ± 9%   4.011Gi ± 3%   -4.84% (p=0.009 n=6)
    Tsc/t3code-server   4.027Gi ± 4%   3.129Gi ± 2%  -22.31% (p=0.002 n=6)
    Tsc/mui-docs        3.029Gi ± 9%   2.573Gi ± 2%  -15.07% (p=0.002 n=6)
    geomean             2.978Gi        2.625Gi       -11.87%

                      │    b8.txt    │               a8.txt                │
                      │    heap-B    │    heap-B     vs base               │
    Tsc/bluesky         1.206Gi ± 0%   1.155Gi ± 0%   -4.17% (p=0.002 n=6)
    Tsc/sentry          3.116Gi ± 0%   2.892Gi ± 0%   -7.18% (p=0.002 n=6)
    Tsc/t3code-server   3.052Gi ± 0%   2.502Gi ± 0%  -18.02% (p=0.002 n=6)
    Tsc/mui-docs        2.478Gi ± 0%   1.917Gi ± 0%  -22.66% (p=0.002 n=6)
    geomean             2.309Gi        2.001Gi       -13.34%
--checkers 4:

                      │   b4.txt   │              a4.txt              │
                      │   sec/op   │   sec/op    vs base              │
    Tsc/bluesky         2.394 ± 5%   2.369 ± 7%       ~ (p=0.240 n=6)
    Tsc/sentry          10.42 ± 3%   10.20 ± 3%       ~ (p=0.180 n=6)
    Tsc/t3code-server   8.336 ± 3%   8.184 ± 2%       ~ (p=0.132 n=6)
    Tsc/mui-docs        6.898 ± 2%   6.537 ± 1%  -5.23% (p=0.002 n=6)
    geomean             6.154        5.996       -2.57%

                      │    b4.txt    │               a4.txt                │
                      │    rss-B     │    rss-B      vs base               │
    Tsc/bluesky         1.160Gi ± 3%   1.167Gi ± 1%        ~ (p=0.093 n=6)
    Tsc/sentry          3.402Gi ± 3%   3.323Gi ± 4%        ~ (p=0.937 n=6)
    Tsc/t3code-server   2.776Gi ± 6%   2.820Gi ± 6%        ~ (p=0.699 n=6)
    Tsc/mui-docs        2.017Gi ± 3%   2.299Gi ± 5%  +14.00% (p=0.002 n=6)
    geomean             2.168Gi        2.239Gi        +3.30%

                      │    b4.txt    │               a4.txt                │
                      │    heap-B    │    heap-B     vs base               │
    Tsc/bluesky         914.8Mi ± 0%   914.9Mi ± 0%        ~ (p=1.000 n=6)
    Tsc/sentry          2.478Gi ± 0%   2.483Gi ± 0%   +0.18% (p=0.004 n=6)
    Tsc/t3code-server   2.178Gi ± 0%   2.178Gi ± 0%        ~ (p=0.699 n=6)
    Tsc/mui-docs        1.562Gi ± 0%   1.732Gi ± 0%  +10.87% (p=0.002 n=6)
    geomean             1.657Gi        1.701Gi        +2.66%

--checkers 8:

                      │   b8.txt   │              a8.txt              │
                      │   sec/op   │   sec/op    vs base              │
    Tsc/bluesky         1.980 ± 3%   1.947 ± 2%       ~ (p=0.180 n=6)
    Tsc/sentry          7.520 ± 4%   7.402 ± 1%  -1.56% (p=0.041 n=6)
    Tsc/t3code-server   7.042 ± 2%   6.837 ± 2%  -2.92% (p=0.004 n=6)
    Tsc/mui-docs        5.859 ± 2%   5.727 ± 1%  -2.25% (p=0.009 n=6)
    geomean             4.978        4.874       -2.10%

                      │    b8.txt    │               a8.txt               │
                      │    rss-B     │    rss-B      vs base              │
    Tsc/bluesky         1.470Gi ± 2%   1.481Gi ± 3%       ~ (p=0.310 n=6)
    Tsc/sentry          4.015Gi ± 3%   4.016Gi ± 3%       ~ (p=0.699 n=6)
    Tsc/t3code-server   3.156Gi ± 3%   3.123Gi ± 1%       ~ (p=0.310 n=6)
    Tsc/mui-docs        2.561Gi ± 7%   2.698Gi ± 7%  +5.38% (p=0.041 n=6)
    geomean             2.628Gi        2.661Gi       +1.24%

                      │    b8.txt    │               a8.txt               │
                      │    heap-B    │    heap-B     vs base              │
    Tsc/bluesky         1.155Gi ± 0%   1.155Gi ± 0%       ~ (p=0.699 n=6)
    Tsc/sentry          2.893Gi ± 0%   2.881Gi ± 0%  -0.40% (p=0.002 n=6)
    Tsc/t3code-server   2.502Gi ± 0%   2.502Gi ± 0%       ~ (p=0.818 n=6)
    Tsc/mui-docs        1.917Gi ± 0%   2.043Gi ± 0%  +6.58% (p=0.002 n=6)
    geomean             2.001Gi        2.031Gi       +1.51%

Each checker builds its own caches for whatever types its files reach,
so files that share a neighborhood (a package's own types, the
framework declarations it uses) should land on the same checker.

Assigning one file at a time did a poor job of that. Direct imports
are only a small part of what neighboring files have in common, so
related files were scattered across checkers, each rebuilding the same
types. Balance suffered too: declaration files excluded by skipLibCheck
still counted toward a checker's load even though no checker spends
time on them, so the real work per checker was uneven.

Now the partition leaves out files that are not type checked and
places whole directories: the outermost directory weighing at most a
quarter of a checker's load is placed as one unit, heaviest units
first. This replaces the calibrated source/declaration policies, which
existed to work around the same problems. Diagnostics are unchanged on
vscode, mui-docs, the TypeScript compiler, and xstate.

--checkers 4:

                      │   b4.txt   │              a4.txt               │
                      │   sec/op   │   sec/op    vs base               │
    Tsc/bluesky         2.684 ± 4%   2.411 ± 2%  -10.17% (p=0.002 n=6)
    Tsc/sentry          10.45 ± 4%   10.32 ± 3%        ~ (p=0.180 n=6)
    Tsc/t3code-server   9.234 ± 4%   8.306 ± 2%  -10.04% (p=0.002 n=6)
    Tsc/mui-docs        7.426 ± 2%   6.714 ± 2%   -9.59% (p=0.002 n=6)
    geomean             6.623        6.103        -7.85%

                      │    b4.txt    │               a4.txt                │
                      │    rss-B     │    rss-B      vs base               │
    Tsc/bluesky         1.184Gi ± 3%   1.167Gi ± 2%        ~ (p=0.065 n=6)
    Tsc/sentry          3.526Gi ± 6%   3.393Gi ± 6%        ~ (p=0.132 n=6)
    Tsc/t3code-server   2.816Gi ± 6%   2.690Gi ± 8%        ~ (p=0.132 n=6)
    Tsc/mui-docs        2.246Gi ± 2%   2.015Gi ± 2%  -10.25% (p=0.002 n=6)
    geomean             2.267Gi        2.153Gi        -5.03%

                      │    b4.txt    │               a4.txt               │
                      │    heap-B    │    heap-B     vs base              │
    Tsc/bluesky         933.1Mi ± 0%   915.0Mi ± 0%  -1.94% (p=0.002 n=6)
    Tsc/sentry          2.605Gi ± 0%   2.479Gi ± 0%  -4.84% (p=0.002 n=6)
    Tsc/t3code-server   2.281Gi ± 0%   2.179Gi ± 0%  -4.50% (p=0.002 n=6)
    Tsc/mui-docs        1.717Gi ± 0%   1.562Gi ± 0%  -9.05% (p=0.002 n=6)
    geomean             1.746Gi        1.657Gi       -5.12%

--checkers 8:

                      │   b8.txt   │              a8.txt               │
                      │   sec/op   │   sec/op    vs base               │
    Tsc/bluesky         2.036 ± 4%   1.915 ± 2%   -5.97% (p=0.004 n=6)
    Tsc/sentry          8.352 ± 3%   7.233 ± 1%  -13.40% (p=0.002 n=6)
    Tsc/t3code-server   7.864 ± 3%   6.880 ± 5%  -12.52% (p=0.002 n=6)
    Tsc/mui-docs        6.367 ± 3%   5.856 ± 2%   -8.04% (p=0.002 n=6)
    geomean             5.402        4.860       -10.03%

                      │    b8.txt    │               a8.txt                │
                      │    rss-B     │    rss-B      vs base               │
    Tsc/bluesky         1.530Gi ± 2%   1.470Gi ± 2%   -3.93% (p=0.002 n=6)
    Tsc/sentry          4.214Gi ± 9%   4.011Gi ± 3%   -4.84% (p=0.009 n=6)
    Tsc/t3code-server   4.027Gi ± 4%   3.129Gi ± 2%  -22.31% (p=0.002 n=6)
    Tsc/mui-docs        3.029Gi ± 9%   2.573Gi ± 2%  -15.07% (p=0.002 n=6)
    geomean             2.978Gi        2.625Gi       -11.87%

                      │    b8.txt    │               a8.txt                │
                      │    heap-B    │    heap-B     vs base               │
    Tsc/bluesky         1.206Gi ± 0%   1.155Gi ± 0%   -4.17% (p=0.002 n=6)
    Tsc/sentry          3.116Gi ± 0%   2.892Gi ± 0%   -7.18% (p=0.002 n=6)
    Tsc/t3code-server   3.052Gi ± 0%   2.502Gi ± 0%  -18.02% (p=0.002 n=6)
    Tsc/mui-docs        2.478Gi ± 0%   1.917Gi ± 0%  -22.66% (p=0.002 n=6)
    geomean             2.309Gi        2.001Gi       -13.34%
A file's estimated work counts each of its imports. A barrel that
re-exports thousands of modules is then estimated as one of the most
expensive files in the program although it is among the cheapest to
check: MUI's icon index re-exports 10,750 modules and takes 20ms, yet
carried 14% of the docs project's estimated work. Its checker was
treated as full, and import affinity pulled every icon onto it too, so
one of four checkers spent the check phase on cheap files while the
other three shared all of the real work.

Counting at most 256 imports per file leaves ordinary files alone (the
largest import count in bluesky, sentry and t3code is 358) and lets
that checker take a full share. Using the fourth checker costs memory
on mui-docs, since it now builds the MUI types as well.

--checkers 4:

                      │   b4.txt   │              a4.txt              │
                      │   sec/op   │   sec/op    vs base              │
    Tsc/bluesky         2.394 ± 5%   2.369 ± 7%       ~ (p=0.240 n=6)
    Tsc/sentry          10.42 ± 3%   10.20 ± 3%       ~ (p=0.180 n=6)
    Tsc/t3code-server   8.336 ± 3%   8.184 ± 2%       ~ (p=0.132 n=6)
    Tsc/mui-docs        6.898 ± 2%   6.537 ± 1%  -5.23% (p=0.002 n=6)
    geomean             6.154        5.996       -2.57%

                      │    b4.txt    │               a4.txt                │
                      │    rss-B     │    rss-B      vs base               │
    Tsc/bluesky         1.160Gi ± 3%   1.167Gi ± 1%        ~ (p=0.093 n=6)
    Tsc/sentry          3.402Gi ± 3%   3.323Gi ± 4%        ~ (p=0.937 n=6)
    Tsc/t3code-server   2.776Gi ± 6%   2.820Gi ± 6%        ~ (p=0.699 n=6)
    Tsc/mui-docs        2.017Gi ± 3%   2.299Gi ± 5%  +14.00% (p=0.002 n=6)
    geomean             2.168Gi        2.239Gi        +3.30%

                      │    b4.txt    │               a4.txt                │
                      │    heap-B    │    heap-B     vs base               │
    Tsc/bluesky         914.8Mi ± 0%   914.9Mi ± 0%        ~ (p=1.000 n=6)
    Tsc/sentry          2.478Gi ± 0%   2.483Gi ± 0%   +0.18% (p=0.004 n=6)
    Tsc/t3code-server   2.178Gi ± 0%   2.178Gi ± 0%        ~ (p=0.699 n=6)
    Tsc/mui-docs        1.562Gi ± 0%   1.732Gi ± 0%  +10.87% (p=0.002 n=6)
    geomean             1.657Gi        1.701Gi        +2.66%

--checkers 8:

                      │   b8.txt   │              a8.txt              │
                      │   sec/op   │   sec/op    vs base              │
    Tsc/bluesky         1.980 ± 3%   1.947 ± 2%       ~ (p=0.180 n=6)
    Tsc/sentry          7.520 ± 4%   7.402 ± 1%  -1.56% (p=0.041 n=6)
    Tsc/t3code-server   7.042 ± 2%   6.837 ± 2%  -2.92% (p=0.004 n=6)
    Tsc/mui-docs        5.859 ± 2%   5.727 ± 1%  -2.25% (p=0.009 n=6)
    geomean             4.978        4.874       -2.10%

                      │    b8.txt    │               a8.txt               │
                      │    rss-B     │    rss-B      vs base              │
    Tsc/bluesky         1.470Gi ± 2%   1.481Gi ± 3%       ~ (p=0.310 n=6)
    Tsc/sentry          4.015Gi ± 3%   4.016Gi ± 3%       ~ (p=0.699 n=6)
    Tsc/t3code-server   3.156Gi ± 3%   3.123Gi ± 1%       ~ (p=0.310 n=6)
    Tsc/mui-docs        2.561Gi ± 7%   2.698Gi ± 7%  +5.38% (p=0.041 n=6)
    geomean             2.628Gi        2.661Gi       +1.24%

                      │    b8.txt    │               a8.txt               │
                      │    heap-B    │    heap-B     vs base              │
    Tsc/bluesky         1.155Gi ± 0%   1.155Gi ± 0%       ~ (p=0.699 n=6)
    Tsc/sentry          2.893Gi ± 0%   2.881Gi ± 0%  -0.40% (p=0.002 n=6)
    Tsc/t3code-server   2.502Gi ± 0%   2.502Gi ± 0%       ~ (p=0.818 n=6)
    Tsc/mui-docs        1.917Gi ± 0%   2.043Gi ± 0%  +6.58% (p=0.002 n=6)
    geomean             2.001Gi        2.031Gi       +1.51%
Copilot AI balanced review requested due to automatic review settings October 19, 2026 17:36
@github-project-automation github-project-automation Bot moved this to Not started in PR Backlog Oct 19, 2026
@typescript-automation typescript-automation Bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Oct 19, 2026
@jakebailey

Copy link
Copy Markdown
Member Author

TypeScript Bot (@typescript-bot) perf test this faster

@typescript-automation

typescript-automation Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this faster ✅ Started 👀 Results

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The scheduling rewrite needs human confirmation of diagnostic stability and workload balance, with direct regression coverage still missing.

1 open issue
What changed in this PR

Refines checker assignment to improve locality and balance parallel type-checking work.

Changes:

  • Groups files by directory and assigns heavier groups first.
  • Excludes skipped declaration/JSON files and caps import-based weights.
  • Replaces obsolete policy tests with directory-grouping tests.
File Description
tsc/​internal/​compiler/​checkerpool.go Implements directory-based grouping and revised workload estimates.
tsc/​internal/​compiler/​checkerpool_test.go Tests grouping and updates partitioning test constants.

Review effort: Balanced

Comment on lines +314 to +318
if (file.IsDeclarationFile || ast.IsJsonSourceFile(file)) && p.program.SkipTypeChecking(file, false) {
continue
}
baseWeights[i] = getCheckerAssociationBaseWeight(file.NodeCount, len(file.Text()))
importCounts[i] = min(len(file.Imports()), checkerAssociationMaxImports)
@typescript-automation

Copy link
Copy Markdown
Contributor

Jake Bailey (@jakebailey)
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - native
Errors 41 41 ~ ~ ~ p=1.000 n=12
Symbols 115,276 115,736 +460 (+ 0.40%) ~ ~ p=0.000 n=12
Types 79,613 79,774 +161 (+ 0.20%) ~ ~ p=0.000 n=12
Memory Used 113,531k (± 0.05%) 113,934k (± 0.07%) +403k (+ 0.35%) 113,712k 114,130k p=0.000 n=12
Memory Allocs 2,212,558 (± 0.01%) 2,214,263 (± 0.01%) +1,705 (+ 0.08%) 2,213,876 2,215,184 p=0.000 n=12
Config Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Parse Time 0.040s (± 4.88%) 0.040s (± 2.65%) ~ 0.037s 0.043s p=0.919 n=12
Bind Time 0.014s (± 7.56%) 0.014s (±11.59%) ~ 0.011s 0.018s p=0.427 n=12
Check Time 0.467s (± 0.55%) 0.463s (± 0.77%) -0.004s (- 0.95%) 0.455s 0.474s p=0.046 n=12
Emit Time 0.269s (± 2.14%) 0.265s (± 2.40%) ~ 0.247s 0.281s p=0.417 n=12
Total Time 0.797s (± 0.78%) 0.787s (± 0.84%) -0.010s (- 1.26%) 0.772s 0.807s p=0.025 n=12
angular-1 - native
Errors 3 3 ~ ~ ~ p=1.000 n=12
Symbols 847,496 (± 0.05%) 811,556 (± 0.15%) 🟩-35,940 (- 4.24%) 808,183 814,157 p=0.000 n=12
Types 247,032 (± 0.00%) 224,454 (± 0.00%) 🟩-22,578 (- 9.14%) 224,450 224,465 p=0.000 n=12
Memory Used 596,409k (± 0.02%) 581,364k (± 0.03%) -15,045k (- 2.52%) 580,774k 581,862k p=0.000 n=12
Memory Allocs 13,139,065 (± 0.04%) 13,060,052 (± 0.16%) -79,013 (- 0.60%) 13,034,553 13,114,193 p=0.000 n=12
Config Time 0.017s (± 1.95%) 0.017s +0.000s (+ 2.51%) ~ ~ p=0.037 n=12
Parse Time 0.224s (± 2.17%) 0.226s (± 3.02%) ~ 0.211s 0.247s p=0.966 n=12
Bind Time 0.079s (±14.94%) 0.078s (±11.94%) ~ 0.053s 0.099s p=0.966 n=12
Check Time 0s 0s ~ ~ ~ p=1.000 n=12
Emit Time 1.550s (± 1.22%) 1.546s (± 1.92%) ~ 1.471s 1.621s p=0.977 n=12
Total Time 1.887s (± 1.48%) 1.888s (± 1.69%) ~ 1.804s 1.944s p=0.899 n=12
mui-docs - native
Errors 11,405 11,405 ~ ~ ~ p=1.000 n=12
Symbols 4,331,983 4,026,970 🟩-305,013 (- 7.04%) ~ ~ p=0.000 n=12
Types 1,370,520 1,272,013 🟩-98,507 (- 7.19%) ~ ~ p=0.000 n=12
Memory Used 2,625,427k (± 0.06%) 2,493,603k (± 0.06%) 🟩-131,824k (- 5.02%) 2,489,015k 2,497,777k p=0.000 n=12
Memory Allocs 27,719,651 (± 0.11%) 26,382,109 (± 0.07%) 🟩-1,337,543 (- 4.83%) 26,343,231 26,434,874 p=0.000 n=12
Config Time 0.017s (± 1.95%) 0.017s +0.000s (+ 2.51%) ~ ~ p=0.037 n=12
Parse Time 0.466s (± 2.14%) 0.463s (± 2.68%) ~ 0.431s 0.495s p=0.660 n=12
Bind Time 0.002s 0.002s ~ ~ ~ p=1.000 n=12
Check Time 7.254s (± 0.36%) 6.847s (± 0.47%) 🟩-0.407s (- 5.61%) 6.785s 6.946s p=0.000 n=12
Emit Time 0.561s (± 7.38%) 0.574s (± 9.01%) ~ 0.422s 0.655s p=0.640 n=12
Total Time 9.052s (± 0.19%) 8.602s (± 0.58%) 🟩-0.450s (- 4.97%) 8.427s 8.683s p=0.000 n=12
strada-build-src - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,388,680 1,377,817 -10,863 (- 0.78%) ~ ~ p=0.000 n=12
Types 441,862 434,670 -7,192 (- 1.63%) ~ ~ p=0.000 n=12
Memory Used 1,309,983k (± 0.61%) 1,315,415k (± 0.85%) ~ 1,296,322k 1,355,077k p=0.600 n=12
Memory Allocs 78,288,302 (± 0.06%) 78,284,069 (± 0.07%) ~ 78,195,019 78,507,228 p=0.478 n=12
Config Time 0.004s (±10.74%) 0.004s (±10.43%) ~ 0.003s 0.005s p=0.379 n=12
Parse Time 0.253s (± 2.53%) 0.254s (± 2.80%) ~ 0.237s 0.282s p=0.989 n=12
Bind Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Check Time 1.671s (± 0.57%) 1.634s (± 0.55%) -0.037s (- 2.24%) 1.617s 1.665s p=0.000 n=12
Emit Time 0.300s (± 3.18%) 0.302s (± 5.29%) ~ 0.255s 0.361s p=0.660 n=12
Total Time 19.869s (± 0.62%) 19.452s (± 0.48%) -0.417s (- 2.10%) 19.272s 19.879s p=0.000 n=12
strada-compiler - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 334,953 328,019 -6,934 (- 2.07%) ~ ~ p=0.000 n=12
Types 197,616 193,641 -3,975 (- 2.01%) ~ ~ p=0.000 n=12
Memory Used 247,499k (± 0.03%) 245,476k (± 0.03%) -2,022k (- 0.82%) 245,311k 245,682k p=0.000 n=12
Memory Allocs 4,329,829 (± 0.01%) 4,314,422 (± 0.01%) -15,407 (- 0.36%) 4,313,499 4,315,874 p=0.000 n=12
Config Time 0.001s 0.001s ~ ~ ~ p=1.000 n=12
Parse Time 0.110s (± 4.26%) 0.118s (± 5.24%) ~ 0.105s 0.131s p=0.103 n=12
Bind Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Check Time 0.959s (± 0.54%) 0.905s (± 0.73%) 🟩-0.054s (- 5.60%) 0.893s 0.928s p=0.000 n=12
Emit Time 0.154s (± 4.93%) 0.163s (± 3.74%) ~ 0.144s 0.177s p=0.065 n=12
Total Time 1.280s (± 0.81%) 1.241s (± 0.80%) -0.038s (- 3.00%) 1.211s 1.268s p=0.000 n=12
ts-pre-modules - native
Errors 87 87 ~ ~ ~ p=1.000 n=12
Symbols 303,834 280,895 🟩-22,939 (- 7.55%) ~ ~ p=0.000 n=12
Types 181,663 157,855 🟩-23,808 (-13.11%) ~ ~ p=0.000 n=12
Memory Used 215,222k (± 0.02%) 204,353k (± 0.01%) 🟩-10,869k (- 5.05%) 204,316k 204,420k p=0.000 n=12
Memory Allocs 1,401,688 (± 0.01%) 1,318,989 (± 0.02%) 🟩-82,699 (- 5.90%) 1,318,164 1,319,414 p=0.000 n=12
Config Time 0.000s 0.000s (±217.90%) ~ 0.000s 0.001s p=1.000 n=12
Parse Time 0.095s (± 2.82%) 0.098s (± 3.95%) ~ 0.088s 0.108s p=0.191 n=12
Bind Time 0.036s (±10.01%) 0.042s (±10.02%) 🔻+0.007s (+18.74%) 0.032s 0.054s p=0.021 n=12
Check Time 0.750s (± 0.74%) 0.770s (± 0.88%) +0.021s (+ 2.73%) 0.760s 0.791s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 0.895s (± 0.78%) 0.925s (± 0.53%) +0.030s (+ 3.35%) 0.916s 0.942s p=0.000 n=12
vscode - native
Errors 396 396 ~ ~ ~ p=1.000 n=12
Symbols 12,386,876 11,914,553 🟩-472,323 (- 3.81%) ~ ~ p=0.000 n=12
Types 4,049,733 3,840,963 🟩-208,770 (- 5.16%) ~ ~ p=0.000 n=12
Memory Used 5,820,744k (± 0.01%) 5,636,076k (± 0.02%) 🟩-184,668k (- 3.17%) 5,634,140k 5,638,212k p=0.000 n=12
Memory Allocs 59,186,667 (± 0.01%) 58,195,629 (± 0.03%) -991,038 (- 1.67%) 58,171,511 58,259,062 p=0.000 n=12
Config Time 0.076s (± 0.51%) 0.077s (± 0.42%) +0.001s (+ 0.87%) 0.076s 0.078s p=0.003 n=12
Parse Time 1.811s (± 2.41%) 1.826s (± 2.97%) ~ 1.729s 2.053s p=0.767 n=12
Bind Time 0.745s (±13.98%) 0.713s (±11.11%) ~ 0.508s 0.959s p=0.702 n=12
Check Time 15.529s (± 1.00%) 17.177s (± 0.76%) 🔻+1.648s (+10.61%) 16.961s 17.485s p=0.000 n=12
Emit Time 5.645s (± 5.78%) 5.674s (± 9.52%) ~ 4.624s 6.521s p=0.590 n=12
Total Time 23.895s (± 1.16%) 25.574s (± 1.82%) 🔻+1.679s (+ 7.03%) 24.526s 26.307s p=0.000 n=12
webpack - native
Errors 598 598 ~ ~ ~ p=1.000 n=12
Symbols 1,398,279 1,369,032 -29,247 (- 2.09%) ~ ~ p=0.000 n=12
Types 670,934 644,433 🟩-26,501 (- 3.95%) ~ ~ p=0.000 n=12
Memory Used 885,624k (± 0.01%) 869,961k (± 0.01%) -15,663k (- 1.77%) 869,766k 870,363k p=0.000 n=12
Memory Allocs 6,927,066 (± 0.02%) 6,772,811 (± 0.01%) -154,254 (- 2.23%) 6,770,732 6,775,035 p=0.000 n=12
Config Time 0.009s 0.009s ~ ~ ~ p=1.000 n=12
Parse Time 0.317s (± 2.34%) 0.313s (± 3.36%) ~ 0.270s 0.334s p=0.452 n=12
Bind Time 0.074s (± 2.29%) 0.078s (±11.90%) ~ 0.068s 0.120s p=0.635 n=12
Check Time 2.970s (± 0.50%) 2.485s (± 0.41%) 🟩-0.485s (-16.33%) 2.449s 2.509s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 3.406s (± 0.46%) 2.923s (± 0.42%) 🟩-0.483s (-14.19%) 2.889s 2.955s p=0.000 n=12
xstate-main - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,060,810 1,045,537 -15,273 (- 1.44%) ~ ~ p=0.000 n=12
Types 389,671 381,949 -7,722 (- 1.98%) ~ ~ p=0.000 n=12
Memory Used 471,813k (± 0.03%) 465,563k (± 0.02%) -6,250k (- 1.32%) 465,362k 465,760k p=0.000 n=12
Memory Allocs 4,221,831 (± 0.12%) 4,175,488 (± 0.13%) -46,342 (- 1.10%) 4,165,643 4,190,601 p=0.000 n=12
Config Time 0.003s (± 8.64%) 0.003s ~ ~ ~ p=0.478 n=12
Parse Time 0.136s (± 1.94%) 0.137s (± 2.35%) ~ 0.127s 0.143s p=0.503 n=12
Bind Time 0.042s (±14.58%) 0.041s (±18.82%) ~ 0.033s 0.074s p=0.366 n=12
Check Time 1.091s (± 0.80%) 1.062s (± 0.54%) -0.030s (- 2.70%) 1.052s 1.083s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 1.281s (± 0.96%) 1.250s (± 0.73%) -0.030s (- 2.38%) 1.231s 1.270s p=0.001 n=12
System info unknown
Hosts
  • native
Scenarios
  • Compiler-Unions - native
  • angular-1 - native
  • mui-docs - native
  • strada-build-src - native
  • strada-compiler - native
  • ts-pre-modules - native
  • vscode - native
  • webpack - native
  • xstate-main - native
Benchmark Name Iterations
Current pr 12
Baseline baseline 12

Developer Information:

Download Benchmarks

Checking takes as long as the slowest checker. Estimated work can be
well off for a whole directory, and with groups of up to a quarter of
a checker's load, a single misjudged group could leave one checker
seconds behind the rest. The perf run on the PR showed a large VS Code
check-time regression this way, while other VS Code snapshots
improved, which fits a result that hinges on where one group lands.

Cap groups at a sixteenth of a checker's load instead. Packages still
stay together in most projects, but no single group can unbalance the
checkers as much.

Wall time at 4 checkers, compared with the previous commit
(vscode-older is the July snapshot from the benchmarking repo):

                 │ /tmp/old.txt │           /tmp/new.txt           │
                 │    sec/op    │   sec/op    vs base              │
Tsc/vscode           14.60 ± 4%   14.49 ± 1%       ~ (p=0.589 n=6)
Tsc/vscode-older     7.773 ± 5%   7.725 ± 3%       ~ (p=0.589 n=6)
Tsc/mui-docs         8.800 ± 3%   7.981 ± 4%  -9.31% (p=0.002 n=6)
geomean              9.995        9.630       -3.66%

                 │ /tmp/old.txt │            /tmp/new.txt            │
                 │    rss-B     │    rss-B      vs base              │
Tsc/vscode         5.805Gi ± 3%   5.907Gi ± 2%       ~ (p=0.093 n=6)
Tsc/vscode-older   3.840Gi ± 2%   3.866Gi ± 3%       ~ (p=0.589 n=6)
Tsc/mui-docs       3.655Gi ± 6%   3.639Gi ± 6%       ~ (p=0.818 n=6)
geomean            4.335Gi        4.364Gi       +0.67%

                 │ /tmp/old.txt │            /tmp/new.txt            │
                 │    heap-B    │    heap-B     vs base              │
Tsc/vscode         4.921Gi ± 0%   4.937Gi ± 0%  +0.33% (p=0.002 n=6)
Tsc/vscode-older   3.205Gi ± 0%   3.213Gi ± 0%  +0.24% (p=0.002 n=6)
Tsc/mui-docs       2.746Gi ± 0%   2.732Gi ± 0%  -0.52% (p=0.002 n=6)
geomean            3.512Gi        3.512Gi       +0.01%
@jakebailey

Copy link
Copy Markdown
Member Author

TypeScript Bot (@typescript-bot) perf test this faster

@typescript-automation

typescript-automation Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this faster ✅ Started 👀 Results

@typescript-automation

Copy link
Copy Markdown
Contributor

Jake Bailey (@jakebailey)
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - native
Errors 41 41 ~ ~ ~ p=1.000 n=12
Symbols 115,276 115,276 ~ ~ ~ p=1.000 n=12
Types 79,613 79,613 ~ ~ ~ p=1.000 n=12
Memory Used 113,498k (± 0.06%) 113,518k (± 0.06%) ~ 113,379k 113,718k p=0.854 n=12
Memory Allocs 2,212,589 (± 0.01%) 2,212,572 (± 0.00%) ~ 2,212,270 2,212,825 p=0.932 n=12
Config Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Parse Time 0.039s (± 3.83%) 0.040s (± 5.63%) ~ 0.035s 0.045s p=0.787 n=12
Bind Time 0.013s (±12.24%) 0.013s (± 8.83%) ~ 0.011s 0.015s p=0.710 n=12
Check Time 0.469s (± 0.97%) 0.463s (± 0.70%) -0.006s (- 1.21%) 0.457s 0.473s p=0.037 n=12
Emit Time 0.268s (± 2.09%) 0.265s (± 2.68%) ~ 0.248s 0.283s p=0.486 n=12
Total Time 0.795s (± 0.79%) 0.786s (± 0.78%) -0.009s (- 1.14%) 0.775s 0.808s p=0.029 n=12
angular-1 - native
Errors 3 3 ~ ~ ~ p=1.000 n=12
Symbols 846,688 (± 0.08%) 818,890 (± 0.08%) 🟩-27,797 (- 3.28%) 817,514 820,491 p=0.000 n=12
Types 247,032 (± 0.00%) 229,093 (± 0.00%) 🟩-17,939 (- 7.26%) 229,089 229,102 p=0.000 n=12
Memory Used 596,522k (± 0.02%) 584,552k (± 0.02%) -11,970k (- 2.01%) 584,244k 584,858k p=0.000 n=12
Memory Allocs 13,143,510 (± 0.05%) 13,023,794 (± 0.01%) -119,716 (- 0.91%) 13,021,744 13,031,158 p=0.000 n=12
Config Time 0.017s (± 1.86%) 0.017s ~ ~ ~ p=0.093 n=12
Parse Time 0.223s (± 2.32%) 0.222s (± 2.30%) ~ 0.211s 0.239s p=0.639 n=12
Bind Time 0.080s (±16.90%) 0.083s (±13.11%) ~ 0.060s 0.110s p=0.522 n=12
Check Time 0s 0s ~ ~ ~ p=1.000 n=12
Emit Time 1.571s (± 1.70%) 1.537s (± 1.48%) ~ 1.482s 1.580s p=0.101 n=12
Total Time 1.910s (± 1.48%) 1.878s (± 1.45%) ~ 1.812s 1.935s p=0.086 n=12
mui-docs - native
Errors 11,405 11,405 ~ ~ ~ p=1.000 n=12
Symbols 4,331,983 4,211,620 -120,363 (- 2.78%) ~ ~ p=0.000 n=12
Types 1,370,520 1,343,389 -27,131 (- 1.98%) ~ ~ p=0.000 n=12
Memory Used 2,626,322k (± 0.06%) 2,573,755k (± 0.04%) -52,567k (- 2.00%) 2,570,544k 2,575,694k p=0.000 n=12
Memory Allocs 27,718,249 (± 0.06%) 26,877,365 (± 0.05%) 🟩-840,884 (- 3.03%) 26,833,159 26,916,781 p=0.000 n=12
Config Time 0.017s (± 1.70%) 0.017s ~ ~ ~ p=0.217 n=12
Parse Time 0.470s (± 2.27%) 0.470s (± 2.02%) ~ 0.441s 0.492s p=0.560 n=12
Bind Time 0.002s 0.002s ~ ~ ~ p=1.000 n=12
Check Time 7.255s (± 0.50%) 7.795s (± 0.52%) 🔻+0.539s (+ 7.43%) 7.681s 7.855s p=0.000 n=12
Emit Time 0.581s (±10.84%) 0.425s (± 0.47%) 🟩-0.156s (-26.89%) 0.420s 0.428s p=0.000 n=12
Total Time 9.076s (± 0.36%) 9.390s (± 0.64%) +0.313s (+ 3.45%) 9.224s 9.477s p=0.000 n=12
strada-build-src - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,388,680 1,381,084 -7,596 (- 0.55%) ~ ~ p=0.000 n=12
Types 441,862 437,491 -4,371 (- 0.99%) ~ ~ p=0.000 n=12
Memory Used 1,322,911k (± 0.80%) 1,324,590k (± 1.53%) ~ 1,303,768k 1,418,164k p=0.590 n=12
Memory Allocs 78,291,837 (± 0.09%) 78,358,509 (± 0.14%) ~ 78,241,629 78,864,413 p=0.514 n=12
Config Time 0.003s (± 9.48%) 0.004s (± 9.39%) ~ 0.003s 0.004s p=1.000 n=12
Parse Time 0.246s (± 1.24%) 0.252s (± 1.88%) ~ 0.241s 0.267s p=0.054 n=12
Bind Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Check Time 1.676s (± 0.54%) 1.710s (± 0.45%) +0.034s (+ 2.01%) 1.684s 1.725s p=0.000 n=12
Emit Time 0.301s (± 5.79%) 0.309s (± 2.98%) ~ 0.286s 0.332s p=0.541 n=12
Total Time 19.780s (± 0.93%) 19.936s (± 0.55%) ~ 19.615s 20.213s p=0.198 n=12
strada-compiler - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 334,953 326,478 -8,475 (- 2.53%) ~ ~ p=0.000 n=12
Types 197,616 193,451 -4,165 (- 2.11%) ~ ~ p=0.000 n=12
Memory Used 247,503k (± 0.02%) 244,059k (± 0.02%) -3,444k (- 1.39%) 243,926k 244,211k p=0.000 n=12
Memory Allocs 4,329,460 (± 0.01%) 4,312,790 (± 0.01%) -16,670 (- 0.39%) 4,311,951 4,313,815 p=0.000 n=12
Config Time 0.001s 0.001s ~ ~ ~ p=1.000 n=12
Parse Time 0.114s (± 4.50%) 0.113s (± 4.99%) ~ 0.101s 0.129s p=0.723 n=12
Bind Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Check Time 0.961s (± 0.58%) 0.923s (± 0.59%) 🟩-0.038s (- 3.93%) 0.913s 0.942s p=0.000 n=12
Emit Time 0.149s (± 6.41%) 0.161s (± 4.24%) ~ 0.142s 0.176s p=0.103 n=12
Total Time 1.282s (± 0.67%) 1.251s (± 0.68%) -0.031s (- 2.43%) 1.226s 1.269s p=0.000 n=12
ts-pre-modules - native
Errors 87 87 ~ ~ ~ p=1.000 n=12
Symbols 303,834 279,513 🟩-24,321 (- 8.00%) ~ ~ p=0.000 n=12
Types 181,663 156,859 🟩-24,804 (-13.65%) ~ ~ p=0.000 n=12
Memory Used 215,232k (± 0.03%) 203,413k (± 0.02%) 🟩-11,819k (- 5.49%) 203,333k 203,492k p=0.000 n=12
Memory Allocs 1,401,503 (± 0.02%) 1,312,953 (± 0.01%) 🟩-88,550 (- 6.32%) 1,312,401 1,313,445 p=0.000 n=12
Config Time 0.000s (±146.91%) 0.000s ~ ~ ~ p=0.478 n=12
Parse Time 0.098s (± 5.41%) 0.099s (± 4.36%) ~ 0.089s 0.111s p=0.809 n=12
Bind Time 0.037s (± 9.20%) 0.040s (± 8.88%) ~ 0.031s 0.050s p=0.091 n=12
Check Time 0.753s (± 0.63%) 0.795s (± 1.01%) 🔻+0.042s (+ 5.56%) 0.779s 0.816s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 0.903s (± 0.75%) 0.949s (± 0.75%) 🔻+0.046s (+ 5.10%) 0.934s 0.970s p=0.000 n=12
vscode - native
Errors 396 396 ~ ~ ~ p=1.000 n=12
Symbols 12,355,236 12,105,462 -249,774 (- 2.02%) ~ ~ p=0.000 n=12
Types 4,042,799 3,926,396 -116,403 (- 2.88%) ~ ~ p=0.000 n=12
Memory Used 5,814,340k (± 0.03%) 5,714,904k (± 0.02%) -99,436k (- 1.71%) 5,712,373k 5,717,760k p=0.000 n=12
Memory Allocs 59,202,042 (± 0.02%) 58,700,256 (± 0.02%) -501,786 (- 0.85%) 58,682,308 58,736,943 p=0.000 n=12
Config Time 0.076s (± 0.51%) 0.077s (± 0.37%) ~ 0.076s 0.077s p=0.076 n=12
Parse Time 1.849s (± 2.15%) 1.850s (± 1.97%) ~ 1.731s 1.917s p=0.856 n=12
Bind Time 0.641s (±12.78%) 0.638s (±13.56%) ~ 0.509s 0.951s p=0.921 n=12
Check Time 15.805s (± 1.08%) 15.589s (± 0.79%) ~ 15.316s 15.814s p=0.319 n=12
Emit Time 5.928s (± 6.49%) 5.335s (± 9.09%) 🟩-0.593s (-10.01%) 4.653s 6.412s p=0.028 n=12
Total Time 24.393s (± 1.04%) 23.593s (± 1.83%) 🟩-0.800s (- 3.28%) 22.947s 24.518s p=0.005 n=12
webpack - native
Errors 598 598 ~ ~ ~ p=1.000 n=12
Symbols 1,343,495 1,322,058 -21,437 (- 1.60%) ~ ~ p=0.000 n=12
Types 651,492 630,925 🟩-20,567 (- 3.16%) ~ ~ p=0.000 n=12
Memory Used 855,315k (± 0.02%) 826,210k (± 0.02%) 🟩-29,105k (- 3.40%) 825,703k 826,481k p=0.000 n=12
Memory Allocs 6,769,515 (± 0.03%) 6,650,608 (± 0.02%) -118,907 (- 1.76%) 6,649,046 6,657,208 p=0.000 n=12
Config Time 0.009s 0.009s ~ ~ ~ p=1.000 n=12
Parse Time 0.316s (± 2.44%) 0.312s (± 3.40%) ~ 0.281s 0.342s p=0.560 n=12
Bind Time 0.072s (± 2.02%) 0.075s (±11.28%) ~ 0.068s 0.116s p=0.637 n=12
Check Time 2.775s (± 0.40%) 2.493s (± 0.36%) 🟩-0.282s (-10.16%) 2.475s 2.518s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 3.208s (± 0.32%) 2.927s (± 0.26%) 🟩-0.281s (- 8.75%) 2.897s 2.938s p=0.000 n=12
xstate-main - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,060,810 1,053,219 -7,591 (- 0.72%) ~ ~ p=0.000 n=12
Types 389,671 384,406 -5,265 (- 1.35%) ~ ~ p=0.000 n=12
Memory Used 471,792k (± 0.02%) 467,624k (± 0.02%) -4,168k (- 0.88%) 467,409k 467,960k p=0.000 n=12
Memory Allocs 4,217,827 (± 0.04%) 4,179,687 (± 0.08%) -38,141 (- 0.90%) 4,172,463 4,188,103 p=0.000 n=12
Config Time 0.003s 0.003s ~ ~ ~ p=1.000 n=12
Parse Time 0.133s (± 2.28%) 0.136s (± 4.03%) ~ 0.128s 0.152s p=0.680 n=12
Bind Time 0.049s (±13.24%) 0.045s (±13.34%) ~ 0.035s 0.067s p=0.367 n=12
Check Time 1.086s (± 0.49%) 1.014s (± 1.13%) 🟩-0.072s (- 6.61%) 0.994s 1.045s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 1.278s (± 0.71%) 1.206s (± 1.07%) 🟩-0.072s (- 5.63%) 1.183s 1.239s p=0.000 n=12
System info unknown
Hosts
  • native
Scenarios
  • Compiler-Unions - native
  • angular-1 - native
  • mui-docs - native
  • strada-build-src - native
  • strada-compiler - native
  • ts-pre-modules - native
  • vscode - native
  • webpack - native
  • xstate-main - native
Benchmark Name Iterations
Current pr 12
Baseline baseline 12

Developer Information:

Download Benchmarks

@jakebailey

Copy link
Copy Markdown
Member Author

TypeScript Bot (@typescript-bot) perf test this faster

@typescript-automation

typescript-automation Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this faster ✅ Started 👀 Results

@typescript-automation

Copy link
Copy Markdown
Contributor

Jake Bailey (@jakebailey)
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - native
Errors 41 41 ~ ~ ~ p=1.000 n=12
Symbols 115,276 115,276 ~ ~ ~ p=1.000 n=12
Types 79,613 79,613 ~ ~ ~ p=1.000 n=12
Memory Used 113,474k (± 0.04%) 113,536k (± 0.06%) ~ 113,430k 113,707k p=0.164 n=12
Memory Allocs 2,212,634 (± 0.01%) 2,212,670 (± 0.01%) ~ 2,212,418 2,212,953 p=0.977 n=12
Config Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Parse Time 0.039s (± 4.37%) 0.040s (± 4.16%) ~ 0.035s 0.043s p=0.108 n=12
Bind Time 0.014s (±12.85%) 0.015s (±10.22%) ~ 0.011s 0.018s p=0.485 n=12
Check Time 0.468s (± 0.90%) 0.465s (± 0.69%) ~ 0.459s 0.473s p=0.449 n=12
Emit Time 0.270s (± 2.60%) 0.266s (± 2.49%) ~ 0.248s 0.277s p=0.234 n=12
Total Time 0.797s (± 0.71%) 0.792s (± 1.07%) ~ 0.767s 0.814s p=0.271 n=12
angular-1 - native
Errors 3 3 ~ ~ ~ p=1.000 n=12
Symbols 846,419 (± 0.11%) 818,558 (± 0.09%) 🟩-27,861 (- 3.29%) 816,163 820,221 p=0.000 n=12
Types 247,032 (± 0.00%) 229,090 (± 0.00%) 🟩-17,942 (- 7.26%) 229,088 229,098 p=0.000 n=12
Memory Used 596,303k (± 0.03%) 584,572k (± 0.02%) -11,730k (- 1.97%) 584,263k 584,880k p=0.000 n=12
Memory Allocs 13,142,974 (± 0.05%) 13,029,569 (± 0.03%) -113,405 (- 0.86%) 13,021,317 13,043,411 p=0.000 n=12
Config Time 0.017s (± 1.70%) 0.017s ~ ~ ~ p=0.217 n=12
Parse Time 0.223s (± 2.59%) 0.226s (± 2.84%) ~ 0.209s 0.248s p=0.247 n=12
Bind Time 0.084s (±12.60%) 0.076s (±13.13%) ~ 0.059s 0.105s p=0.202 n=12
Check Time 0s 0s ~ ~ ~ p=1.000 n=12
Emit Time 1.561s (± 1.59%) 1.563s (± 1.58%) ~ 1.504s 1.618s p=0.921 n=12
Total Time 1.905s (± 1.39%) 1.906s (± 1.18%) ~ 1.830s 1.956s p=0.932 n=12
arktype - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,757,463 1,733,125 -24,338 (- 1.38%) ~ ~ p=0.000 n=12
Types 943,966 851,913 🟩-92,053 (- 9.75%) ~ ~ p=0.000 n=12
Memory Used 703,726k (± 0.01%) 670,178k (± 0.01%) 🟩-33,548k (- 4.77%) 669,891k 670,362k p=0.000 n=12
Memory Allocs 13,291,836 (± 0.01%) 12,027,612 (± 0.01%) 🟩-1,264,224 (- 9.51%) 12,025,468 12,032,754 p=0.000 n=12
Config Time 0.002s 0.002s ~ ~ ~ p=1.000 n=12
Parse Time 0.071s (± 5.60%) 0.071s (± 3.94%) ~ 0.065s 0.076s p=0.876 n=12
Bind Time 0.021s (±11.06%) 0.020s (± 3.07%) ~ 0.019s 0.022s p=0.402 n=12
Check Time 2.066s (± 0.41%) 2.089s (± 0.36%) +0.023s (+ 1.13%) 2.067s 2.107s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 2.202s (± 0.41%) 2.221s (± 0.33%) +0.019s (+ 0.85%) 2.201s 2.242s p=0.001 n=12
mui-docs - native
Errors 11,405 11,405 ~ ~ ~ p=1.000 n=12
Symbols 4,331,983 4,211,620 -120,363 (- 2.78%) ~ ~ p=0.000 n=12
Types 1,370,520 1,343,389 -27,131 (- 1.98%) ~ ~ p=0.000 n=12
Memory Used 2,626,674k (± 0.06%) 2,574,789k (± 0.05%) -51,885k (- 1.98%) 2,571,842k 2,577,669k p=0.000 n=12
Memory Allocs 27,710,946 (± 0.06%) 26,883,956 (± 0.06%) -826,989 (- 2.98%) 26,839,976 26,929,313 p=0.000 n=12
Config Time 0.017s 0.017s ~ ~ ~ p=1.000 n=12
Parse Time 0.482s (± 2.19%) 0.473s (± 2.25%) ~ 0.443s 0.505s p=0.131 n=12
Bind Time 0.002s 0.002s (±24.21%) ~ 0.002s 0.005s p=1.000 n=12
Check Time 7.257s (± 0.47%) 7.809s (± 0.35%) 🔻+0.553s (+ 7.62%) 7.706s 7.853s p=0.000 n=12
Emit Time 0.511s (±10.27%) 0.442s (± 5.87%) ~ 0.422s 0.566s p=0.270 n=12
Total Time 9.019s (± 0.62%) 9.454s (± 0.20%) 🔻+0.435s (+ 4.83%) 9.407s 9.522s p=0.000 n=12
sentry - native
Errors 2 2 ~ ~ ~ p=1.000 n=12
Symbols 7,459,653 6,934,523 🟩-525,130 (- 7.04%) ~ ~ p=0.000 n=12
Types 2,647,172 2,444,276 🟩-202,896 (- 7.66%) ~ ~ p=0.000 n=12
Memory Used 3,803,754k (± 0.04%) 3,572,136k (± 0.02%) 🟩-231,619k (- 6.09%) 3,570,251k 3,575,448k p=0.000 n=12
Memory Allocs 77,176,673 (± 0.08%) 73,161,720 (± 0.11%) 🟩-4,014,953 (- 5.20%) 72,991,329 73,502,303 p=0.000 n=12
Config Time 0.060s (± 0.47%) 0.061s (± 0.30%) +0.001s (+ 1.38%) 0.061s 0.062s p=0.000 n=12
Parse Time 1.010s (± 2.17%) 1.009s (± 1.93%) ~ 0.967s 1.060s p=1.000 n=12
Bind Time 0.341s (±13.22%) 0.342s (±13.71%) ~ 0.262s 0.470s p=0.932 n=12
Check Time 14.911s (± 0.63%) 13.937s (± 0.94%) 🟩-0.974s (- 6.53%) 13.753s 14.247s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 16.402s (± 0.57%) 15.444s (± 0.66%) 🟩-0.958s (- 5.84%) 15.305s 15.690s p=0.000 n=12
social-app - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 2,568,629 2,463,405 🟩-105,224 (- 4.10%) ~ ~ p=0.000 n=12
Types 1,163,035 1,073,249 🟩-89,786 (- 7.72%) ~ ~ p=0.000 n=12
Memory Used 1,212,976k (± 0.01%) 1,164,141k (± 0.01%) 🟩-48,835k (- 4.03%) 1,163,853k 1,164,569k p=0.000 n=12
Memory Allocs 13,977,083 (± 0.12%) 13,077,705 (± 0.26%) 🟩-899,377 (- 6.43%) 13,010,783 13,175,582 p=0.000 n=12
Config Time 0.017s (± 1.65%) 0.018s (± 1.75%) ~ 0.017s 0.018s p=0.100 n=12
Parse Time 0.355s (± 1.88%) 0.343s (± 1.51%) 🟩-0.012s (- 3.31%) 0.331s 0.360s p=0.021 n=12
Bind Time 0.077s (± 5.07%) 0.091s (±12.91%) 🔻+0.013s (+17.42%) 0.075s 0.129s p=0.017 n=12
Check Time 3.618s (± 0.36%) 3.395s (± 0.45%) 🟩-0.223s (- 6.17%) 3.330s 3.421s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 4.093s (± 0.39%) 3.878s (± 0.54%) 🟩-0.214s (- 5.24%) 3.806s 3.922s p=0.000 n=12
strada-build-src - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,388,680 1,381,084 -7,596 (- 0.55%) ~ ~ p=0.000 n=12
Types 441,862 437,491 -4,371 (- 0.99%) ~ ~ p=0.000 n=12
Memory Used 1,313,210k (± 0.72%) 1,317,847k (± 0.73%) ~ 1,297,617k 1,337,873k p=0.347 n=12
Memory Allocs 78,230,405 (± 0.07%) 78,297,204 (± 0.05%) ~ 78,198,764 78,396,452 p=0.101 n=12
Config Time 0.004s (±12.82%) 0.004s (±15.55%) ~ 0.003s 0.006s p=0.428 n=12
Parse Time 0.311s (± 2.17%) 0.315s (± 1.35%) ~ 0.306s 0.326s p=0.619 n=12
Bind Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Check Time 2.002s (± 0.44%) 2.053s (± 0.44%) +0.050s (+ 2.52%) 2.032s 2.077s p=0.000 n=12
Emit Time 0.351s (± 3.24%) 0.379s (± 4.92%) 🔻+0.028s (+ 8.02%) 0.332s 0.421s p=0.021 n=12
Total Time 23.811s (± 0.66%) 24.094s (± 0.81%) +0.283s (+ 1.19%) 23.582s 24.502s p=0.028 n=12
strada-compiler - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 334,953 326,478 -8,475 (- 2.53%) ~ ~ p=0.000 n=12
Types 197,616 193,451 -4,165 (- 2.11%) ~ ~ p=0.000 n=12
Memory Used 247,512k (± 0.04%) 244,127k (± 0.02%) -3,386k (- 1.37%) 243,947k 244,257k p=0.000 n=12
Memory Allocs 4,329,246 (± 0.01%) 4,313,135 (± 0.01%) -16,111 (- 0.37%) 4,311,898 4,314,014 p=0.000 n=12
Config Time 0.001s 0.001s ~ ~ ~ p=1.000 n=12
Parse Time 0.118s (± 4.59%) 0.114s (± 3.42%) ~ 0.105s 0.126s p=0.163 n=12
Bind Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Check Time 0.962s (± 0.78%) 0.926s (± 0.49%) 🟩-0.036s (- 3.79%) 0.915s 0.935s p=0.000 n=12
Emit Time 0.148s (± 7.41%) 0.161s (± 3.55%) 🔻+0.012s (+ 8.26%) 0.148s 0.174s p=0.029 n=12
Total Time 1.281s (± 1.15%) 1.256s (± 0.74%) -0.025s (- 1.96%) 1.231s 1.275s p=0.010 n=12
t3code-server - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 11,588,087 9,333,683 🟩-2,254,404 (-19.45%) ~ ~ p=0.000 n=12
Types 3,062,556 2,554,646 🟩-507,910 (-16.58%) ~ ~ p=0.000 n=12
Memory Used 3,946,336k (± 0.01%) 3,323,630k (± 0.01%) 🟩-622,706k (-15.78%) 3,322,862k 3,325,423k p=0.000 n=12
Memory Allocs 41,539,686 (± 0.03%) 33,931,758 (± 0.05%) 🟩-7,607,928 (-18.31%) 33,886,323 33,969,774 p=0.000 n=12
Config Time 0.006s 0.006s ~ ~ ~ p=1.000 n=12
Parse Time 0.565s (± 2.60%) 0.566s (± 2.76%) ~ 0.531s 0.603s p=0.898 n=12
Bind Time 0.189s (±12.00%) 0.189s (±15.18%) ~ 0.149s 0.265s p=0.599 n=12
Check Time 18.750s (± 0.54%) 17.463s (± 0.50%) 🟩-1.287s (- 6.86%) 17.231s 17.760s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 19.526s (± 0.50%) 18.244s (± 0.47%) 🟩-1.282s (- 6.57%) 18.009s 18.514s p=0.000 n=12
ts-pre-modules - native
Errors 87 87 ~ ~ ~ p=1.000 n=12
Symbols 303,834 279,513 🟩-24,321 (- 8.00%) ~ ~ p=0.000 n=12
Types 181,663 156,859 🟩-24,804 (-13.65%) ~ ~ p=0.000 n=12
Memory Used 215,247k (± 0.01%) 203,409k (± 0.01%) 🟩-11,837k (- 5.50%) 203,374k 203,464k p=0.000 n=12
Memory Allocs 1,401,629 (± 0.01%) 1,313,052 (± 0.01%) 🟩-88,577 (- 6.32%) 1,312,497 1,313,427 p=0.000 n=12
Config Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Parse Time 0.095s (± 3.99%) 0.097s (± 4.49%) ~ 0.086s 0.106s p=0.560 n=12
Bind Time 0.042s (± 9.09%) 0.041s (±10.29%) ~ 0.033s 0.051s p=0.701 n=12
Check Time 0.744s (± 0.91%) 0.787s (± 0.73%) 🔻+0.043s (+ 5.76%) 0.774s 0.804s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 0.896s (± 0.89%) 0.939s (± 0.72%) 🔻+0.043s (+ 4.76%) 0.925s 0.960s p=0.000 n=12
vscode - native
Errors 396 396 ~ ~ ~ p=1.000 n=12
Symbols 12,464,383 12,115,031 -349,352 (- 2.80%) ~ ~ p=0.000 n=12
Types 4,076,536 3,926,794 🟩-149,742 (- 3.67%) ~ ~ p=0.000 n=12
Memory Used 5,841,194k (± 0.02%) 5,716,013k (± 0.02%) -125,182k (- 2.14%) 5,713,916k 5,718,292k p=0.000 n=12
Memory Allocs 59,410,530 (± 0.01%) 58,708,800 (± 0.03%) -701,730 (- 1.18%) 58,690,361 58,784,932 p=0.000 n=12
Config Time 0.076s (± 0.41%) 0.077s (± 0.37%) +0.001s (+ 1.20%) 0.077s 0.078s p=0.001 n=12
Parse Time 1.839s (± 2.32%) 1.865s (± 2.64%) ~ 1.722s 2.002s p=0.326 n=12
Bind Time 0.670s (±13.40%) 0.619s (±15.44%) ~ 0.515s 0.957s p=0.298 n=12
Check Time 16.106s (± 1.15%) 15.704s (± 0.89%) -0.403s (- 2.50%) 15.352s 16.024s p=0.002 n=12
Emit Time 5.632s (± 7.07%) 5.395s (± 8.03%) ~ 4.490s 6.393s p=0.378 n=12
Total Time 24.414s (± 1.14%) 23.763s (± 1.68%) -0.651s (- 2.67%) 22.850s 24.556s p=0.012 n=12
webpack - native
Errors 598 598 ~ ~ ~ p=1.000 n=12
Symbols 1,343,495 1,322,058 -21,437 (- 1.60%) ~ ~ p=0.000 n=12
Types 651,492 630,925 🟩-20,567 (- 3.16%) ~ ~ p=0.000 n=12
Memory Used 855,244k (± 0.01%) 826,198k (± 0.03%) 🟩-29,046k (- 3.40%) 825,570k 826,753k p=0.000 n=12
Memory Allocs 6,769,130 (± 0.01%) 6,651,290 (± 0.01%) -117,840 (- 1.74%) 6,649,464 6,654,323 p=0.000 n=12
Config Time 0.009s 0.009s ~ ~ ~ p=1.000 n=12
Parse Time 0.315s (± 2.38%) 0.303s (± 4.06%) ~ 0.260s 0.328s p=0.139 n=12
Bind Time 0.079s (±14.55%) 0.083s (±14.80%) ~ 0.070s 0.131s p=0.897 n=12
Check Time 2.784s (± 0.52%) 2.500s (± 0.49%) 🟩-0.284s (-10.20%) 2.460s 2.528s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 3.224s (± 0.26%) 2.937s (± 0.58%) 🟩-0.288s (- 8.92%) 2.899s 2.989s p=0.000 n=12
xstate-main - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,060,810 1,053,219 -7,591 (- 0.72%) ~ ~ p=0.000 n=12
Types 389,671 384,406 -5,265 (- 1.35%) ~ ~ p=0.000 n=12
Memory Used 471,744k (± 0.02%) 467,606k (± 0.03%) -4,138k (- 0.88%) 467,386k 467,935k p=0.000 n=12
Memory Allocs 4,221,519 (± 0.13%) 4,179,143 (± 0.09%) -42,376 (- 1.00%) 4,169,495 4,189,645 p=0.000 n=12
Config Time 0.002s 0.002s ~ ~ ~ p=1.000 n=12
Parse Time 0.106s (± 3.20%) 0.109s (± 3.87%) ~ 0.101s 0.122s p=0.295 n=12
Bind Time 0.039s (±17.44%) 0.033s (±16.99%) ~ 0.026s 0.051s p=0.214 n=12
Check Time 0.913s (± 0.83%) 0.839s (± 1.25%) 🟩-0.074s (- 8.08%) 0.819s 0.870s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 1.066s (± 0.61%) 0.988s (± 1.23%) 🟩-0.077s (- 7.25%) 0.965s 1.025s p=0.000 n=12
System info unknown
Hosts
  • native
Scenarios
  • Compiler-Unions - native
  • angular-1 - native
  • arktype - native
  • mui-docs - native
  • sentry - native
  • social-app - native
  • strada-build-src - native
  • strada-compiler - native
  • t3code-server - native
  • ts-pre-modules - native
  • vscode - native
  • webpack - native
  • xstate-main - native
Benchmark Name Iterations
Current pr 12
Baseline baseline 12

Developer Information:

Download Benchmarks

An import's weight is the total syntax weight divided by the number of
imports, so that imports and syntax contribute equally. A program with
almost no imports, such as the pre-modules TypeScript compiler, which is
built from namespaces, has 12 imports in total, making each one worth
14 times an average file. sys.ts was then estimated at more than twice
checker.ts, got a checker to itself, and the checker with checker.ts
also had to take a share of the remaining files.

An import is now worth at most twice the average weighted file. Every
other measured project has imports worth well under that, so their
assignments do not change.

Measured with --checkers 4 (n=6):

                       │  before.txt  │             after.txt              │
                       │    sec/op    │   sec/op     vs base               │
    Tsc/ts-pre-modules   730.0m ±  4%   653.0m ± 3%  -10.55% (p=0.002 n=6)
    Tsc/mui-docs          7.781 ±  1%    7.723 ± 1%        ~ (p=0.121 n=6)
    Tsc/bluesky           2.915 ±  4%    2.909 ± 1%        ~ (p=0.513 n=6)
    Tsc/sentry            11.90 ±  3%    11.85 ± 2%        ~ (p=0.937 n=6)
    Tsc/t3code-server     15.35 ± 10%    15.33 ± 4%        ~ (p=0.565 n=6)
    Tsc/botvscode         7.429 ±  0%    7.518 ± 3%   +1.20% (p=0.004 n=6)
    geomean               5.312          5.211        -1.89%

                       │  before.txt  │             after.txt              │
                       │  check-sec   │  check-sec   vs base               │
    Tsc/ts-pre-modules   618.5m ±  3%   539.5m ± 3%  -12.77% (p=0.002 n=6)
    Tsc/mui-docs          7.232 ±  1%    7.174 ± 1%        ~ (p=0.100 n=6)
    Tsc/bluesky           2.620 ±  3%    2.615 ± 2%        ~ (p=0.974 n=6)
    Tsc/sentry            11.03 ±  3%    10.99 ± 2%        ~ (p=1.000 n=6)
    Tsc/t3code-server     14.89 ± 10%    14.90 ± 4%        ~ (p=0.818 n=6)
    Tsc/botvscode         6.453 ±  0%    6.530 ± 3%   +1.19% (p=0.026 n=6)
    geomean               4.812          4.703        -2.28%

                       │  before.txt  │             after.txt              │
                       │    rss-B     │    rss-B      vs base              │
    Tsc/ts-pre-modules   262.5Mi ± 2%   271.7Mi ± 2%  +3.51% (p=0.002 n=6)
    Tsc/mui-docs         3.700Gi ± 7%   3.544Gi ± 8%       ~ (p=0.699 n=6)
    Tsc/bluesky          1.401Gi ± 4%   1.378Gi ± 2%       ~ (p=0.394 n=6)
    Tsc/sentry           4.473Gi ± 7%   4.418Gi ± 5%       ~ (p=0.589 n=6)
    Tsc/t3code-server    3.865Gi ± 2%   3.851Gi ± 2%       ~ (p=0.937 n=6)
    Tsc/botvscode        3.785Gi ± 2%   3.834Gi ± 2%       ~ (p=1.000 n=6)
    geomean              2.105Gi        2.095Gi       -0.47%

                       │  before.txt  │             after.txt              │
                       │    heap-B    │    heap-B     vs base              │
    Tsc/ts-pre-modules   198.5Mi ± 0%   206.8Mi ± 0%  +4.18% (p=0.002 n=6)
    Tsc/mui-docs         2.732Gi ± 0%   2.732Gi ± 0%       ~ (p=0.589 n=6)
    Tsc/bluesky          1.105Gi ± 0%   1.106Gi ± 0%       ~ (p=1.000 n=6)
    Tsc/sentry           3.400Gi ± 0%   3.400Gi ± 0%       ~ (p=1.000 n=6)
    Tsc/t3code-server    3.168Gi ± 0%   3.169Gi ± 0%       ~ (p=0.132 n=6)
    Tsc/botvscode        3.212Gi ± 0%   3.212Gi ± 0%       ~ (p=0.589 n=6)
    geomean              1.651Gi        1.662Gi       +0.69%
@jakebailey

Copy link
Copy Markdown
Member Author

TypeScript Bot (@typescript-bot) perf test this faster

@typescript-automation

typescript-automation Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this faster ✅ Started 👀 Results

@typescript-automation

Copy link
Copy Markdown
Contributor

Jake Bailey (@jakebailey)
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - native
Errors 41 41 ~ ~ ~ p=1.000 n=12
Symbols 115,276 115,276 ~ ~ ~ p=1.000 n=12
Types 79,613 79,613 ~ ~ ~ p=1.000 n=12
Memory Used 113,537k (± 0.05%) 113,502k (± 0.04%) ~ 113,399k 113,710k p=0.514 n=12
Memory Allocs 2,212,554 (± 0.01%) 2,212,636 (± 0.01%) ~ 2,212,306 2,213,005 p=0.319 n=12
Config Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Parse Time 0.038s (± 2.50%) 0.040s (± 3.39%) 🔻+0.002s (+ 6.64%) 0.036s 0.044s p=0.005 n=12
Bind Time 0.014s (±10.26%) 0.014s (± 9.95%) ~ 0.011s 0.017s p=0.670 n=12
Check Time 0.468s (± 0.83%) 0.464s (± 0.99%) ~ 0.454s 0.475s p=0.234 n=12
Emit Time 0.267s (± 1.87%) 0.264s (± 1.32%) ~ 0.249s 0.270s p=0.367 n=12
Total Time 0.792s (± 0.79%) 0.789s (± 0.64%) ~ 0.779s 0.803s p=0.399 n=12
angular-1 - native
Errors 3 3 ~ ~ ~ p=1.000 n=12
Symbols 847,366 (± 0.05%) 817,851 (± 0.11%) 🟩-29,515 (- 3.48%) 815,894 820,112 p=0.000 n=12
Types 247,031 (± 0.00%) 229,092 (± 0.00%) 🟩-17,939 (- 7.26%) 229,088 229,101 p=0.000 n=12
Memory Used 596,387k (± 0.02%) 584,457k (± 0.03%) -11,929k (- 2.00%) 584,104k 585,090k p=0.000 n=12
Memory Allocs 13,140,524 (± 0.03%) 13,027,117 (± 0.05%) -113,407 (- 0.86%) 13,021,029 13,054,315 p=0.000 n=12
Config Time 0.017s (± 1.86%) 0.017s ~ ~ ~ p=0.093 n=12
Parse Time 0.223s (± 2.45%) 0.223s (± 2.74%) ~ 0.210s 0.248s p=0.832 n=12
Bind Time 0.079s (±13.48%) 0.087s (±13.74%) ~ 0.058s 0.109s p=0.291 n=12
Check Time 0s 0s ~ ~ ~ p=1.000 n=12
Emit Time 1.561s (± 1.70%) 1.541s (± 1.18%) ~ 1.512s 1.612s p=0.213 n=12
Total Time 1.900s (± 1.54%) 1.888s (± 0.85%) ~ 1.844s 1.935s p=0.311 n=12
arktype - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,757,463 1,733,125 -24,338 (- 1.38%) ~ ~ p=0.000 n=12
Types 943,966 851,913 🟩-92,053 (- 9.75%) ~ ~ p=0.000 n=12
Memory Used 703,745k (± 0.01%) 670,212k (± 0.01%) 🟩-33,534k (- 4.77%) 670,053k 670,388k p=0.000 n=12
Memory Allocs 13,290,587 (± 0.01%) 12,028,274 (± 0.02%) 🟩-1,262,313 (- 9.50%) 12,024,325 12,040,140 p=0.000 n=12
Config Time 0.002s 0.002s ~ ~ ~ p=1.000 n=12
Parse Time 0.069s (± 2.35%) 0.069s (± 3.64%) ~ 0.062s 0.076s p=0.465 n=12
Bind Time 0.020s (± 3.07%) 0.023s (±17.57%) ~ 0.018s 0.034s p=0.754 n=12
Check Time 2.066s (± 0.46%) 2.081s (± 0.45%) +0.015s (+ 0.73%) 2.051s 2.104s p=0.021 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 2.198s (± 0.47%) 2.215s (± 0.35%) +0.016s (+ 0.75%) 2.192s 2.233s p=0.009 n=12
mui-docs - native
Errors 11,405 11,405 ~ ~ ~ p=1.000 n=12
Symbols 4,331,983 4,211,620 -120,363 (- 2.78%) ~ ~ p=0.000 n=12
Types 1,370,520 1,343,389 -27,131 (- 1.98%) ~ ~ p=0.000 n=12
Memory Used 2,625,842k (± 0.04%) 2,574,155k (± 0.07%) -51,686k (- 1.97%) 2,569,757k 2,579,163k p=0.000 n=12
Memory Allocs 27,712,451 (± 0.07%) 26,885,857 (± 0.07%) -826,594 (- 2.98%) 26,834,293 26,916,603 p=0.000 n=12
Config Time 0.017s (± 1.95%) 0.017s +0.000s (+ 2.51%) ~ ~ p=0.037 n=12
Parse Time 0.465s (± 2.54%) 0.475s (± 2.35%) ~ 0.451s 0.506s p=0.224 n=12
Bind Time 0.002s 0.002s ~ ~ ~ p=1.000 n=12
Check Time 7.237s (± 0.42%) 7.796s (± 0.43%) 🔻+0.559s (+ 7.73%) 7.681s 7.840s p=0.000 n=12
Emit Time 0.561s (± 9.98%) 0.420s (± 0.51%) 🟩-0.141s (-25.07%) 0.414s 0.424s p=0.000 n=12
Total Time 9.027s (± 0.45%) 9.407s (± 0.56%) 🔻+0.380s (+ 4.21%) 9.200s 9.485s p=0.000 n=12
sentry - native
Errors 2 2 ~ ~ ~ p=1.000 n=12
Symbols 7,459,653 6,934,523 🟩-525,130 (- 7.04%) ~ ~ p=0.000 n=12
Types 2,647,172 2,444,276 🟩-202,896 (- 7.66%) ~ ~ p=0.000 n=12
Memory Used 3,803,383k (± 0.03%) 3,572,396k (± 0.03%) 🟩-230,987k (- 6.07%) 3,569,806k 3,576,644k p=0.000 n=12
Memory Allocs 77,170,308 (± 0.09%) 73,162,901 (± 0.09%) 🟩-4,007,407 (- 5.19%) 72,957,891 73,328,083 p=0.000 n=12
Config Time 0.060s (± 0.30%) 0.061s (± 0.30%) +0.001s (+ 1.39%) 0.060s 0.061s p=0.000 n=12
Parse Time 1.027s (± 1.59%) 1.031s (± 2.54%) ~ 0.981s 1.120s p=0.921 n=12
Bind Time 0.309s (± 6.68%) 0.292s (± 8.29%) ~ 0.249s 0.360s p=0.131 n=12
Check Time 14.928s (± 0.60%) 13.984s (± 0.73%) 🟩-0.945s (- 6.33%) 13.785s 14.252s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 16.404s (± 0.64%) 15.461s (± 0.63%) 🟩-0.943s (- 5.75%) 15.296s 15.700s p=0.000 n=12
social-app - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 2,568,629 2,463,405 🟩-105,224 (- 4.10%) ~ ~ p=0.000 n=12
Types 1,163,035 1,073,249 🟩-89,786 (- 7.72%) ~ ~ p=0.000 n=12
Memory Used 1,212,895k (± 0.01%) 1,164,076k (± 0.02%) 🟩-48,819k (- 4.03%) 1,163,690k 1,164,577k p=0.000 n=12
Memory Allocs 13,995,170 (± 0.17%) 13,066,511 (± 0.21%) 🟩-928,659 (- 6.64%) 13,008,850 13,137,162 p=0.000 n=12
Config Time 0.017s (± 1.06%) 0.018s (± 1.75%) +0.001s (+ 3.41%) 0.017s 0.018s p=0.009 n=12
Parse Time 0.351s (± 1.64%) 0.354s (± 1.68%) ~ 0.335s 0.365s p=0.417 n=12
Bind Time 0.080s (±11.17%) 0.079s (± 9.43%) ~ 0.073s 0.115s p=0.989 n=12
Check Time 3.630s (± 0.30%) 3.388s (± 0.22%) 🟩-0.242s (- 6.66%) 3.370s 3.419s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 4.100s (± 0.36%) 3.865s (± 0.29%) 🟩-0.235s (- 5.73%) 3.840s 3.905s p=0.000 n=12
strada-build-src - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,388,680 1,381,084 -7,596 (- 0.55%) ~ ~ p=0.000 n=12
Types 441,862 437,491 -4,371 (- 0.99%) ~ ~ p=0.000 n=12
Memory Used 1,309,764k (± 0.58%) 1,314,746k (± 0.51%) ~ 1,300,553k 1,331,091k p=0.410 n=12
Memory Allocs 78,245,821 (± 0.06%) 78,279,781 (± 0.06%) ~ 78,143,577 78,378,587 p=0.319 n=12
Config Time 0.005s (±10.49%) 0.005s (±14.41%) ~ 0.004s 0.008s p=0.584 n=12
Parse Time 0.310s (± 1.38%) 0.309s (± 2.72%) ~ 0.289s 0.334s p=0.943 n=12
Bind Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Check Time 2.007s (± 0.53%) 2.053s (± 0.53%) +0.046s (+ 2.30%) 2.029s 2.084s p=0.000 n=12
Emit Time 0.345s (± 4.11%) 0.368s (± 4.23%) 🔻+0.023s (+ 6.69%) 0.328s 0.420s p=0.013 n=12
Total Time 23.799s (± 0.57%) 24.055s (± 0.56%) +0.256s (+ 1.08%) 23.704s 24.445s p=0.010 n=12
strada-compiler - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 334,953 326,478 -8,475 (- 2.53%) ~ ~ p=0.000 n=12
Types 197,616 193,451 -4,165 (- 2.11%) ~ ~ p=0.000 n=12
Memory Used 247,531k (± 0.04%) 244,078k (± 0.03%) -3,452k (- 1.39%) 243,877k 244,329k p=0.000 n=12
Memory Allocs 4,329,432 (± 0.01%) 4,312,961 (± 0.01%) -16,471 (- 0.38%) 4,311,414 4,313,967 p=0.000 n=12
Config Time 0.001s 0.001s ~ ~ ~ p=1.000 n=12
Parse Time 0.109s (± 4.07%) 0.114s (± 5.05%) ~ 0.100s 0.126s p=0.164 n=12
Bind Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Check Time 0.965s (± 0.65%) 0.932s (± 0.74%) 🟩-0.032s (- 3.35%) 0.914s 0.954s p=0.000 n=12
Emit Time 0.146s (± 6.70%) 0.154s (± 4.29%) ~ 0.134s 0.166s p=0.193 n=12
Total Time 1.276s (± 1.03%) 1.252s (± 1.03%) -0.024s (- 1.89%) 1.208s 1.278s p=0.019 n=12
t3code-server - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 11,588,087 9,333,683 🟩-2,254,404 (-19.45%) ~ ~ p=0.000 n=12
Types 3,062,556 2,554,646 🟩-507,910 (-16.58%) ~ ~ p=0.000 n=12
Memory Used 3,946,612k (± 0.03%) 3,323,643k (± 0.01%) 🟩-622,969k (-15.78%) 3,322,583k 3,324,653k p=0.000 n=12
Memory Allocs 41,548,645 (± 0.07%) 33,967,225 (± 0.09%) 🟩-7,581,420 (-18.25%) 33,901,605 34,085,086 p=0.000 n=12
Config Time 0.006s 0.006s (± 2.98%) ~ 0.006s 0.007s p=1.000 n=12
Parse Time 0.577s (± 2.70%) 0.573s (± 2.82%) ~ 0.536s 0.610s p=0.799 n=12
Bind Time 0.181s (±12.55%) 0.187s (±13.81%) ~ 0.149s 0.261s p=0.640 n=12
Check Time 18.687s (± 0.33%) 17.484s (± 0.63%) 🟩-1.202s (- 6.43%) 17.293s 17.872s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 19.469s (± 0.33%) 18.272s (± 0.60%) 🟩-1.197s (- 6.15%) 18.103s 18.663s p=0.000 n=12
ts-pre-modules - native
Errors 87 87 ~ ~ ~ p=1.000 n=12
Symbols 303,834 294,666 🟩-9,168 (- 3.02%) ~ ~ p=0.000 n=12
Types 181,663 176,285 -5,378 (- 2.96%) ~ ~ p=0.000 n=12
Memory Used 215,252k (± 0.02%) 211,908k (± 0.02%) -3,345k (- 1.55%) 211,791k 212,036k p=0.000 n=12
Memory Allocs 1,401,528 (± 0.01%) 1,389,183 (± 0.03%) -12,345 (- 0.88%) 1,388,492 1,390,509 p=0.000 n=12
Config Time 0.000s 0.000s (±217.90%) ~ 0.000s 0.001s p=1.000 n=12
Parse Time 0.098s (± 3.92%) 0.097s (± 3.12%) ~ 0.090s 0.105s p=0.522 n=12
Bind Time 0.043s (± 8.33%) 0.044s (± 8.52%) ~ 0.030s 0.051s p=0.560 n=12
Check Time 0.741s (± 0.96%) 0.672s (± 0.63%) 🟩-0.069s (- 9.27%) 0.665s 0.685s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 0.897s (± 0.64%) 0.826s (± 0.43%) 🟩-0.071s (- 7.91%) 0.817s 0.836s p=0.000 n=12
vscode - native
Errors 396 396 ~ ~ ~ p=1.000 n=12
Symbols 12,554,726 12,232,006 -322,720 (- 2.57%) ~ ~ p=0.000 n=12
Types 4,118,060 3,966,973 🟩-151,087 (- 3.67%) ~ ~ p=0.000 n=12
Memory Used 5,881,511k (± 0.02%) 5,757,903k (± 0.01%) -123,607k (- 2.10%) 5,756,012k 5,760,375k p=0.000 n=12
Memory Allocs 59,798,376 (± 0.02%) 59,029,629 (± 0.01%) -768,747 (- 1.29%) 59,019,750 59,044,860 p=0.000 n=12
Config Time 0.076s (± 0.48%) 0.077s (± 0.32%) +0.001s (+ 0.88%) 0.076s 0.077s p=0.010 n=12
Parse Time 1.855s (± 2.47%) 1.853s (± 2.53%) ~ 1.748s 1.984s p=0.965 n=12
Bind Time 0.656s (±12.56%) 0.647s (±15.93%) ~ 0.521s 0.941s p=0.898 n=12
Check Time 14.284s (± 1.43%) 16.191s (± 0.77%) 🔻+1.906s (+13.34%) 15.891s 16.379s p=0.000 n=12
Emit Time 5.556s (± 6.46%) 5.345s (± 8.34%) ~ 4.579s 6.448s p=0.291 n=12
Total Time 22.520s (± 1.03%) 24.214s (± 1.58%) 🔻+1.694s (+ 7.52%) 23.467s 25.218s p=0.000 n=12
webpack - native
Errors 598 598 ~ ~ ~ p=1.000 n=12
Symbols 1,343,495 1,322,058 -21,437 (- 1.60%) ~ ~ p=0.000 n=12
Types 651,491 630,925 🟩-20,566 (- 3.16%) ~ ~ p=0.000 n=12
Memory Used 855,250k (± 0.01%) 826,178k (± 0.01%) 🟩-29,072k (- 3.40%) 825,834k 826,530k p=0.000 n=12
Memory Allocs 6,769,127 (± 0.01%) 6,649,811 (± 0.01%) -119,316 (- 1.76%) 6,648,600 6,651,260 p=0.000 n=12
Config Time 0.009s 0.009s ~ ~ ~ p=1.000 n=12
Parse Time 0.313s (± 1.58%) 0.302s (± 3.39%) ~ 0.272s 0.327s p=0.091 n=12
Bind Time 0.074s (± 4.49%) 0.076s (± 7.67%) ~ 0.066s 0.096s p=0.943 n=12
Check Time 2.796s (± 0.51%) 2.520s (± 0.25%) 🟩-0.275s (- 9.85%) 2.504s 2.540s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 3.228s (± 0.39%) 2.948s (± 0.34%) 🟩-0.280s (- 8.69%) 2.921s 2.968s p=0.000 n=12
xstate-main - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,060,810 1,053,219 -7,591 (- 0.72%) ~ ~ p=0.000 n=12
Types 389,671 384,406 -5,265 (- 1.35%) ~ ~ p=0.000 n=12
Memory Used 471,835k (± 0.01%) 467,574k (± 0.02%) -4,261k (- 0.90%) 467,414k 467,958k p=0.000 n=12
Memory Allocs 4,222,122 (± 0.08%) 4,180,711 (± 0.12%) -41,410 (- 0.98%) 4,165,667 4,193,517 p=0.000 n=12
Config Time 0.002s 0.002s ~ ~ ~ p=1.000 n=12
Parse Time 0.109s (± 3.40%) 0.110s (± 2.74%) ~ 0.102s 0.118s p=0.599 n=12
Bind Time 0.032s (±14.99%) 0.029s (± 8.33%) ~ 0.027s 0.038s p=0.707 n=12
Check Time 0.907s (± 0.99%) 0.836s (± 0.75%) 🟩-0.071s (- 7.87%) 0.821s 0.852s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 1.058s (± 1.00%) 0.984s (± 0.85%) 🟩-0.074s (- 7.01%) 0.963s 1.001s p=0.000 n=12
System info unknown
Hosts
  • native
Scenarios
  • Compiler-Unions - native
  • angular-1 - native
  • arktype - native
  • mui-docs - native
  • sentry - native
  • social-app - native
  • strada-build-src - native
  • strada-compiler - native
  • t3code-server - native
  • ts-pre-modules - native
  • vscode - native
  • webpack - native
  • xstate-main - native
Benchmark Name Iterations
Current pr 12
Baseline baseline 12

Developer Information:

Download Benchmarks

@jakebailey

Copy link
Copy Markdown
Member Author

vscode is really sensitive to checker assignment, they have one file that takes a lot more time than otherwise, and benchmarks run unpinned and therefore we see noise

@jakebailey

Copy link
Copy Markdown
Member Author

Note

This comment was written by Copilot.

Why the bot regressed on vscode and mui-docs

The extra time comes from checkers that ended up with too much work. It isn't environmental noise.

  • vscode isn't pinned in typescript-benchmarking. Every bot run clones HEAD, so results from different runs aren't measuring the same code. I reproduced the regression locally on 8 vscode snapshots.
  • mui-docs is pinned (6780195), and local runs match the bot's numbers exactly: main 6.80s, this PR 7.35s (+8%).
  • Root cause: this PR groups files by directory, with each group about 1/64 of the total estimated weight, then assigns whole groups to checkers. The per-file cost estimate is often off by around 40%, and the error is correlated within a directory. One example: vscode's agentHost area costs about 1.7× its estimate. With only ~16 groups per checker, these errors don't average out, so each checker's real load can be off by roughly 10%. Whichever checker is overloaded sets the wall time. Main assigns individual files in an interleaved order, so its errors average out. The cost is lost locality: more duplicated types and more heap.

What I tried

All variants were deterministic, using static information only (no stealing, no runtime feedback):

  1. A one-sided variance penalty (κ) that spreads each directory area across checkers.
  2. Finer groups (1/32 and 1/64 of the total).
  3. Merging groups with label propagation.
  4. LPT (longest-processing-time) assignment.
  5. Discounting files that many others import (hubs).
  6. A package first-touch cost.
  7. Hedging against estimate error.
  8. Isolating very heavy files.

Options 4–8 were neutral or worse.

A single run isn't enough to judge a variant: two identical assignments can differ by 4–5% in wall time, and a small change to an assignment can swing results by ±10%. So I evaluated each variant as a distribution. I applied a deterministic ±5% per-file weight jitter under 5 seeds, giving 5 different assignments per variant, and ran each on 6 projects at --checkers 4, pinned to 7 cores.

Each cell is mean / worst run, as % vs main's mean (lower is better):

this PR κ=2 κ=2, groups 1/64 κ=2 + label propagation
vscode (Oct b) +6.6 / +10.9 −0.9 / +0.8 +1.3 / +4.1 +1.1 / +4.0
vscode (Oct a) +7.6 / +12.9 −0.3 / +18.1 +3.2 / +5.7 +0.9 / +4.3
mui-docs (bot pin) −0.1 / +1.5 −0.2 / +3.8 +0.5 / +8.1 −5.1 / +2.3
mui-docs (newer) −4.6 / −2.4 −1.1 / +1.5 −0.3 / +3.9 +0.8 / +4.0
t3code-server −4.5 / +0.3 +8.8 / +9.7 +5.2 / +6.1 +3.3 / +7.4
sentry −8.5 / −5.2 −6.1 / −3.7 −0.2 / +2.3 −8.3 / −3.0
geomean −0.8 −0.1 +1.6 −1.3
worst +12.9 +18.1 +8.1 +7.4

Main's own worst seed is +9.4% over its mean.

Takeaways

  • This PR's vscode regression is systematic. It's +6–8% on average across assignments, not an unlucky draw.
  • Locality and balance trade off per project. Keeping files together helps t3code-server and sentry (lower heap, fewer types). Spreading them out fixes vscode but costs t3code-server about 9% time and about 10% heap. No setting wins everywhere.
  • The best variant is only slightly better than main. κ=2 + label propagation averages −1.3%, with a worst case of +7.4%. That is within main's own spread and adds noticeable complexity.
  • Neither syntactic nor import-closure features predict where checking cost lands. Cost mostly falls on whichever checker first touches shared types, so I don't see a cheap estimator that would remove the noise.

Pinning vscode and webpack in typescript-benchmarking would make future comparisons like this much easier to interpret.

Estimated checker work is often far off, and the error is shared by a
whole directory. With directories placed as units, an underestimated
area can land mostly on one checker, which then sets the overall time.
This is what made VS Code (+13%) and MUI docs (+8%) regress: VS Code's
agentHost area, for example, costs about 1.7 times its estimate and was
concentrated on one checker.

Each checker's load now includes a penalty for holding more than its
fair share of any area, so large areas are spread while small ones stay
together. To recover locality across directory boundaries, groups are
built from smaller directories merged along imports with label
propagation, and a second pass lets groups follow neighbors placed
after them. The penalty uses integer arithmetic so that assignments do
not depend on the platform.

Because a single assignment's time swings by up to 10% with small
estimate changes, variants were compared over five assignments per
project, each with a different deterministic per-file weight jitter
(--checkers 4; mean / worst vs main's mean):

                      before          after
    VS Code (Oct b)   +6.6 / +10.9    +1.1 / +4.0
    VS Code (Oct a)   +7.6 / +12.9    +0.9 / +4.3
    MUI docs (bot)    -0.1 /  +1.5    -5.1 / +2.3
    MUI docs (newer)  -4.6 /  -2.4    +0.8 / +4.0
    t3code-server     -4.5 /  +0.3    +3.3 / +7.4
    sentry            -8.5 /  -5.2    -8.3 / -3.0
    geomean           -0.8            -1.3
    worst            +12.9            +7.4
@jakebailey

Copy link
Copy Markdown
Member Author

TypeScript Bot (@typescript-bot) perf test

@typescript-automation

typescript-automation Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test ✅ Started 👀 Results

@typescript-automation

Copy link
Copy Markdown
Contributor

Jake Bailey (@jakebailey)
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - native
Errors 41 41 ~ ~ ~ p=1.000 n=12
Symbols 115,276 115,720 +444 (+ 0.39%) ~ ~ p=0.000 n=12
Types 79,613 79,765 +152 (+ 0.19%) ~ ~ p=0.000 n=12
Memory Used 113,533k (± 0.06%) 113,910k (± 0.08%) +377k (+ 0.33%) 113,746k 114,198k p=0.000 n=12
Memory Allocs 2,212,582 (± 0.00%) 2,214,898 (± 0.01%) +2,316 (+ 0.10%) 2,214,577 2,215,237 p=0.000 n=12
Config Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Parse Time 0.048s (± 3.31%) 0.050s (± 4.76%) ~ 0.044s 0.056s p=0.294 n=12
Bind Time 0.017s (±11.80%) 0.016s (± 8.59%) ~ 0.013s 0.019s p=0.899 n=12
Check Time 0.561s (± 0.70%) 0.559s (± 0.42%) ~ 0.553s 0.565s p=0.539 n=12
Emit Time 0.326s (± 2.07%) 0.320s (± 1.88%) ~ 0.302s 0.335s p=0.117 n=12
Total Time 0.959s (± 0.65%) 0.953s (± 0.79%) ~ 0.938s 0.980s p=0.124 n=12
angular-1 - native
Errors 3 3 ~ ~ ~ p=1.000 n=12
Symbols 847,050 (± 0.05%) 826,702 (± 0.06%) -20,348 (- 2.40%) 825,174 827,843 p=0.000 n=12
Types 247,032 (± 0.00%) 238,757 (± 0.00%) 🟩-8,274 (- 3.35%) 238,754 238,766 p=0.000 n=12
Memory Used 596,384k (± 0.02%) 594,829k (± 0.02%) -1,555k (- 0.26%) 594,593k 595,050k p=0.000 n=12
Memory Allocs 13,144,084 (± 0.06%) 13,116,338 (± 0.05%) -27,746 (- 0.21%) 13,107,435 13,131,227 p=0.000 n=12
Config Time 0.020s 0.021s 🔻+0.001s (+ 5.00%) ~ ~ p=0.000 n=12
Parse Time 0.281s (± 2.17%) 0.289s (± 3.59%) ~ 0.269s 0.321s p=0.271 n=12
Bind Time 0.101s (±15.77%) 0.095s (±11.47%) ~ 0.072s 0.129s p=0.580 n=12
Check Time 0s 0s ~ ~ ~ p=1.000 n=12
Emit Time 1.839s (± 1.16%) 1.917s (± 2.02%) 🔻+0.078s (+ 4.26%) 1.839s 2.017s p=0.000 n=12
Total Time 2.258s (± 1.14%) 2.356s (± 1.53%) 🔻+0.098s (+ 4.35%) 2.255s 2.440s p=0.000 n=12
arktype - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,757,463 1,793,803 +36,340 (+ 2.07%) ~ ~ p=0.000 n=12
Types 943,966 963,697 +19,731 (+ 2.09%) ~ ~ p=0.000 n=12
Memory Used 703,677k (± 0.01%) 712,045k (± 0.01%) +8,368k (+ 1.19%) 711,766k 712,263k p=0.000 n=12
Memory Allocs 13,293,408 (± 0.02%) 13,402,375 (± 0.01%) +108,967 (+ 0.82%) 13,400,090 13,405,211 p=0.000 n=12
Config Time 0.003s 0.003s ~ ~ ~ p=1.000 n=12
Parse Time 0.090s (± 3.75%) 0.088s (± 3.81%) ~ 0.082s 0.097s p=0.310 n=12
Bind Time 0.028s (±14.12%) 0.030s (±16.07%) ~ 0.023s 0.044s p=0.700 n=12
Check Time 2.450s (± 0.26%) 2.385s (± 0.57%) -0.065s (- 2.65%) 2.354s 2.412s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 2.622s (± 0.26%) 2.571s (± 0.35%) -0.051s (- 1.95%) 2.547s 2.595s p=0.000 n=12
mui-docs - native
Errors 11,405 11,405 ~ ~ ~ p=1.000 n=12
Symbols 4,331,983 4,500,720 +168,737 (+ 3.90%) ~ ~ p=0.000 n=12
Types 1,370,520 1,457,129 🔻+86,609 (+ 6.32%) ~ ~ p=0.000 n=12
Memory Used 2,625,660k (± 0.06%) 2,843,642k (± 0.05%) 🔻+217,983k (+ 8.30%) 2,840,592k 2,848,061k p=0.000 n=12
Memory Allocs 27,732,111 (± 0.13%) 29,661,999 (± 0.10%) 🔻+1,929,888 (+ 6.96%) 29,622,520 29,801,692 p=0.000 n=12
Config Time 0.020s (± 1.21%) 0.020s (± 1.59%) ~ 0.020s 0.021s p=0.371 n=12
Parse Time 0.594s (± 3.25%) 0.589s (± 2.21%) ~ 0.546s 0.620s p=0.854 n=12
Bind Time 0.002s (±13.40%) 0.003s (±23.37%) ~ 0.002s 0.006s p=0.293 n=12
Check Time 8.628s (± 0.22%) 9.800s (± 0.30%) 🔻+1.172s (+13.59%) 9.722s 9.867s p=0.000 n=12
Emit Time 0.638s (±13.22%) 0.534s (± 3.93%) ~ 0.504s 0.579s p=0.541 n=12
Total Time 10.800s (± 0.81%) 11.825s (± 0.31%) 🔻+1.026s (+ 9.50%) 11.710s 11.918s p=0.000 n=12
sentry - native
Errors 2 2 ~ ~ ~ p=1.000 n=12
Symbols 7,459,653 7,234,728 🟩-224,925 (- 3.02%) ~ ~ p=0.000 n=12
Types 2,647,172 2,540,951 🟩-106,221 (- 4.01%) ~ ~ p=0.000 n=12
Memory Used 3,804,633k (± 0.05%) 3,679,536k (± 0.04%) 🟩-125,096k (- 3.29%) 3,676,507k 3,683,939k p=0.000 n=12
Memory Allocs 77,204,557 (± 0.09%) 74,041,504 (± 0.07%) 🟩-3,163,053 (- 4.10%) 73,876,900 74,193,353 p=0.000 n=12
Config Time 0.060s 0.061s (± 0.30%) +0.001s (+ 1.81%) 0.061s 0.062s p=0.000 n=12
Parse Time 1.011s (± 2.02%) 0.991s (± 1.57%) ~ 0.948s 1.030s p=0.147 n=12
Bind Time 0.321s (±12.13%) 0.328s (±11.94%) ~ 0.263s 0.452s p=0.810 n=12
Check Time 14.742s (± 0.38%) 14.029s (± 0.60%) 🟩-0.713s (- 4.84%) 13.898s 14.211s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 16.209s (± 0.37%) 15.540s (± 0.42%) 🟩-0.669s (- 4.13%) 15.418s 15.690s p=0.000 n=12
social-app - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 2,568,629 2,521,478 -47,151 (- 1.84%) ~ ~ p=0.000 n=12
Types 1,163,035 1,137,387 -25,648 (- 2.21%) ~ ~ p=0.000 n=12
Memory Used 1,213,026k (± 0.01%) 1,202,197k (± 0.01%) -10,828k (- 0.89%) 1,201,997k 1,202,400k p=0.000 n=12
Memory Allocs 13,986,896 (± 0.15%) 13,914,108 (± 0.09%) -72,788 (- 0.52%) 13,878,235 13,941,751 p=0.000 n=12
Config Time 0.017s (± 1.65%) 0.018s (± 1.84%) ~ 0.017s 0.018s p=0.214 n=12
Parse Time 0.353s (± 1.82%) 0.356s (± 1.63%) ~ 0.337s 0.367s p=0.259 n=12
Bind Time 0.080s (± 8.38%) 0.076s (± 3.72%) ~ 0.072s 0.089s p=0.210 n=12
Check Time 3.646s (± 0.40%) 3.469s (± 0.24%) 🟩-0.177s (- 4.87%) 3.449s 3.489s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 4.121s (± 0.52%) 3.954s (± 0.23%) 🟩-0.167s (- 4.05%) 3.934s 3.975s p=0.000 n=12
strada-build-src - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,388,680 1,361,755 -26,925 (- 1.94%) ~ ~ p=0.000 n=12
Types 441,862 411,422 🟩-30,440 (- 6.89%) ~ ~ p=0.000 n=12
Memory Used 1,316,453k (± 0.67%) 1,297,654k (± 0.48%) -18,800k (- 1.43%) 1,289,455k 1,326,687k p=0.000 n=12
Memory Allocs 78,189,581 (± 0.14%) 77,433,055 (± 0.04%) -756,526 (- 0.97%) 77,384,702 77,559,131 p=0.000 n=12
Config Time 0.005s (±13.27%) 0.005s (±10.49%) ~ 0.004s 0.006s p=1.000 n=12
Parse Time 0.311s (± 2.24%) 0.312s (± 2.57%) ~ 0.284s 0.328s p=0.629 n=12
Bind Time 0.000s 0.001s (±217.90%) ~ 0.000s 0.006s p=1.000 n=12
Check Time 1.999s (± 0.55%) 1.949s (± 0.58%) -0.050s (- 2.51%) 1.924s 1.992s p=0.000 n=12
Emit Time 0.367s (± 4.38%) 0.377s (± 5.34%) ~ 0.334s 0.442s p=0.340 n=12
Total Time 23.946s (± 0.44%) 23.104s (± 0.75%) 🟩-0.842s (- 3.52%) 22.538s 23.401s p=0.000 n=12
strada-compiler - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 334,953 311,043 🟩-23,910 (- 7.14%) ~ ~ p=0.000 n=12
Types 197,616 169,251 🟩-28,365 (-14.35%) ~ ~ p=0.000 n=12
Memory Used 247,549k (± 0.02%) 235,052k (± 0.02%) 🟩-12,497k (- 5.05%) 234,948k 235,206k p=0.000 n=12
Memory Allocs 4,329,730 (± 0.01%) 4,243,810 (± 0.01%) -85,919 (- 1.98%) 4,242,479 4,244,529 p=0.000 n=12
Config Time 0.001s 0.001s ~ ~ ~ p=1.000 n=12
Parse Time 0.111s (± 3.49%) 0.110s (± 3.67%) ~ 0.102s 0.122s p=0.809 n=12
Bind Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Check Time 0.964s (± 0.71%) 0.850s (± 0.74%) 🟩-0.114s (-11.85%) 0.834s 0.860s p=0.000 n=12
Emit Time 0.149s (± 4.69%) 0.159s (± 2.86%) 🔻+0.010s (+ 6.93%) 0.150s 0.171s p=0.043 n=12
Total Time 1.277s (± 0.54%) 1.170s (± 0.57%) 🟩-0.108s (- 8.42%) 1.159s 1.190s p=0.000 n=12
t3code-server - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 11,588,087 11,712,751 +124,664 (+ 1.08%) ~ ~ p=0.000 n=12
Types 3,062,556 3,111,245 +48,689 (+ 1.59%) ~ ~ p=0.000 n=12
Memory Used 3,946,503k (± 0.02%) 4,000,680k (± 0.03%) +54,177k (+ 1.37%) 3,998,673k 4,005,151k p=0.000 n=12
Memory Allocs 41,538,460 (± 0.03%) 42,034,673 (± 0.08%) +496,213 (+ 1.19%) 41,973,491 42,186,642 p=0.000 n=12
Config Time 0.005s 0.005s ~ ~ ~ p=1.000 n=12
Parse Time 0.441s (± 2.44%) 0.460s (± 2.23%) 🔻+0.019s (+ 4.32%) 0.432s 0.489s p=0.034 n=12
Bind Time 0.151s (±14.99%) 0.125s (± 5.79%) ~ 0.115s 0.157s p=0.075 n=12
Check Time 15.529s (± 0.45%) 14.892s (± 0.10%) 🟩-0.637s (- 4.10%) 14.857s 14.948s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 16.142s (± 0.42%) 15.504s (± 0.11%) 🟩-0.639s (- 3.96%) 15.462s 15.548s p=0.000 n=12
ts-pre-modules - native
Errors 87 87 ~ ~ ~ p=1.000 n=12
Symbols 303,834 280,430 🟩-23,404 (- 7.70%) ~ ~ p=0.000 n=12
Types 181,663 156,295 🟩-25,368 (-13.96%) ~ ~ p=0.000 n=12
Memory Used 215,248k (± 0.02%) 204,264k (± 0.03%) 🟩-10,984k (- 5.10%) 204,108k 204,383k p=0.000 n=12
Memory Allocs 1,401,773 (± 0.02%) 1,336,626 (± 0.02%) 🟩-65,147 (- 4.65%) 1,336,024 1,337,634 p=0.000 n=12
Config Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Parse Time 0.077s (± 3.34%) 0.078s (± 4.91%) ~ 0.069s 0.089s p=0.679 n=12
Bind Time 0.034s (± 8.02%) 0.031s (± 8.48%) ~ 0.025s 0.036s p=0.087 n=12
Check Time 0.619s (± 1.00%) 0.552s (± 0.48%) 🟩-0.067s (-10.80%) 0.545s 0.558s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 0.741s (± 0.68%) 0.672s (± 0.77%) 🟩-0.069s (- 9.36%) 0.657s 0.685s p=0.000 n=12
vscode - native
Errors 396 396 ~ ~ ~ p=1.000 n=12
Symbols 12,521,033 12,327,234 -193,799 (- 1.55%) ~ ~ p=0.000 n=12
Types 4,102,183 4,019,502 -82,681 (- 2.02%) ~ ~ p=0.000 n=12
Memory Used 5,875,826k (± 0.02%) 5,802,384k (± 0.02%) -73,441k (- 1.25%) 5,800,796k 5,808,180k p=0.000 n=12
Memory Allocs 59,736,580 (± 0.04%) 59,380,924 (± 0.01%) -355,656 (- 0.60%) 59,371,369 59,407,452 p=0.000 n=12
Config Time 0.076s (± 0.37%) 0.077s (± 0.51%) ~ 0.076s 0.078s p=0.076 n=12
Parse Time 1.857s (± 3.15%) 1.865s (± 2.45%) ~ 1.755s 1.959s p=0.671 n=12
Bind Time 0.687s (±13.17%) 0.613s (±14.22%) ~ 0.513s 0.891s p=0.143 n=12
Check Time 15.049s (± 1.35%) 15.366s (± 1.18%) +0.317s (+ 2.11%) 14.914s 15.617s p=0.012 n=12
Emit Time 5.435s (± 5.89%) 5.549s (± 6.97%) ~ 4.795s 6.714s p=0.920 n=12
Total Time 23.194s (± 0.94%) 23.626s (± 1.17%) +0.432s (+ 1.86%) 22.907s 24.481s p=0.010 n=12
webpack - native
Errors 598 598 ~ ~ ~ p=1.000 n=12
Symbols 1,343,495 1,340,533 -2,962 (- 0.22%) ~ ~ p=0.000 n=12
Types 651,491 647,981 -3,510 (- 0.54%) ~ ~ p=0.000 n=12
Memory Used 855,318k (± 0.01%) 853,859k (± 0.02%) -1,460k (- 0.17%) 853,502k 854,433k p=0.000 n=12
Memory Allocs 6,769,398 (± 0.02%) 6,761,042 (± 0.02%) -8,356 (- 0.12%) 6,759,217 6,766,998 p=0.000 n=12
Config Time 0.008s (± 4.38%) 0.008s 🔻+0.000s (+ 6.67%) ~ ~ p=0.014 n=12
Parse Time 0.246s (± 3.00%) 0.246s (± 1.89%) ~ 0.231s 0.259s p=0.620 n=12
Bind Time 0.071s (±19.65%) 0.057s (± 4.52%) ~ 0.051s 0.066s p=0.322 n=12
Check Time 2.304s (± 0.71%) 2.325s (± 0.41%) +0.021s (+ 0.91%) 2.287s 2.350s p=0.015 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 2.660s (± 0.45%) 2.672s (± 0.37%) ~ 2.643s 2.695s p=0.139 n=12
xstate-main - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,060,810 1,083,654 +22,844 (+ 2.15%) ~ ~ p=0.000 n=12
Types 389,671 397,590 +7,919 (+ 2.03%) ~ ~ p=0.000 n=12
Memory Used 471,876k (± 0.03%) 479,645k (± 0.01%) +7,768k (+ 1.65%) 479,436k 479,831k p=0.000 n=12
Memory Allocs 4,223,342 (± 0.10%) 4,286,072 (± 0.08%) +62,730 (+ 1.49%) 4,278,263 4,296,880 p=0.000 n=12
Config Time 0.003s (± 6.23%) 0.003s ~ ~ ~ p=1.000 n=12
Parse Time 0.136s (± 2.78%) 0.133s (± 3.49%) ~ 0.121s 0.143s p=0.223 n=12
Bind Time 0.042s (±16.29%) 0.043s (±16.14%) ~ 0.034s 0.065s p=0.697 n=12
Check Time 1.096s (± 0.93%) 0.989s (± 0.70%) 🟩-0.107s (- 9.80%) 0.973s 1.011s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 1.284s (± 0.91%) 1.176s (± 0.91%) 🟩-0.108s (- 8.42%) 1.151s 1.199s p=0.000 n=12
System info unknown
Hosts
  • native
Scenarios
  • Compiler-Unions - native
  • angular-1 - native
  • arktype - native
  • mui-docs - native
  • sentry - native
  • social-app - native
  • strada-build-src - native
  • strada-compiler - native
  • t3code-server - native
  • ts-pre-modules - native
  • vscode - native
  • webpack - native
  • xstate-main - native
Benchmark Name Iterations
Current pr 12
Baseline baseline 12

lsp

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
CompilerLSP - native
Req 1 - updateOpen 11ms (±11.74%) 13ms (±13.49%) 🔻+2ms (+22.48%) 10ms 16ms p=0.031 n=12
Req 2 - geterr 284ms (± 0.60%) 285ms (± 0.62%) ~ 280ms 290ms p=0.430 n=12
Req 3 - references 48ms (± 1.09%) 50ms (± 2.37%) +2ms (+ 3.11%) 47ms 54ms p=0.013 n=12
Req 4 - navto 18ms (± 4.84%) 18ms (± 6.43%) ~ 17ms 22ms p=0.780 n=12
Req 5 - completionInfo count 1,525 1,525 ~ ~ ~ p=1.000 n=12
Req 5 - completionInfo 20ms (±10.11%) 18ms (±10.88%) ~ 15ms 23ms p=0.573 n=12
Compiler-UnionsLSP - native
Req 1 - updateOpen 12ms (±12.87%) 12ms (±14.84%) ~ 9ms 17ms p=0.999 n=12
Req 2 - geterr 538ms (± 0.40%) 535ms (± 0.47%) ~ 531ms 543ms p=0.055 n=12
Req 3 - references 46ms (± 4.61%) 47ms (± 4.35%) ~ 43ms 50ms p=0.505 n=12
Req 4 - navto 16ms (± 1.90%) 16ms (± 2.56%) ~ 15ms 17ms p=0.680 n=12
Req 5 - completionInfo count 1,363 1,363 ~ ~ ~ p=1.000 n=12
Req 5 - completionInfo 16ms (± 5.18%) 16ms (± 8.79%) ~ 14ms 22ms p=0.319 n=12
System info unknown
Hosts
  • native
Scenarios
  • CompilerLSP - native
  • Compiler-UnionsLSP - native
  • xstate-main-1-LSP - native
Benchmark Name Iterations
Current pr 12
Baseline baseline 12

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
lsp-startup - native
Execution time 4.67ms (±365.09%) 4.67ms (±365.08%) +0.00ms (+ 0.10%) 0.00ms 470.33ms p=0.000 n=1200
tsgo-startup - native
Execution time 5.74ms (±365.10%) 5.76ms (±365.09%) +0.02ms (+ 0.31%) 0.00ms 581.64ms p=0.000 n=1200
System info unknown
Hosts
  • native
Scenarios
  • lsp-startup - native
  • tsgo-startup - native
Benchmark Name Iterations
Current pr 12
Baseline baseline 12

Developer Information:

Download Benchmarks

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug

Projects

Status: Not started

Development

Successfully merging this pull request may close these issues.

2 participants