Skip to content

[profiler] Display global pipeline stats in profiler#6518

Merged
Karakatiza666 merged 2 commits into
mainfrom
extra-profile-stats
Jul 10, 2026
Merged

[profiler] Display global pipeline stats in profiler#6518
Karakatiza666 merged 2 commits into
mainfrom
extra-profile-stats

Conversation

@Karakatiza666

@Karakatiza666 Karakatiza666 commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Fix #6492: Profiler should show pipeline config

Testing: manual, added unit tests for extracting global metrics and pipeline config from the profile

image image

When the config not present in the bundle:
image

sources?: string[]
logText?: string
pipelineName?: string
/** Cumulative pipeline-wide metrics from `stats.json`, when the bundle includes it. */

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it -> them

logText = decoder.decode(await logsFile.read())
}

// `stats.json` is the `/stats` response; the overview tile only needs its `global_metrics`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i wonder why we have so many ways to report stuff
maybe the profile should include the stats

//
// The overview tile reports a profile of the run, so only the cumulative readings are
// meaningful here — a momentary RSS or storage figure says nothing about the profile as a
// whole. We therefore select from an explicit allowlist; the current gauges are left out by

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would strive for shorter comments in general, these take a very long time to read, and have a lot of redundant information. Just say "we use an allow list"


const millisecondsToTime: ToValue = (ms) => new TimeValue(ms / 1000)

/** The cumulative metrics to display, in presentation order. Each descriptor names a

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

again a redundant comment. "Metrics to display" is enough.

@mythical-fred mythical-fred left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Draft, so high-level only.

Direction is right and the slice through the code is well-judged. The allowlist-vs-denylist call for which global_metrics fields to display — and the explanatory comment at the top of globalMetrics.ts — is exactly the kind of decision I want to see captured in code: "current gauges are left out by omission rather than by a fragile deny-list that new fields could slip past." That sentence alone will save someone a confused hour in two years.

Architecturally, two questions worth resolving before this goes ready-for-review:

Why is GlobalMetrics redeclared instead of imported from a generated type? stats.json is the /stats HTTP response, which is ControllerStatus, which has a Rust-side definition that the OpenAPI generator emits as a TypeScript type elsewhere in the repo (see how web-console consumes pipeline configuration types). Hand-writing a parallel interface here means the next time someone renames a metric in the controller — or adds one to the cumulative set — this file silently goes out of sync and the overview tile loses a row that the bundle actually carries. The DESCRIPTORS list is presentation logic and absolutely belongs in this file; the shape of GlobalMetrics does not. Either import from the generated openapi types, or — if that import would pull in too much for the profiler app — at minimum add a doc comment pointing at crates/feldera-types/src/pipeline_manager/controller_status.rs (or wherever global_metrics actually lives) so the next reader knows which side is authoritative.

What's the failure mode when stats.json is malformed? processZipBundle.ts catches the parse error and sets globalMetrics = undefined, which means the tile silently disappears. For a profile viewer that's the right tradeoff (the bundle is still useful), but a console.warn with the filename and the parse error would make it possible to debug "why is my tile gone?" without diffing two bundles. Same for stats.global_metrics being absent — currently silently undefined; a console.warn("stats.json carried no global_metrics object") would catch the case where a backwards-incompatible rename of the field slips through. Cheap, defensive, fixes a real "wait, where did the tile go" moment.

Smaller things, save for full review:

  • KeyValueBlock.svelte is a 28-line generic key/value tile. Good. Worth confirming whether it's intended for reuse anywhere else (the path metrics/blocks/ suggests yes, alongside MetricsDistributionBlock) — if no, the comment "Unlike the distribution block it has no per-worker bars" is accurate forever; if yes, the props need a class passthrough so callers can style spacing without forking the component.
  • DESCRIPTORS mixes CountValue.fromNumber and BytesValue.fromNumber as ToValue. The cast pattern works but it would be more honest to make ToValue a discriminated union ({kind: 'count'} | {kind: 'bytes'} | {kind: 'time'}) and have the rendering pick the formatter from the kind. That way unit-vs-value mismatches are caught by the type system, not by fromNumber's permissive signature.
  • millisecondsToTime: ToValue = (ms) => new TimeValue(ms / 1000) — the unit assumption ("the field is named *_msecs therefore the value is milliseconds") is fine, but cpu_msecs and runtime_elapsed_msecs and uptime_msecs all funnel through the same converter via a single constant. If one day a metric arrives in microseconds, the constant name (millisecondsToTime) will lie. Either name the metric kind in the descriptor ({ unit: 'ms' }) or accept a TimeValue.fromMilliseconds(ms) factory on the lib side.
  • globalMetrics.test.ts — 72 lines is a nice unit-test footprint and the cases look right (presence, absence, non-finite). Worth one more case: a metric that's present but null (since the openapi-generated type often uses T | null); currently typeof null === 'object' so the guard correctly drops it, but a regression test pins that behavior.
  • MetricsView.svelte derives globalMetricEntries only for the overview mode. Good. Worth a $inspect or a small comment that this empty-on-non-overview behavior is what hides the tile elsewhere, rather than relying on the {#if entries.length > 0} check downstream.
  • <dl class="grid grid-cols-[1fr_auto] ..."> — semantically correct for a description list. Verify the screen-reader experience (the <dt> truncation with truncate is likely to cut off long labels for AT users); a title attribute mirroring entry.label on the <dt> would let hover/AT users see the full text.

Not blocking anything since it's a draft. The "import the generated type instead of redeclaring it" question is the one I'd want answered before ready-for-review.

Signed-off-by: Karakatiza666 <bulakh.96@gmail.com>
Signed-off-by: Karakatiza666 <bulakh.96@gmail.com>
@Karakatiza666
Karakatiza666 marked this pull request as ready for review July 10, 2026 00:17
@Karakatiza666
Karakatiza666 enabled auto-merge July 10, 2026 00:17
@Karakatiza666
Karakatiza666 added this pull request to the merge queue Jul 10, 2026

@mythical-fred mythical-fred left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 10, 2026
@Karakatiza666
Karakatiza666 added this pull request to the merge queue Jul 10, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 10, 2026
@Karakatiza666
Karakatiza666 added this pull request to the merge queue Jul 10, 2026
Merged via the queue into main with commit b0c1180 Jul 10, 2026
1 check passed
@Karakatiza666
Karakatiza666 deleted the extra-profile-stats branch July 10, 2026 06:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[profiler] Profiler should show pipeline config

3 participants