[profiler] Display global pipeline stats in profiler#6518
Conversation
| sources?: string[] | ||
| logText?: string | ||
| pipelineName?: string | ||
| /** Cumulative pipeline-wide metrics from `stats.json`, when the bundle includes it. */ |
| logText = decoder.decode(await logsFile.read()) | ||
| } | ||
|
|
||
| // `stats.json` is the `/stats` response; the overview tile only needs its `global_metrics`. |
There was a problem hiding this comment.
i wonder why we have so many ways to report stuff
maybe the profile should include the stats
| // | ||
| // The overview tile reports a profile of the run, so only the cumulative readings are | ||
| // meaningful here — a momentary RSS or storage figure says nothing about the profile as a | ||
| // whole. We therefore select from an explicit allowlist; the current gauges are left out by |
There was a problem hiding this comment.
I would strive for shorter comments in general, these take a very long time to read, and have a lot of redundant information. Just say "we use an allow list"
|
|
||
| const millisecondsToTime: ToValue = (ms) => new TimeValue(ms / 1000) | ||
|
|
||
| /** The cumulative metrics to display, in presentation order. Each descriptor names a |
There was a problem hiding this comment.
again a redundant comment. "Metrics to display" is enough.
mythical-fred
left a comment
There was a problem hiding this comment.
Draft, so high-level only.
Direction is right and the slice through the code is well-judged. The allowlist-vs-denylist call for which global_metrics fields to display — and the explanatory comment at the top of globalMetrics.ts — is exactly the kind of decision I want to see captured in code: "current gauges are left out by omission rather than by a fragile deny-list that new fields could slip past." That sentence alone will save someone a confused hour in two years.
Architecturally, two questions worth resolving before this goes ready-for-review:
Why is GlobalMetrics redeclared instead of imported from a generated type? stats.json is the /stats HTTP response, which is ControllerStatus, which has a Rust-side definition that the OpenAPI generator emits as a TypeScript type elsewhere in the repo (see how web-console consumes pipeline configuration types). Hand-writing a parallel interface here means the next time someone renames a metric in the controller — or adds one to the cumulative set — this file silently goes out of sync and the overview tile loses a row that the bundle actually carries. The DESCRIPTORS list is presentation logic and absolutely belongs in this file; the shape of GlobalMetrics does not. Either import from the generated openapi types, or — if that import would pull in too much for the profiler app — at minimum add a doc comment pointing at crates/feldera-types/src/pipeline_manager/controller_status.rs (or wherever global_metrics actually lives) so the next reader knows which side is authoritative.
What's the failure mode when stats.json is malformed? processZipBundle.ts catches the parse error and sets globalMetrics = undefined, which means the tile silently disappears. For a profile viewer that's the right tradeoff (the bundle is still useful), but a console.warn with the filename and the parse error would make it possible to debug "why is my tile gone?" without diffing two bundles. Same for stats.global_metrics being absent — currently silently undefined; a console.warn("stats.json carried no global_metrics object") would catch the case where a backwards-incompatible rename of the field slips through. Cheap, defensive, fixes a real "wait, where did the tile go" moment.
Smaller things, save for full review:
KeyValueBlock.svelteis a 28-line generic key/value tile. Good. Worth confirming whether it's intended for reuse anywhere else (the pathmetrics/blocks/suggests yes, alongsideMetricsDistributionBlock) — if no, the comment "Unlike the distribution block it has no per-worker bars" is accurate forever; if yes, the props need aclasspassthrough so callers can style spacing without forking the component.DESCRIPTORSmixesCountValue.fromNumberandBytesValue.fromNumberasToValue. The cast pattern works but it would be more honest to makeToValuea discriminated union ({kind: 'count'}|{kind: 'bytes'}|{kind: 'time'}) and have the rendering pick the formatter from the kind. That way unit-vs-value mismatches are caught by the type system, not byfromNumber's permissive signature.millisecondsToTime: ToValue = (ms) => new TimeValue(ms / 1000)— the unit assumption ("the field is named*_msecstherefore the value is milliseconds") is fine, butcpu_msecsandruntime_elapsed_msecsanduptime_msecsall funnel through the same converter via a single constant. If one day a metric arrives in microseconds, the constant name (millisecondsToTime) will lie. Either name the metric kind in the descriptor ({ unit: 'ms' }) or accept aTimeValue.fromMilliseconds(ms)factory on the lib side.globalMetrics.test.ts— 72 lines is a nice unit-test footprint and the cases look right (presence, absence, non-finite). Worth one more case: a metric that's present butnull(since the openapi-generated type often usesT | null); currentlytypeof null === 'object'so the guard correctly drops it, but a regression test pins that behavior.MetricsView.sveltederivesglobalMetricEntriesonly for theoverviewmode. Good. Worth a$inspector a small comment that this empty-on-non-overview behavior is what hides the tile elsewhere, rather than relying on the{#if entries.length > 0}check downstream.<dl class="grid grid-cols-[1fr_auto] ...">— semantically correct for a description list. Verify the screen-reader experience (the<dt>truncation withtruncateis likely to cut off long labels for AT users); atitleattribute mirroringentry.labelon the<dt>would let hover/AT users see the full text.
Not blocking anything since it's a draft. The "import the generated type instead of redeclaring it" question is the one I'd want answered before ready-for-review.
d3f0d38 to
514ab45
Compare
Signed-off-by: Karakatiza666 <bulakh.96@gmail.com>
Signed-off-by: Karakatiza666 <bulakh.96@gmail.com>
514ab45 to
315597c
Compare
Fix #6492: Profiler should show pipeline config
Testing: manual, added unit tests for extracting global metrics and pipeline config from the profile
When the config not present in the bundle:
