Skip to content

metricdatatest: Improve printing of diffs#8073

Merged
dashpole merged 6 commits intoopen-telemetry:mainfrom
dashpole:better_errors
Mar 30, 2026
Merged

metricdatatest: Improve printing of diffs#8073
dashpole merged 6 commits intoopen-telemetry:mainfrom
dashpole:better_errors

Conversation

@dashpole
Copy link
Copy Markdown
Contributor

I've been getting tired of trying to parse blobs of OTLP-like text when debugging metrics SDK test failures. This PR updating the printing of diffs between metrics.

Changes

This combines the compareDiff, and diffSlices functions into a single diffSlices function. The compare parameter now returns a []string instead of a bool, to allow us to get a list of specific differences between elements. diffSlices now also accepts a formatContext parameter that provides the context (e.g. which scope, which metric, etc) to print before printing the difference itself.

I've chosen to use the scope name, metric name, and attributes as the context for scope, metric, and datapoint respectively. We could print out more (as those are not the only identifying fields), but that is usually enough to point to the problem.

Example

I "broke" one of the OpenCensus tests by changing the expected SpanID from 2 -> 1. You can see the resulting error message before and after below:

Before:

--- FAIL: TestConvertMetrics (0.00s)
    --- FAIL: TestConvertMetrics/normal_Histogram,_summary,_gauges,_and_sums (0.00s)
        metric_test.go:940: [ScopeMetrics Metrics not equal:
            missing expected values:
            metricdata.Metrics{Name:"foo.com/histogram-a", Description:"a testing histogram", Unit:"1", 
Data:metricdata.Histogram[float64]{DataPoints:[]metricdata.HistogramDataPoint[float64]
{metricdata.HistogramDataPoint[float64]{Attributes:attribute.Set{hash:0x83dbf482c8017da3, data:
[2]attribute.KeyValue{attribute.KeyValue{Key:"a", Value:attribute.Value{vtype:4, numeric:0x0, stringly:"hello", 
slice:interface {}(nil)}}, attribute.KeyValue{Key:"b", Value:attribute.Value{vtype:4, numeric:0x0, stringly:"world", 
slice:interface {}(nil)}}}}, StartTime:time.Date(2026, time.March, 18, 15, 46, 53, 76753334, time.Local), 
Time:time.Date(2026, time.March, 18, 15, 47, 53, 77753334, time.Local), Count:0x8, Bounds:[]float64{1, 2, 3}, 
BucketCounts:[]uint64{0x1, 0x2, 0x5}, Min:metricdata.Extrema[float64]{value:0, valid:false}, 
Max:metricdata.Extrema[float64]{value:0, valid:false}, Sum:100, Exemplars:[]metricdata.Exemplar[float64]
{metricdata.Exemplar[float64]{FilteredAttributes:[]attribute.KeyValue{attribute.KeyValue{Key:"bool", 
Value:attribute.Value{vtype:1, numeric:0x1, stringly:"", slice:interface {}(nil)}}}, Time:time.Date(2026, time.March, 18, 15, 
47, 43, 77753334, time.Local), Value:0.8, SpanID:[]uint8{0x2, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, TraceID:[]uint8{0x1, 0x0, 
0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}, metricdata.Exemplar[float64]{FilteredAttributes:
[]attribute.KeyValue(nil), Time:time.Date(2026, time.March, 18, 15, 47, 43, 77753334, time.Local), Value:1.5, SpanID:
[]uint8{0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, TraceID:[]uint8{0x3, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 
0x0, 0x0, 0x0, 0x0}}, metricdata.Exemplar[float64]{FilteredAttributes:[]attribute.KeyValue(nil), Time:time.Date(2026, 
time.March, 18, 15, 47, 43, 77753334, time.Local), Value:2.6, SpanID:[]uint8{0x6, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, 
TraceID:[]uint8{0x5, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}}}, 
metricdata.HistogramDataPoint[float64]{Attributes:attribute.Set{hash:0x83dbf482c8017da3, data:
[2]attribute.KeyValue{attribute.KeyValue{Key:"a", Value:attribute.Value{vtype:4, numeric:0x0, stringly:"hello", 
slice:interface {}(nil)}}, attribute.KeyValue{Key:"b", Value:attribute.Value{vtype:4, numeric:0x0, stringly:"world", 
slice:interface {}(nil)}}}}, StartTime:time.Date(2026, time.March, 18, 15, 46, 53, 76753334, time.Local), 
Time:time.Date(2026, time.March, 18, 15, 47, 53, 76753334, time.Local), Count:0xa, Bounds:[]float64{1, 2, 3}, 
BucketCounts:[]uint64{0x1, 0x4, 0x5}, Min:metricdata.Extrema[float64]{value:0, valid:false}, 
Max:metricdata.Extrema[float64]{value:0, valid:false}, Sum:110, Exemplars:[]metricdata.Exemplar[float64]
{metricdata.Exemplar[float64]{FilteredAttributes:[]attribute.KeyValue(nil), Time:time.Date(2026, time.March, 18, 15, 47, 
43, 77753334, time.Local), Value:0.9, SpanID:[]uint8{0x8, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, TraceID:[]uint8{0x7, 0x0, 
0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}, metricdata.Exemplar[float64]{FilteredAttributes:
[]attribute.KeyValue(nil), Time:time.Date(2026, time.March, 18, 15, 47, 43, 77753334, time.Local), Value:1.1, SpanID:
[]uint8{0xa, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, TraceID:[]uint8{0x9, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 
0x0, 0x0, 0x0, 0x0}}, metricdata.Exemplar[float64]{FilteredAttributes:[]attribute.KeyValue(nil), Time:time.Date(2026, 
time.March, 18, 15, 47, 43, 77753334, time.Local), Value:2.7, SpanID:[]uint8{0xc, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, 
TraceID:[]uint8{0xb, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}}}}, Temporality:0x1}}
            unexpected additional values:
            metricdata.Metrics{Name:"foo.com/histogram-a", Description:"a testing histogram", Unit:"1", 
Data:metricdata.Histogram[float64]{DataPoints:[]metricdata.HistogramDataPoint[float64]
{metricdata.HistogramDataPoint[float64]{Attributes:attribute.Set{hash:0x83dbf482c8017da3, data:
[2]attribute.KeyValue{attribute.KeyValue{Key:"a", Value:attribute.Value{vtype:4, numeric:0x0, stringly:"hello", 
slice:interface {}(nil)}}, attribute.KeyValue{Key:"b", Value:attribute.Value{vtype:4, numeric:0x0, stringly:"world", 
slice:interface {}(nil)}}}}, StartTime:time.Date(2026, time.March, 18, 15, 46, 53, 76753334, time.Local), 
Time:time.Date(2026, time.March, 18, 15, 47, 53, 77753334, time.Local), Count:0x8, Bounds:[]float64{1, 2, 3}, 
BucketCounts:[]uint64{0x1, 0x2, 0x5}, Min:metricdata.Extrema[float64]{value:0, valid:false}, 
Max:metricdata.Extrema[float64]{value:0, valid:false}, Sum:100, Exemplars:[]metricdata.Exemplar[float64]
{metricdata.Exemplar[float64]{FilteredAttributes:[]attribute.KeyValue{attribute.KeyValue{Key:"bool", 
Value:attribute.Value{vtype:1, numeric:0x1, stringly:"", slice:interface {}(nil)}}}, Time:time.Date(2026, time.March, 18, 15, 
47, 43, 77753334, time.Local), Value:0.8, SpanID:[]uint8{0x1, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, TraceID:[]uint8{0x1, 0x0, 
0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}, metricdata.Exemplar[float64]{FilteredAttributes:
[]attribute.KeyValue(nil), Time:time.Date(2026, time.March, 18, 15, 47, 43, 77753334, time.Local), Value:1.5, SpanID:
[]uint8{0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, TraceID:[]uint8{0x3, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 
0x0, 0x0, 0x0, 0x0}}, metricdata.Exemplar[float64]{FilteredAttributes:[]attribute.KeyValue(nil), Time:time.Date(2026, 
time.March, 18, 15, 47, 43, 77753334, time.Local), Value:2.6, SpanID:[]uint8{0x6, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, 
TraceID:[]uint8{0x5, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}}}, 
metricdata.HistogramDataPoint[float64]{Attributes:attribute.Set{hash:0x83dbf482c8017da3, data:
[2]attribute.KeyValue{attribute.KeyValue{Key:"a", Value:attribute.Value{vtype:4, numeric:0x0, stringly:"hello", 
slice:interface {}(nil)}}, attribute.KeyValue{Key:"b", Value:attribute.Value{vtype:4, numeric:0x0, stringly:"world", 
slice:interface {}(nil)}}}}, StartTime:time.Date(2026, time.March, 18, 15, 46, 53, 76753334, time.Local), 
Time:time.Date(2026, time.March, 18, 15, 47, 53, 76753334, time.Local), Count:0xa, Bounds:[]float64{1, 2, 3}, 
BucketCounts:[]uint64{0x1, 0x4, 0x5}, Min:metricdata.Extrema[float64]{value:0, valid:false}, 
Max:metricdata.Extrema[float64]{value:0, valid:false}, Sum:110, Exemplars:[]metricdata.Exemplar[float64]
{metricdata.Exemplar[float64]{FilteredAttributes:[]attribute.KeyValue(nil), Time:time.Date(2026, time.March, 18, 15, 47, 
43, 77753334, time.Local), Value:0.9, SpanID:[]uint8{0x8, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, TraceID:[]uint8{0x7, 0x0, 
0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}, metricdata.Exemplar[float64]{FilteredAttributes:
[]attribute.KeyValue(nil), Time:time.Date(2026, time.March, 18, 15, 47, 43, 77753334, time.Local), Value:1.1, SpanID:
[]uint8{0xa, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, TraceID:[]uint8{0x9, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 
0x0, 0x0, 0x0, 0x0}}, metricdata.Exemplar[float64]{FilteredAttributes:[]attribute.KeyValue(nil), Time:time.Date(2026, 
time.March, 18, 15, 47, 43, 77753334, time.Local), Value:2.7, SpanID:[]uint8{0xc, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, 
TraceID:[]uint8{0xb, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}}}}, Temporality:0x1}}
            ]
FAIL
FAIL    go.opentelemetry.io/otel/bridge/opencensus/internal/ocmetric    0.037s

After:

--- FAIL: TestConvertMetrics (0.00s)
    --- FAIL: TestConvertMetrics/normal_Histogram,_summary,_gauges,_and_sums (0.00s)
        metric_test.go:940: [ScopeMetrics Metrics not equal:
            Metric "foo.com/histogram-a":
                Metrics Data not equal:
                Histogram not equal:
                Histogram DataPoints not equal:
                HistogramDataPoint [a=hello,b=world]:
                        Exemplars not equal:
                        Exemplar:
                                SpanID not equal:
                                expected: [2 0 0 0 0 0 0 0]
                                actual: [1 0 0 0 0 0 0 0]
            ]
FAIL
FAIL    go.opentelemetry.io/otel/bridge/opencensus/internal/ocmetric    0.036s

@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 18, 2026

Codecov Report

❌ Patch coverage is 96.55172% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.7%. Comparing base (3ff9ad7) to head (438b362).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
...dk/metric/metricdata/metricdatatest/comparisons.go 96.5% 2 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##            main   #8073     +/-   ##
=======================================
- Coverage   81.7%   81.7%   -0.1%     
=======================================
  Files        308     308             
  Lines      23655   23686     +31     
=======================================
+ Hits       19346   19368     +22     
- Misses      3928    3934      +6     
- Partials     381     384      +3     
Files with missing lines Coverage Δ
...dk/metric/metricdata/metricdatatest/comparisons.go 92.7% <96.5%> (-0.4%) ⬇️

... and 5 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@dashpole dashpole marked this pull request as ready for review March 18, 2026 19:19
@dashpole
Copy link
Copy Markdown
Contributor Author

We don't currently have unit tests for the comparison text. Reviewers can let me know if we want those.

Copy link
Copy Markdown
Member

@dmathieu dmathieu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests would indeed be nice, but this is already a great improvement.

Comment thread sdk/metric/metricdata/metricdatatest/comparisons.go Outdated
Comment thread sdk/metric/metricdata/metricdatatest/comparisons.go Outdated
@dashpole dashpole merged commit ab27913 into open-telemetry:main Mar 30, 2026
34 checks passed
@pellared pellared added this to the v1.43.0 milestone Apr 1, 2026
dmathieu added a commit that referenced this pull request Apr 3, 2026
Release issue:
#8127

## Added

- Add `IsRandom` and `WithRandom` on `TraceFlags`, and `IsRandom` on
`SpanContext` in `go.opentelemetry.io/otel/trace`
for [W3C Trace Context Level 2 Random Trace ID
Flag](https://www.w3.org/TR/trace-context-2/#random-trace-id-flag)
support. (#8012)
- Add service detection with `WithService` in
`go.opentelemetry.io/otel/sdk/resource`. (#7642)
- Add `DefaultWithContext` and `EnvironmentWithContext` in
`go.opentelemetry.io/otel/sdk/resource` to support plumbing
`context.Context` through default and environment detectors. (#8051)
- Support attributes with empty value (`attribute.EMPTY`) in
`go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc`.
(#8038)
- Support attributes with empty value (`attribute.EMPTY`) in
`go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc`.
(#8038)
- Support attributes with empty value (`attribute.EMPTY`) in
`go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc`. (#8038)
- Support attributes with empty value (`attribute.EMPTY`) in
`go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp`.
(#8038)
- Support attributes with empty value (`attribute.EMPTY`) in
`go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp`.
(#8038)
- Support attributes with empty value (`attribute.EMPTY`) in
`go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp`. (#8038)
- Support attributes with empty value (`attribute.EMPTY`) in
`go.opentelemetry.io/otel/sdk/metric/metricdata/metricdatatest`. (#8038)
- Add support for per-series start time tracking for cumulative metrics
in `go.opentelemetry.io/otel/sdk/metric`.
  Set `OTEL_GO_X_PER_SERIES_START_TIMESTAMPS=true` to enable. (#8060)
- Add `WithCardinalityLimitSelector` for metric reader for configuring
cardinality limits specific to the instrument kind. (#7855)

## Changed

- Introduce the `EMPTY` Type in `go.opentelemetry.io/otel/attribute` to
reflect that an empty value is now a valid value, with `INVALID`
remaining as a deprecated alias of `EMPTY`. (#8038)
- Refactor slice handling in `go.opentelemetry.io/otel/attribute` to
optimize short slice values with fixed-size fast paths. (#8039)
- Improve performance of span metric recording in
`go.opentelemetry.io/otel/sdk/trace` by returning early if
self-observability is not enabled. (#8067)
- Improve formatting of metric data diffs in
`go.opentelemetry.io/otel/sdk/metric/metricdata/metricdatatest`. (#8073)

## Deprecated

- Deprecate `INVALID` in `go.opentelemetry.io/otel/attribute`. Use
`EMPTY` instead. (#8038)

## Fixed

- Return spec-compliant `TraceIdRatioBased` description. This is a
breaking behavioral change, but it is necessary to
make the implementation
[spec-compliant](https://opentelemetry.io/docs/specs/otel/trace/sdk/#traceidratiobased).
(#8027)
- Fix a race condition in `go.opentelemetry.io/otel/sdk/metric` where
the lastvalue aggregation could collect the value 0 even when no
zero-value measurements were recorded. (#8056)
- Limit HTTP response body to 4 MiB in
`go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp` to
mitigate excessive memory usage caused by a misconfigured or malicious
server.
Responses exceeding the limit are treated as non-retryable errors.
(#8108)
- Limit HTTP response body to 4 MiB in
`go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp` to
mitigate excessive memory usage caused by a misconfigured or malicious
server.
Responses exceeding the limit are treated as non-retryable errors.
(#8108)
- Limit HTTP response body to 4 MiB in
`go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp` to
mitigate excessive memory usage caused by a misconfigured or malicious
server.
Responses exceeding the limit are treated as non-retryable errors.
(#8108)
- `WithHostID` detector in `go.opentelemetry.io/otel/sdk/resource` to
use full path for `kenv` command on BSD. (#8113)
- Fix missing `request.GetBody` in
`go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp` to
correctly handle HTTP2 GOAWAY frame. (#8096)

---------

Co-authored-by: Tyler Yahn <MrAlias@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants