Skip to content

Extend bazel test scope - #44455

Closed
rdesgroppes wants to merge 1 commit into
mainfrom
regis.desgroppes/extend-bazel-test-scope
Closed

rdesgroppes wants to merge 1 commit into
mainfrom
regis.desgroppes/extend-bazel-test-scope

Conversation

@rdesgroppes

@rdesgroppes rdesgroppes commented Dec 19, 2025 •

Copy link
Copy Markdown
Collaborator

What does this PR do?

  • extend bazel test scope in CI from //bazel/tests/... to //...,
  • fix remaining failure after doing so.

Motivation

  1. the deps/acl/acl.MODULE.bazel file was out of sync with its generated version (using url instead of urls), but this inconsistency was not caught: bazel test //bazel/tests/... was missing consistency tests in //deps/** (addressed by Fix bazel test //deps/acl:module_consistency_test #44750 since),
  2. after discovery: Import sd-agent #44881, was merged, //pkg/discovery/module/rust:dd_discovery_test panicked on assertion failed: result.is_some() (addressed by discovery: Add missing files from sd-agent import #45140 since),
  3. more to come...

Describe how you validated your changes

CI.

Additional Notes

We should make sure bazel test //... always works, whatever the OS our colleagues are on.

@rdesgroppes
rdesgroppes requested a review from a team as a code owner December 19, 2025 11:06
@rdesgroppes rdesgroppes added changelog/no-changelog No changelog entry needed qa/no-code-change No code change in Agent code requiring validation labels Dec 19, 2025
@github-actions github-actions Bot added medium review PR review might take time team/agent-build labels Dec 19, 2025
@rdesgroppes
rdesgroppes force-pushed the regis.desgroppes/extend-bazel-test-scope branch 3 times, most recently from 41faa56 to c408cf2 Compare December 19, 2025 11:45
@agent-platform-auto-pr

agent-platform-auto-pr Bot commented Dec 19, 2025 •

Copy link
Copy Markdown
Contributor

Static quality checks

✅ Please find below the results from static quality gates
Comparison made with ancestor 45cdc09
📊 Static Quality Gates Dashboard

31 successful checks with minimal change (< 2 KiB)
Quality gate Current Size
✅ agent_deb_amd64 746.980 MiB
✅ agent_deb_amd64_fips 707.009 MiB
✅ agent_heroku_amd64 324.136 MiB
✅ agent_msi 658.952 MiB
✅ agent_rpm_amd64 746.964 MiB
✅ agent_rpm_amd64_fips 706.992 MiB
✅ agent_rpm_arm64 726.148 MiB
✅ agent_rpm_arm64_fips 688.661 MiB
✅ agent_suse_amd64 746.964 MiB
✅ agent_suse_amd64_fips 706.992 MiB
✅ agent_suse_arm64 726.148 MiB
✅ agent_suse_arm64_fips 688.661 MiB
✅ docker_agent_amd64 809.452 MiB
✅ docker_agent_arm64 813.234 MiB
✅ docker_agent_jmx_amd64 1000.363 MiB
✅ docker_agent_jmx_arm64 992.832 MiB
✅ docker_cluster_agent_amd64 180.867 MiB
✅ docker_cluster_agent_arm64 196.732 MiB
✅ docker_cws_instrumentation_amd64 7.135 MiB
✅ docker_cws_instrumentation_arm64 6.689 MiB
✅ docker_dogstatsd_amd64 38.449 MiB
✅ docker_dogstatsd_arm64 36.812 MiB
✅ dogstatsd_deb_amd64 29.669 MiB
✅ dogstatsd_deb_arm64 27.833 MiB
✅ dogstatsd_rpm_amd64 29.669 MiB
✅ dogstatsd_suse_amd64 29.669 MiB
✅ iot_agent_deb_amd64 42.766 MiB
✅ iot_agent_deb_arm64 39.884 MiB
✅ iot_agent_deb_armhf 40.454 MiB
✅ iot_agent_rpm_amd64 42.767 MiB
✅ iot_agent_suse_amd64 42.767 MiB
On-wire sizes (compressed)
Quality gate Change Size (prev → curr → max)
✅ agent_deb_amd64 -8.7 KiB (0.00% reduction) 182.663 → 182.655 → 184.810
✅ agent_deb_amd64_fips -5.58 KiB (0.00% reduction) 174.144 → 174.139 → 177.560
✅ agent_heroku_amd64 neutral 87.173 MiB → 88.450
✅ agent_msi -44.0 KiB (0.03% reduction) 142.492 → 142.449 → 143.300
✅ agent_rpm_amd64 neutral 185.686 MiB → 188.160
✅ agent_rpm_amd64_fips +14.57 KiB (0.01% increase) 176.461 → 176.475 → 178.900
✅ agent_rpm_arm64 +27.13 KiB (0.02% increase) 168.259 → 168.286 → 169.930
✅ agent_rpm_arm64_fips +19.25 KiB (0.01% increase) 160.612 → 160.631 → 163.120
✅ agent_suse_amd64 neutral 185.686 MiB → 188.160
✅ agent_suse_amd64_fips +14.57 KiB (0.01% increase) 176.461 → 176.475 → 178.900
✅ agent_suse_arm64 +27.13 KiB (0.02% increase) 168.259 → 168.286 → 169.930
✅ agent_suse_arm64_fips +19.25 KiB (0.01% increase) 160.612 → 160.631 → 163.120
✅ docker_agent_amd64 +3.6 KiB (0.00% increase) 274.752 → 274.756 → 277.400
✅ docker_agent_arm64 -3.01 KiB (0.00% reduction) 262.311 → 262.308 → 266.040
✅ docker_agent_jmx_amd64 neutral 343.404 MiB → 346.020
✅ docker_agent_jmx_arm64 +3.09 KiB (0.00% increase) 326.918 → 326.921 → 330.660
✅ docker_cluster_agent_amd64 neutral 63.887 MiB → 64.510
✅ docker_cluster_agent_arm64 neutral 60.151 MiB → 61.170
✅ docker_cws_instrumentation_amd64 neutral 2.994 MiB → 3.330
✅ docker_cws_instrumentation_arm64 neutral 2.726 MiB → 3.090
✅ docker_dogstatsd_amd64 neutral 14.877 MiB → 15.820
✅ docker_dogstatsd_arm64 neutral 14.219 MiB → 14.830
✅ dogstatsd_deb_amd64 neutral 7.839 MiB → 8.790
✅ dogstatsd_deb_arm64 neutral 6.726 MiB → 7.710
✅ dogstatsd_rpm_amd64 neutral 7.850 MiB → 8.800
✅ dogstatsd_suse_amd64 neutral 7.850 MiB → 8.800
✅ iot_agent_deb_amd64 neutral 11.217 MiB → 12.040
✅ iot_agent_deb_arm64 neutral 9.589 MiB → 10.450
✅ iot_agent_deb_armhf neutral 9.784 MiB → 10.620
✅ iot_agent_rpm_amd64 -2.56 KiB (0.02% reduction) 11.236 → 11.234 → 12.060
✅ iot_agent_suse_amd64 -2.56 KiB (0.02% reduction) 11.236 → 11.234 → 12.060

@rdesgroppes
rdesgroppes force-pushed the regis.desgroppes/extend-bazel-test-scope branch from c408cf2 to 3c73c00 Compare December 19, 2025 12:03
@cit-pr-commenter

cit-pr-commenter Bot commented Dec 19, 2025 •

Copy link
Copy Markdown

Regression Detector

Regression Detector Results

Metrics dashboard
Target profiles
Run ID: cb1f1b22-aa23-4e86-bd5d-19a753a24058

Baseline: 500dcf3
Comparison: 4bbe0a0ebdcd0b196005355e8d9d55f37a689f40
Diff

Optimization Goals: ✅ No significant changes detected

Experiments ignored for regressions

Regressions in experiments with settings containing erratic: true are ignored.

perf experiment goal Δ mean % Δ mean % CI trials links
❌ docker_containers_cpu % cpu utilization +5.13 [+2.12, +8.14] 1 Logs

Fine details of change detection per experiment

perf experiment goal Δ mean % Δ mean % CI trials links
❌ docker_containers_cpu % cpu utilization +5.13 [+2.12, +8.14] 1 Logs
➖ quality_gate_logs % cpu utilization +0.84 [-0.67, +2.34] 1 Logs bounds checks dashboard
➖ quality_gate_metrics_logs memory utilization +0.76 [+0.53, +0.99] 1 Logs bounds checks dashboard
➖ otlp_ingest_logs memory utilization +0.38 [+0.28, +0.49] 1 Logs
➖ uds_dogstatsd_20mb_12k_contexts_20_senders memory utilization +0.31 [+0.25, +0.36] 1 Logs
➖ ddot_logs memory utilization +0.09 [+0.03, +0.14] 1 Logs
➖ quality_gate_idle_all_features memory utilization +0.06 [+0.02, +0.09] 1 Logs bounds checks dashboard
➖ file_to_blackhole_1000ms_latency egress throughput +0.03 [-0.38, +0.43] 1 Logs
➖ uds_dogstatsd_to_api ingress throughput +0.01 [-0.13, +0.14] 1 Logs
➖ file_to_blackhole_0ms_latency egress throughput +0.01 [-0.50, +0.51] 1 Logs
➖ tcp_dd_logs_filter_exclude ingress throughput +0.00 [-0.10, +0.10] 1 Logs
➖ ddot_metrics_sum_delta memory utilization -0.00 [-0.21, +0.21] 1 Logs
➖ quality_gate_idle memory utilization -0.01 [-0.06, +0.03] 1 Logs bounds checks dashboard
➖ uds_dogstatsd_to_api_v3 ingress throughput -0.02 [-0.14, +0.11] 1 Logs
➖ file_to_blackhole_500ms_latency egress throughput -0.02 [-0.40, +0.37] 1 Logs
➖ file_to_blackhole_100ms_latency egress throughput -0.02 [-0.07, +0.02] 1 Logs
➖ ddot_metrics_sum_cumulativetodelta_exporter memory utilization -0.02 [-0.26, +0.21] 1 Logs
➖ ddot_metrics memory utilization -0.12 [-0.34, +0.11] 1 Logs
➖ ddot_metrics_sum_cumulative memory utilization -0.26 [-0.43, -0.10] 1 Logs
➖ file_tree memory utilization -0.37 [-0.42, -0.31] 1 Logs
➖ otlp_ingest_metrics memory utilization -0.37 [-0.52, -0.21] 1 Logs
➖ docker_containers_memory memory utilization -0.37 [-0.45, -0.30] 1 Logs
➖ tcp_syslog_to_blackhole ingress throughput -2.18 [-2.25, -2.10] 1 Logs

Bounds Checks: ✅ Passed

perf experiment bounds_check_name replicates_passed links
✅ docker_containers_cpu simple_check_run 10/10
✅ docker_containers_memory memory_usage 10/10
✅ docker_containers_memory simple_check_run 10/10
✅ file_to_blackhole_0ms_latency lost_bytes 10/10
✅ file_to_blackhole_0ms_latency memory_usage 10/10
✅ file_to_blackhole_1000ms_latency lost_bytes 10/10
✅ file_to_blackhole_1000ms_latency memory_usage 10/10
✅ file_to_blackhole_100ms_latency lost_bytes 10/10
✅ file_to_blackhole_100ms_latency memory_usage 10/10
✅ file_to_blackhole_500ms_latency lost_bytes 10/10
✅ file_to_blackhole_500ms_latency memory_usage 10/10
✅ quality_gate_idle intake_connections 10/10 bounds checks dashboard
✅ quality_gate_idle memory_usage 10/10 bounds checks dashboard
✅ quality_gate_idle_all_features intake_connections 10/10 bounds checks dashboard
✅ quality_gate_idle_all_features memory_usage 10/10 bounds checks dashboard
✅ quality_gate_logs intake_connections 10/10 bounds checks dashboard
✅ quality_gate_logs lost_bytes 10/10 bounds checks dashboard
✅ quality_gate_logs memory_usage 10/10 bounds checks dashboard
✅ quality_gate_metrics_logs cpu_usage 10/10 bounds checks dashboard
✅ quality_gate_metrics_logs intake_connections 10/10 bounds checks dashboard
✅ quality_gate_metrics_logs lost_bytes 10/10 bounds checks dashboard
✅ quality_gate_metrics_logs memory_usage 10/10 bounds checks dashboard

Explanation

Confidence level: 90.00%
Effect size tolerance: |Δ mean %| ≥ 5.00%

Performance changes are noted in the perf column of each table:

  • ✅ = significantly better comparison variant performance
  • ❌ = significantly worse comparison variant performance
  • ➖ = no significant change in performance

A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".

For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:

  1. Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.

  2. Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.

  3. Its configuration does not mark it "erratic".

CI Pass/Fail Decision

✅ Passed. All Quality Gates passed.

  • quality_gate_metrics_logs, bounds check intake_connections: 10/10 replicas passed. Gate passed.
  • quality_gate_metrics_logs, bounds check memory_usage: 10/10 replicas passed. Gate passed.
  • quality_gate_metrics_logs, bounds check cpu_usage: 10/10 replicas passed. Gate passed.
  • quality_gate_metrics_logs, bounds check lost_bytes: 10/10 replicas passed. Gate passed.
  • quality_gate_idle_all_features, bounds check intake_connections: 10/10 replicas passed. Gate passed.
  • quality_gate_idle_all_features, bounds check memory_usage: 10/10 replicas passed. Gate passed.
  • quality_gate_idle, bounds check intake_connections: 10/10 replicas passed. Gate passed.
  • quality_gate_idle, bounds check memory_usage: 10/10 replicas passed. Gate passed.
  • quality_gate_logs, bounds check intake_connections: 10/10 replicas passed. Gate passed.
  • quality_gate_logs, bounds check lost_bytes: 10/10 replicas passed. Gate passed.
  • quality_gate_logs, bounds check memory_usage: 10/10 replicas passed. Gate passed.

@rdesgroppes
rdesgroppes force-pushed the regis.desgroppes/extend-bazel-test-scope branch 2 times, most recently from d968077 to b8e899a Compare December 19, 2025 15:33
@github-actions github-actions Bot added long review PR is complex, plan time to review it and removed medium review PR review might take time labels Dec 19, 2025

@aiuto aiuto left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The module generator is something we should do.
bazel test //... is probably not.

script:
- bazel test //bazel/tests/...
- bazel test //...

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's discuss this as a team first. It is going to get big and complex soon, especially with multiple platforms. It is not always possible to write strict test skipping with target_compatible_with.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. I've been extracting "less disputed" bits into small PRs until we're all back.

@rdesgroppes
rdesgroppes force-pushed the regis.desgroppes/extend-bazel-test-scope branch from b8e899a to 6acf7cc Compare January 8, 2026 13:01
@github-actions github-actions Bot added medium review PR review might take time and removed long review PR is complex, plan time to review it labels Jan 8, 2026
@rdesgroppes rdesgroppes changed the title Extend bazel deps module consistency enforcement Extend bazel test scope Jan 8, 2026
@rdesgroppes
rdesgroppes force-pushed the regis.desgroppes/extend-bazel-test-scope branch 2 times, most recently from e5f9dba to d1864d3 Compare January 16, 2026 11:08
@agent-platform-auto-pr

agent-platform-auto-pr Bot commented Jan 16, 2026 •

Copy link
Copy Markdown
Contributor

Gitlab CI Configuration Changes

Modified Jobs

bazel:test:windows-amd64
  bazel:test:windows-amd64:
    before_script:
    - C:\ci-identities-gitlab-job-client.exe assume-role
    id_tokens:
      CI_IDENTITIES_GITLAB_ID_TOKEN:
        aud: ci-identities
    needs: []
    rules:
-   - if: $SKIP_WINDOWS == "true"
-     when: never
    - when: on_success
    script:
    - '$FailFast = @''
  
      $ErrorActionPreference = ''Stop''
  
      $PSNativeCommandUseErrorActionPreference = $true
  
      Set-StrictMode -Version 3.0
  
      ''@
  
      Invoke-Expression $FailFast
  
      $Utf8Script = [Text.Encoding]::UTF8.GetString([Text.Encoding]::GetEncoding([Console]::OutputEncoding.CodePage).GetBytes($POWERSHELL_SCRIPT))
  
      $EncodedCommand = [Convert]::ToBase64String([Text.Encoding]::Unicode.GetBytes("$FailFast`n$Utf8Script"))'
    - .\tools\ci\docker-run-with-bazel-cache.ps1 --env=CI_IDENTITIES_GITLAB_ID_TOKEN
      --volume="${CI_PROJECT_DIR}:$CI_PROJECT_DIR" --workdir="$CI_PROJECT_DIR" "$WINBUILDIMAGE"
      powershell -NonInteractive -NoLogo -NoProfile -InputFormat Text -OutputFormat
      Text -EncodedCommand $EncodedCommand
    stage: source_test
    tags:
    - windows-v2:2022
    timeout: 60m
    variables:
      ARCH: x64
      AWS_SHARED_CREDENTIALS_FILE: ${CI_PROJECT_DIR}\.aws\credentials-by-job-id\${CI_JOB_ID}
      BAZELISK_HOME: $XDG_CACHE_HOME/bazelisk
      GIT_DEPTH: 2
      OVERRIDE_GIT_STRATEGY: clone
-     POWERSHELL_SCRIPT: bazel test //bazel/tests/... //rtloader/...
+     POWERSHELL_SCRIPT: bazel test //...
      WINBUILDIMAGE: registry.ddbuild.io/ci/datadog-agent-buildimages/windows_ltsc2022_${ARCH}${CI_IMAGE_WIN_LTSC2022_X64_SUFFIX}:${CI_IMAGE_WIN_LTSC2022_X64}
      XDG_CACHE_HOME: c:/bzl

Changes Summary

Removed Modified Added Renamed
0 1 0 0

ℹ️ Diff available in the job log.

@rdesgroppes
rdesgroppes force-pushed the regis.desgroppes/extend-bazel-test-scope branch from d1864d3 to ed34984 Compare January 19, 2026 12:15
dd-mergequeue Bot pushed a commit that referenced this pull request Jan 19, 2026
### What does this PR do?
This upgrade includes a Windows compatibility fix for batch file line endings (bazel-contrib/bazel-lib#1222), which resolves a cmd parsing bug where GOTO/CALL to labels fails when batch files use non CRLF-only line endings and a label crosses a 512-byte boundary:
```diff
> bazel test //deps/openssl3:module_consistency_test
[...]
==================== Test output for //deps/openssl3:module_consistency_test:
-The system cannot find the batch label specified - compare_files
+compare_files
+FAIL: files "C:\[...]\bazel-out\[...]\deps\openssl3\openssl3.MODULE.bazel.new" and "C:\[workspace]\deps\openssl3\openssl3.MODULE.bazel" differ.
+
+
+@@//deps/openssl3:openssl3.MODULE.bazel is out of date. To update this file, run:
+
+    bazel run //deps/openssl3:module_consistency
+To see differences run:
+
+    diff "C:\[...]\bazel-out\[...]\deps\openssl3\openssl3.MODULE.bazel.new" "C:\[workspace]\deps\openssl3\openssl3.MODULE.bazel"
+
================================================================================
```

Because this now surfaces we'd been generating `*.MODULE.bazel` with OS-specific newlines, the present change fixes it by always issuing UNIX-style newlines:
```
> bazel test //deps/openssl3:module_consistency_test
[...]
//deps/openssl3:module_consistency_test                                  PASSED in 0.4s
```

### Motivation
Keep up to date and better support colleagues working on Windows.

### Describe how you validated your changes
Locally for the time being because we still have very limited test coverage due to #44455 being disputed.

### Additional Notes
`bazel_lib` 3.1.0/3.1.1 comes with further improvements, like:
- bazel-contrib/bazel-lib#1217
- bazel-contrib/bazel-lib#1220
- bazel-contrib/bazel-lib#1232


Co-authored-by: regis.desgroppes <regis.desgroppes@datadoghq.com>
@rdesgroppes
rdesgroppes force-pushed the regis.desgroppes/extend-bazel-test-scope branch 2 times, most recently from 1de9de4 to 4bbe0a0 Compare January 22, 2026 15:23
@rdesgroppes
rdesgroppes force-pushed the regis.desgroppes/extend-bazel-test-scope branch from 4bbe0a0 to 7e238ba Compare February 3, 2026 08:32
### What does this PR do?
- extend `bazel test` scope in CI from `//bazel/tests/...` to `//...`,
- fix remaining failures after doing so.

### Motivation
The `deps/acl/acl.MODULE.bazel` file was out of sync with its generated
version (using `url` instead of `urls`), but this inconsistency was not
caught.
@rdesgroppes
rdesgroppes force-pushed the regis.desgroppes/extend-bazel-test-scope branch from 7e238ba to ad896a6 Compare March 21, 2026 02:59
@dd-octo-sts dd-octo-sts Bot added the internal Identify a non-fork PR label Mar 21, 2026
@agent-platform-auto-pr

Copy link
Copy Markdown
Contributor

Files inventory check summary

File checks results against ancestor b1362953:

Results for datadog-agent_7.79.0~devel.git.40.ad896a6.pipeline.103842958-1_amd64.deb:

No change detected

@dd-octo-sts

dd-octo-sts Bot commented Apr 5, 2026

Copy link
Copy Markdown
Contributor

This pull request has been automatically marked as stale because it has not had activity in the past 15 days.

It will be closed in 30 days if no further activity occurs. If this pull request is still relevant, adding a comment or pushing new commits will keep it open. Also, you can always reopen the pull request if you missed the window.

Thank you for your contributions!

@dd-octo-sts dd-octo-sts Bot added the stale label Apr 5, 2026
@dd-octo-sts

dd-octo-sts Bot commented May 6, 2026

Copy link
Copy Markdown
Contributor

This pull request was automatically closed because it has been stale for 15 days with no activity.

If this pull request is still relevant, please reopen it or create a new pull request with updated information.

Thanks!

@dd-octo-sts dd-octo-sts Bot closed this May 6, 2026
@gh-worker-devtools-f43747
gh-worker-devtools-f43747 Bot deleted the regis.desgroppes/extend-bazel-test-scope branch September 20, 2026 03:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-closed changelog/no-changelog No changelog entry needed internal Identify a non-fork PR medium review PR review might take time qa/no-code-change No code change in Agent code requiring validation stale team/agent-build

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants