Skip to content

Avoid reactor netty continuation leakage#11559

Draft
amarziali wants to merge 3 commits into
masterfrom
andrea.marziali/netty-http
Draft

Avoid reactor netty continuation leakage#11559
amarziali wants to merge 3 commits into
masterfrom
andrea.marziali/netty-http

Conversation

@amarziali
Copy link
Copy Markdown
Contributor

@amarziali amarziali commented Jun 4, 2026

What Does This Do

This PR fixes some scope propagation leakage happening on netty-reactor. This also impacts the spring-webflux client since it's based on this last one.

A couple of different fixes:

  1. Add propagation barrier on HttpClientConnect$MonoHttpConnect.subscribe(). During the http client subscription, the event-loop tasks are submitted while the span scope is active; hence the concurrent instrumentation would capture that continuation that survives when the span finishes and it's not needed
  2. Fix the TransferConnectSpan for http2: In a http2 context the channel is multiplexed. We previously captured the connect span but in case of http2 the parent context is not used hence the continuation was dangling. This PR ensures that we properly cancel it

Please note that removeStrictTraces has been successfully removed from the concerned tests that are now passing OK (without this PR they are timing out as expected)

Note: Perhaps a broader re-evaluation of reactor context propagation is needed to avoid doing defensive fixes like this one

Motivation

Additional Notes

Contributor Checklist

  • Format the title according to the contribution guidelines
  • Assign the type: and (comp: or inst:) labels in addition to any other useful labels
  • Avoid using close, fix, or any linking keywords when referencing an issue
    Use solves instead, and assign the PR milestone to the issue
  • Update the CODEOWNERS file on source file addition, migration, or deletion
  • Update public documentation with any new configuration flags or behaviors
  • Add your completed PR to the merge queue by commenting /merge. You can also:
    • Customize the commit message associated with the merge with /merge --commit-message "..."
    • Remove your PR from the merge queue with /merge -c
    • Skip all merge queue checks with /merge -f --reason "reason"; please use this judiciously, as some checks do not run at the PR-level (note: the PR still needs to be mergeable, this will only skip the pre-merge build)
    • Get more information in this doc

Jira ticket: [PROJ-IDENT]

@amarziali amarziali requested review from a team as code owners June 4, 2026 14:11
@amarziali amarziali added the type: bug Bug report and fix label Jun 4, 2026
@amarziali amarziali requested review from bric3 and removed request for a team June 4, 2026 14:11
@amarziali amarziali added the inst: netty Netty instrumentation label Jun 4, 2026
@amarziali amarziali requested review from mcculls and ygree and removed request for a team June 4, 2026 14:11
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 480f2c0d1e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

// Avoid leaking continuations on the netty event loop that initiates the tasks that handle the
// request lifecycle
transformer.applyAdvice(
named("subscribe").and(isDeclaredBy(REACTOR_NETTY_HTTP_CONNECT)), advice);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve connect error propagation under the barrier

For Reactor Netty requests that fail during TCP connect before doOnRequest/the HTTP write runs (for example a refused or unroutable endpoint), this disables async propagation for the whole MonoHttpConnect.subscribe call, so Netty's ConnectAdvice sees captureActiveSpan() as noop and never stores CONNECT_PARENT_CONTINUATION_ATTRIBUTE_KEY; ChannelFutureListenerInstrumentation then returns when that continuation is missing and skips the netty.connect error span. TransferConnectSpan only runs once a request exists, so it cannot restore the parent for these connect-failure paths.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well to be fair the testConnectionFailure looks disabled today, sign that it looks uncovered/not working. I'll try to do an effort to see if it's somehow feaseable

@datadog-official

This comment has been minimized.

@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented Jun 4, 2026

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 13.98 s 13.97 s [-1.0%; +1.1%] (no difference)
startup:insecure-bank:tracing:Agent 12.94 s 12.91 s [-0.8%; +1.4%] (no difference)
startup:petclinic:appsec:Agent 16.67 s 16.46 s [+0.1%; +2.5%] (maybe worse)
startup:petclinic:iast:Agent 16.47 s 16.65 s [-1.9%; -0.2%] (maybe better)
startup:petclinic:profiling:Agent 16.69 s 16.65 s [-0.9%; +1.4%] (no difference)
startup:petclinic:tracing:Agent 15.71 s 15.73 s [-1.5%; +1.2%] (no difference)

Commit: c57ea0d0 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@amarziali amarziali marked this pull request as draft June 4, 2026 15:25
@amarziali amarziali force-pushed the andrea.marziali/netty-http branch from 480f2c0 to c57ea0d Compare June 4, 2026 16:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

inst: netty Netty instrumentation type: bug Bug report and fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant