feat(langgraph): Gate prompt/response collection on data_collection option - #7175
feat(langgraph): Gate prompt/response collection on data_collection option#7175ericapisani wants to merge 2 commits into
Conversation
…ption Modify the LangGraph integration to respect the data_collection config for controlling whether prompts, responses, tool calls, and available tools are captured in spans. When data collection is enabled, the gen_ai.inputs flag controls request messages, tool calls, and available tools, while gen_ai.outputs controls the response text. Tool calls are gated on inputs because they are fed back to the model as input. Available tools are only gated once data collection is configured, since they were never gated on the legacy PII settings. When data collection is not configured, falls back to legacy send_default_pii and include_prompts settings for compatibility. Refs PY-2588 Refs #6748
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit f54e39e. Configure here.
| and should_send_default_pii() | ||
| and integration.include_prompts | ||
| ): | ||
| if len(args) > 0 and _should_record_inputs(integration): |
There was a problem hiding this comment.
Inputs gate breaks message delta
Medium Severity
Parsing input_messages is gated on _should_record_inputs, but those messages are also required by _get_new_messages to compute the output delta. With gen_ai.inputs false and gen_ai.outputs true, input_messages stays None, so the full history is treated as new. That can put prior-turn or user input content into gen_ai.response.text and inflate usage totals.
Additional Locations (2)
Reviewed by Cursor Bugbot for commit f54e39e. Configure here.
Rename the data collection tests and the invoke span helper so the names state what is being verified, and drop the now-redundant docstrings.
Codecov Results 📊✅ 104618 passed | ⏭️ 6677 skipped | Total: 111295 | Pass Rate: 94% | Execution Time: 366m 57s 📊 Comparison with Base Branch
All tests are passing successfully. ✅ Patch coverage is 96.67%. Project has 2478 uncovered lines. Files with missing lines (1)
Coverage diff@@ Coverage Diff @@
## main #PR +/-##
==========================================
+ Coverage 90.14% 90.16% +0.02%
==========================================
Files 193 193 —
Lines 25183 25192 +9
Branches 9176 9186 +10
==========================================
+ Hits 22700 22714 +14
- Misses 2483 2478 -5
- Partials 1429 1431 +2Generated by Codecov Action |
| True, | ||
| False, | ||
| True, | ||
| id="gen-ai-inputs-disabled-outputs-enabled", |
There was a problem hiding this comment.
Disabling gen_ai.inputs still leaks prompts via response collection
When gen_ai.inputs is false and gen_ai.outputs is true, invoke wrappers leave input_messages as None, so _get_new_messages() treats the full result history as new output and can attach user prompts to gen_ai.response.text (especially non-string/multimodal assistant content or the raw-result fallback). Always parse inputs for diffing; only gate writing gen_ai.request.messages.
Evidence
- In
sentry_sdk/integrations/langgraph.py,_wrap_pregel_invoke/_wrap_pregel_ainvokeonly call_parse_langgraph_messages(args[0])when_should_record_inputs()is true; otherwiseinput_messagesstaysNone. _set_response_attributes()still runs when_should_record_outputs()is true and calls_get_new_messages(input_messages, parsed_response_messages)._get_new_messages()returns the entire output message list wheninput_messagesis falsy, so prior user/system messages are treated as new output.- If
_extract_llm_response_text()does not find string assistant content, the code setsGEN_AI_RESPONSE_TEXTto that fullnew_messageslist (or rawresult), sending prompts despitegen_ai.inputs: false. - This independent inputs/outputs split is new; legacy gating tied both to the same PII/include_prompts check, so this path was not reachable before.
Identified by Warden · security-review · WEN-SJ5
| elif new_messages: | ||
| set_data_normalized(span, SPANDATA.GEN_AI_RESPONSE_TEXT, new_messages) |
There was a problem hiding this comment.
Disabling gen_ai.inputs leaks original prompts via gen_ai.response.text fallback
When gen_ai.inputs is disabled but gen_ai.outputs is enabled, and the LLM returns a non-text response (e.g. a tool call), the fallback that writes new_messages into gen_ai.response.text serialises the full result state—including the original user prompts that were meant to be withheld.
Evidence
_wrap_pregel_invokeparsesinput_messagesonly when_should_record_inputs(integration)is True, otherwise passingNoneinto_set_response_attributes._get_new_messages(None, output_messages)returns the entire result state, not just newly-added messages, because it has no input list to slice against._extract_llm_response_textreturnsNonefor assistant messages whosecontentis empty or missing (typical for pure tool-call responses).- The
elif new_messages:branch then writes that full state list intoGEN_AI_RESPONSE_TEXT, which includes the original user prompts thatgen_ai.inputs=Falsewas intended to suppress.
Also found at 1 additional location
tests/integrations/langgraph/test_langgraph.py:2443
Identified by Warden · find-bugs · VW6-3TE
Modify the LangGraph integration to respect the data_collection config for controlling whether prompts, responses, tool calls, and available tools are captured in spans.
When data collection is enabled, the gen_ai.inputs flag controls request messages, tool calls, and available tools, while gen_ai.outputs controls the response text. Tool calls are gated on inputs because they are fed back to the model as input. Available tools are only gated once data collection is configured, since they were never gated on the legacy PII settings.
When data collection is not configured, falls back to legacy send_default_pii and include_prompts settings for compatibility.
Refs PY-2588
Refs #6748