Skip to content

fix!: errors and redaction (pickling, query masking, DecodeError message, read-only STATUS_TO_EXCEPTION) - #166

Merged
lesnik512 merged 6 commits into
mainfrom
fix/errors-redaction
Oct 17, 2026
Merged

lesnik512 merged 6 commits into
mainfrom
fix/errors-redaction

Conversation

@lesnik512

Copy link
Copy Markdown
Member

What changed

Pickling and copying

ClientError now has a single __reduce__. It rebuilds the instance with cls.__new__(cls, *args), skipping __init__, and restores __dict__. This replaces _KeywordReduceMixin, _reconstruct_kwonly and _reconstruct_status_error. Before, pickle, copy.copy and copy.deepcopy lost add_note() notes on every StatusError and raised TypeError (unexpected keyword __notes__) on the six keyword-only classes. A user-set attribute broke them the same way. That hits anything that ships exceptions across processes, such as a Celery worker or a multiprocessing pool, and retry already calls add_note() on the exceptions it gives up on. A new class no longer needs pickling code, just picklable attributes. The old invariant test that checked __dict__ against the __init__ keywords had nothing left to guard, so it is gone. Two tests replace it: a round trip over every exported error class (pickle, copy and deepcopy, each with a note and a user attribute), and an invariant that every exported ClientError class has a case in that round trip. The errors.py bullet in AGENTS.md now describes this rule.

Redaction

redact_url replaces every query-parameter value with REDACTED and keeps the keys as written, so ?page=2&X-Amz-Signature=abc becomes ?page=REDACTED&X-Amz-Signature=REDACTED. The old allowlist missed presigned-URL signatures and any other secret under a name it did not know. Fragment values get the same treatment, and a fragment with no = in it, such as #section-3, is kept. A bare query key like ?flag has no value and stays as it is. Userinfo is still stripped, now cut at the last @ (rpartition), so a raw user:p@ss@host no longer leaves ss@ behind. httpx2 percent-encodes @ in userinfo, so this only matters for raw strings passed to redact_url. Keys are no longer re-encoded through urlencode. The unredacted URL is still on exc.response.request.url.

Callers: StatusError.__str__/__repr__ (errors.py) and _internal/observability._emit_event, which redacts the url attribute of every event. All resilience events (retry, bulkhead, circuit breaker, timeout) go through _emit_event, and nothing builds a url attribute or exception message around it.

DecodeError message

The message is now failed to decode response into <model>: <OriginalType>, with (<n> errors) appended when the original has an error_count() method (pydantic's ValidationError; the count is singular for one error). It used to embed str(original), and pydantic puts the rejected input values in that string, so response body content ended up in the message. <model> is model.__name__ for a class and repr(model) otherwise. That fixes the AttributeError on Python 3.11 to 3.13 when response_model is X | None, and list[X] no longer shows as list. exc.original is unchanged.

STATUS_TO_EXCEPTION

STATUS_TO_EXCEPTION is a types.MappingProxyType, so assigning to it raises TypeError.

Phase decorator reprs

The phase decorators' reprs use getattr(f, '__qualname__', repr(f)), so repr() of a middleware built from a functools.partial or a callable instance no longer raises. The six ty: ignore comments are gone.

Behaviour changes for the 0.22.0 release notes

  • BREAKING: STATUS_TO_EXCEPTION is read-only. Code that assigned to it to remap a status now gets TypeError. To raise your own class for a status, catch the httpware error in a middleware (for example an on_error handler) and raise yours from it. Subclassing an httpware error does not remap anything by itself.
  • Exception messages, reprs, log records and span events now show every query value as REDACTED. Log queries or alerts that matched on query values in the url field or in str(exc) need updating.
  • str(DecodeError) no longer contains the decoder's own message. Read exc.original when you need it.
  • Notes and attributes added to any httpware exception now survive pickle and copy.

Test evidence

Each fix started with a failing test:

  1. test_error_round_trip_keeps_notes_and_attributes: 54 of 66 cases failed, with AttributeError: 'NotFoundError' object has no attribute '__notes__' on the status classes and TypeError: DecodeError.__init__() got an unexpected keyword argument '__notes__' on the keyword-only ones.
  2. test_status_to_exception_cannot_be_mutated: Failed: DID NOT RAISE TypeError.
  3. test_redact_url[...page=2&limit=10...] and its neighbours, test_status_error_message_masks_every_query_value, and both test_emit_event_redacts_url_secret_* tests failed with the values still in place. The rpartition case failed with 'https://ss@example.test/p' == 'https://example.test/p'.
  4. The test_decode_error_summary_* tests failed with the pydantic message (1 validation error for _DecodeErrorModel ...) in str(exc). On 3.14 X | None rendered as Union and list[X] as list. The same test on 3.13 raised AttributeError: 'types.UnionType' object has no attribute '__name__'.
  5. test_decorator_repr_falls_back_to_repr_without_qualname (sync and async): 12 failures, AttributeError: 'functools.partial' object has no attribute '__qualname__' and the same for a callable instance.

just lint, just lint-ci and just test pass: 1087 tests on 3.14, and the full suite also passes on 3.13.

Performance

timeit on CPython 3.14.7, best of 7, ns per call, before and after:

before after
redact_url, no query 166 75
redact_url, ?page=2&limit=50&sort=name 2110 1034
redact_url, query with api_key 5558 1092
redact_url, presigned S3 URL 4967 1711
redact_url, userinfo plus query 2249 1843
NotFoundError(response), URL with query 6586 2142
DecodeError(...) with a pydantic ValidationError 1888 711

Masking reuses the string pieces instead of running parse_qsl and urlencode, and returns early when the URL has no =.

One __reduce__ on ClientError rebuilds from args without calling __init__
and restores __dict__, replacing _KeywordReduceMixin and the StatusError
reconstructor. add_note() notes and user-set attributes now survive pickle,
copy.copy and copy.deepcopy for every error class.
STATUS_TO_EXCEPTION is now a types.MappingProxyType. Code that assigned to
it to remap a status now gets TypeError; catch the error in a middleware
and raise your own class instead.
redact_url now replaces the value of every query and fragment parameter
with REDACTED and keeps the keys, instead of masking an allowlist of
sensitive names, so presigned-URL signatures and other unlisted secrets
no longer reach messages, reprs, log records or span events. Userinfo is
cut at the last @ so a raw password containing @ is stripped whole.
The message is now 'failed to decode response into <model>: <type>' with
the error count appended when the original exception has error_count().
A non-class model such as X | None is shown by repr, which stops the
AttributeError on Python 3.11 to 3.13. exc.original is unchanged.
repr() of a middleware built by before_request, after_response, on_error
or their async siblings no longer raises AttributeError for a
functools.partial or a callable instance; it falls back to repr(f).
@lesnik512
lesnik512 merged commit b00487c into main Oct 17, 2026
13 checks passed
@lesnik512
lesnik512 deleted the fix/errors-redaction branch October 17, 2026 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant