fix: capped responses keep elapsed, next_request and default_encoding; capped redirect and auth hops reuse their connection - #179

What changed

Both fixes apply only when max_response_body_bytes is set, and to both Client and AsyncClient.

The rebuilt response keeps three attributes

With a cap, httpware reads the body under the cap and rebuilds the response with httpx2.Response(content=...). That rebuild used to lose:

• elapsed, which raised RuntimeError;
• next_request, so a 3xx returned with follow_redirects=False had next_request = None;
• the client's default_encoding, so café sent as latin-1 with no charset decoded as caf� on a client built with default_encoding="latin-1".

All three are now copied with public httpx2 API: the default_encoding constructor argument, the next_request attribute and the elapsed setter. To get elapsed, the streaming response is closed once its body has been read in full, because httpx2 records elapsed time when the stream closes. Closing is idempotent, so the callers' own close() is unchanged.

Redirect and auth hops reuse their connection

Under a cap, httpware follows redirects and runs the auth flow itself. It used to close each intermediate response (a redirect, a DigestAuth challenge) without reading the body. httpx2 cannot return a connection with an unread body to the pool, so every hop opened a new TCP connection. httpware now reads and discards each intermediate body before closing it. The cap still bounds that read. A hop whose Content-Length is over the cap is not read at all, and a hop body that grows past the cap is abandoned once it does. Either way the connection is closed, as before, and the request carries on. Nothing is kept, so response.history entries still have no content.

ADR-0009

ADR-0009 said keeping elapsed would need httpx2 internals. That was wrong: Response.elapsed has a public setter. The ADR now describes what the rebuilt response keeps and points to #170, which tracks counting bytes in a stream wrapper so the rebuild can go away. The decision itself (decoded bytes, one read path) is unchanged. docs/errors.md describes the new hop behaviour and what the rebuilt response keeps and loses.

Behaviour changes for the 0.22.0 release notes

• With a cap set, response.elapsed, response.next_request and response.text (through the client's default_encoding) now match what you get without a cap.
• With a cap set, redirect and auth hop bodies are read up to the cap and thrown away, so hops reuse the pooled connection. A hop body declared or read past the cap is still not an error. A hop body that fails mid-read now raises the mapped transport error, as it does without a cap, where httpx2 reads every hop body.
• Still lost on a capped response: num_bytes_downloaded reads 0, because httpx2 has no public setter for it. The Content-Encoding and Transfer-Encoding headers are dropped and Content-Length is the decoded size, as before.

Why not raise on an oversized hop body

Raising ResponseTooLargeError for an oversized hop body would match the final response. It would also break requests that work today: the existing tests pin that a 100 KB redirect or challenge body under a 1 KB cap does not fail the request. Abandoning the hop gives the same memory bound without the break, and costs one dropped connection only in that rare case.

Test evidence

Each fix was test-first, in both worlds:

• tests/test_client_body_cap.py: test_{sync,async}_response_keeps_{elapsed,next_request,default_encoding}_with_or_without_a_cap. Before the fix, the capped cases failed with RuntimeError: '.elapsed' may only be accessed after the response has been read or closed., assert None is not None (next_request) and assert 'caf�' == 'café'.
• tests/test_client_body_cap_redirects.py and tests/test_client_body_cap_auth.py: test_{sync,async}_reads_an_intermediate_redirect_body_to_the_end_with_or_without_a_cap and test_{sync,async}_reads_a_digest_challenge_body_to_the_end_with_or_without_a_cap use a hop body generator that records when it is read to the end. They failed before the fix with assert [] == [True]. The old tests asserting that no hop body is read under a cap now assert that it is read only up to the cap: 2 of 100 chunks, for send() and stream(). A new parametrized test pins that a hop declaring a Content-Length over the cap is not read, except for HEAD.

MockTransport cannot show connection reuse, so I checked it against a local http.server with keep-alive, counting accepted connections over 200 GETs through a 5-hop redirect chain:

On loopback the capped sync GET went from 3.78 ms to 2.46 ms, about what the uncapped path takes (2.37 ms).

just lint, just lint-ci and just test-ci pass: 1219 tests, 100% coverage. The cap tests also pass on Python 3.11.

Performance

The single-request capped path does about 0.4 µs more work: two attribute sets, one constructor argument, and a close() that is a no-op on a fully read stream. Against roughly 60 µs per capped get() on MockTransport, that is below the noise of the before/after timeit runs. Hops cost one read of the body to free the connection, which saves a TCP connect per hop.