Repository navigation
gh-99927: Preserve UTF-8 headers under the SMTPUTF8 email policy - #159508
charan-rathore wants to merge 2 commits into
Conversation
Signed-off-by: Charan Rathore <180254320+charan-rathore@users.noreply.github.com>
|
The following commit authors need to sign the Contributor License Agreement: |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4966a32a52
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| value = value.encode('utf-8', 'surrogateescape').decode( | ||
| 'utf-8', 'surrogateescape') |
There was a problem hiding this comment.
Preserve address structure when switching to compat32
When a bytes message parsed with SMTPUTF8 is later serialized with policy=compat32 (the supported policy-switching flow described just above this method), this decoding changes a raw address from surrogateescaped bytes to an ordinary str. msg.as_bytes(policy=compat32) then passes To: Jörg <a@b> through Compat32's unstructured Header and emits To: =?utf-8?b?SsO2cmcgPGFAYj4=?=; reparsing that produces an invalid addr-spec instead of recipient a@b because the structural delimiters were encoded too. Before this change the default 8-bit fold_binary path preserved the structured raw bytes, so the UTF-8 conversion needs to be deferred to the SMTPUTF8-specific fold/fetch path or otherwise retain structure across the advertised policy switch.
Useful? React with 👍 / 👎.
Decoding surrogateescaped header bytes at parse time changed the value seen by every downstream policy: serializing an SMTPUTF8-parsed message with policy=compat32 then encoded-worded whole address headers and destroyed the addr-spec. Perform the conversion only when a utf8 policy actually refolds a source value, preserving structure under compat32. Adds a regression test for the SMTPUTF8-parse/compat32-serialize flow. Signed-off-by: Charan Rathore <180254320+charan-rathore@users.noreply.github.com>
When a message is parsed from bytes with
policy=SMTPUTF8and then flattened back to bytes, UTF-8 header values got mangled and the body could be demoted tounknown-8bit.header_source_parsenow round-trips the parsed header value through UTF-8 (withsurrogateescape) when the policy hasutf8enabled, so a parse/flatten round trip is stable.Adds regression tests covering the UTF-8 round trip and invalid header bytes.