You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Optimize matching of category escapes (\d, \w, ...) outside character sets #152033
The character class escapes \d, \D, \s, \S, \w and \W are currently always compiled to an IN block containing a single CATEGORY item, even when they occur outside a character set:
The IN wrapper costs three extra code words (IN, the skip, and FAILURE), an extra dispatch, and an indirect SRE(charset) call per character — even though there is only a single alternative.
A category escape that appears outside a set can instead be compiled directly to a bare CATEGORY opcode:
This also makes such an escape a "simple" repeatable unit, so \d+ uses the REPEAT_ONE fast path (handled by SRE(count)) instead of the generic REPEAT/MAX_UNTIL loop; a CATEGORY case is added to SRE(count) accordingly.
The transformation preserves behaviour exactly (the engine already matched the same category) and only changes the compiled byte code.
In a release build I measure ~1.3x geometric-mean speedup across a range of category-heavy patterns — roughly 1.7–2.0x on scans like \d+, \s+, \S+, and ~1.1–1.2x on realistic tokenizing, date, and IP-address patterns — together with ~20% smaller compiled byte code for those patterns. Patterns that do not use bare category escapes are unaffected.
Activity
pythongh-152033: Use a leading category as the INFO charset prefix
gh-152033: Optimize category escapes outside character sets (GH-152035)