Repository navigation
Raw f-string format spec undocumented behavior change between 3.11 and 3.12 #137314
Description
Bug report
Bug description:
I found this while working on astral-sh/ruff#19546 .
Between 3.11 and 3.12, the behavior of raw f-string format specifiers changed.
On 3.11 and earlier, the rawness of the f-string is respected in the format specifier:
PS ~>uvx python@3.11 -c @'
class UnchangedFormat:
def __format__(self, format):
return format
print("Non-raw output:", repr(f"{UnchangedFormat():\xFF}"))
print("Raw output:", repr(rf"{UnchangedFormat():\xFF}"))
'@
Non-raw output: 'ÿ'
Raw output: '\\xFF'On 3.12 and later, the rawness of the f-string is not respected:
PS ~>uvx python@3.12 -c @'
class UnchangedFormat:
def __format__(self, format):
return format
print("Non-raw output:", repr(f"{UnchangedFormat():\xFF}"))
print("Raw output:", repr(rf"{UnchangedFormat():\xFF}"))
'@
Non-raw output: 'ÿ'
Raw output: 'ÿ'This new behavior looks to have been carried to t-strings on 3.14 and later:
PS ~>uvx python@3.14 -c @'
class UnchangedFormat:
def __format__(self, format):
return format
print("Non-raw output:", repr(t"{UnchangedFormat():\xFF}"))
print("Raw output:", repr(rt"{UnchangedFormat():\xFF}"))
'@
Non-raw output: Template(strings=('', ''), interpolations=(Interpolation(<__main__.UnchangedFormat object at 0x00000183F9904EC0>, 'UnchangedFormat()', None, 'ÿ'),))
Raw output: Template(strings=('', ''), interpolations=(Interpolation(<__main__.UnchangedFormat object at 0x00000183F9908A50>, 'UnchangedFormat()', None, 'ÿ'),))I could not find any information/documentation about how the rawness of f-strings affect their format specifiers, despite looking in:
- The input output tutorial f-string section
- The lexical analysis section on f-strings
- PEP 498 – Literal String Interpolation
- PEP 701 – Syntactic formalization of f-strings
- The 3.12 what's new section
- Multiple CPython github issue searches
- Asking in the Python discord
CPython versions tested on:
3.12, 3.11, 3.14
Operating systems tested on:
Windows
Activity
picnixz commentedon Aug 1, 2025
Unfortunately, 3.11 and 3.12 are both security-only and I don't think this is a security issue. AFAIK, there isn't an exceptional rule for such bugs, so even if this is a regression, we won't patch it :( However, we may fix it in 3.13+ (maybe?)
cc @pablogsal
ericvsmith commentedon Aug 2, 2025
This looks like a bug to me. I think it should be fixed in 3.14 and backported to 3.13.
pablogsal commentedon Aug 2, 2025
Agreed 👍
gh-137314: Fix incorrect treatment of format specs in raw fstrings (#…
pythongh-137314: Fix incorrect treatment of format specs in raw fstri…
pythongh-137314: Fix incorrect treatment of format specs in raw fstri…
fixup! pythongh-137314: Fix incorrect treatment of format specs in ra…
[3.14] gh-137314: Fix incorrect treatment of format specs in raw fstr…
[3.13] gh-137314: Fix incorrect treatment of format specs in raw fstr…
MeGaGiGaGon commentedon Aug 3, 2025
Looks like the PRs have gone through, wow that was fast :)
The only remaining questions I have are on documentation:
pablogsal commentedon Aug 3, 2025
Yeah. Would you like to do the PR to the docs?
It depends on @Yhg1s (see #137328 (comment)) but likely it won't be back ported
MeGaGiGaGon commentedon Aug 3, 2025
Not particularly, I'm both not sure on the wording and on where it should go.
Yhg1s commentedon Aug 4, 2025
Indeed, it's not really the kind of thing we'd backport at this stage. Documenting the difference might be worth the effort, although given that it took this long for people to report it... maybe?
pablogsal commentedon Aug 4, 2025
yeah to me this looks fundamentally low risk, likely only affecting linters in their test suite :)
ericvsmith commentedon Aug 6, 2025
I agree we don't want to backport it: there's only been one bug report in the 2+ years it's been a problem.
picnixz commentedon Aug 7, 2025
Sorry I closed it but I think we want to add a doc note in which case the issue is still needed (?)
ericvsmith commentedon Aug 7, 2025
I'm not sure we need any documentation (it's just how raw strings are expected to work).
MeGaGiGaGon commentedon Aug 7, 2025
Given all the discussion above this sounds good to me too, thank you all for your help on this
pythongh-137314: Fix incorrect treatment of format specs in raw fstri…
[3.14] pythongh-137314: Fix incorrect treatment of format specs in ra…