gh-151316: Prefer UTF-8 for the en_IN locale alias - #155549
gh-151316: Prefer UTF-8 for the en_IN locale alias#155549sankalpsthakur wants to merge 1 commit into
Conversation
|
The following commit authors need to sign the Contributor License Agreement: |
|
You'll have to sign the CLA before we can review. |
e0bc095 to
ffd9b41
Compare
X11 locale.alias mapped en_IN to ISO8859-1, so getlocale() invented a codeset that does not exist on modern UTF-8-only systems and broke setlocale(getlocale()) round-trips. Prefer en_IN.UTF-8 (mirroring other modernized aliases) and cover the round-trip in tests. Signed-off-by: Sankalp Thakur <sankalphimself@gmail.com>
ffd9b41 to
b959df9
Compare
picnixz
left a comment
There was a problem hiding this comment.
Is it the only locale that could have this issue? I would prefer we sweep them all to check if this is the case first instead of having commits for each locale. Also, I think we should instead have a test class that checks for hardcoded replacements thatwe introduce instead of just the case of en_IN.
| self.assertEqual(locale.getlocale(locale.LC_CTYPE), loc) | ||
|
|
||
| def test_setlocale_from_getlocale_tuple(self): | ||
| # Reproduces the issue report: setlocale(LC_*, getlocale()). |
There was a problem hiding this comment.
This is irrelevant as a comment.
|
A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated. Once you have made the requested changes, please leave a comment on this pull request containing the phrase |
Summary
locale.locale_alias['en_in']incorrectly mapped toen_IN.ISO8859-1because X11locale.aliasoverrides glibc's UTF-8en_INduringmakelocalealiasregeneration.en_INis UTF-8-only, sogetlocale()inventedISO8859-1andsetlocale(getlocale())failed.en_IN.UTF-8in the alias table and hardcode the same carve-out inTools/i18n/makelocalealias.py(mirroringc.utf8).Validation
Lib/locale.py(system Python cannot usePYTHONPATH=Libwith this partial checkout due to encodings clash):normalize('en_IN') == 'en_IN.UTF-8'_parse_localename('en_IN') == ('en_IN', 'UTF-8')setlocale(LC_CTYPE, 'en_IN')→getlocale()→setlocale(..., loc)round-trip succeeded on this host (('en_IN', 'UTF-8'))./configure && make+./python.exe -m test test_localewas not run here (sparse/partial clone); relying on CI for the built interpreter suite.Fixes #151316
AI/LLM disclosure