Skip to content

bpo-43667: Fix broken Unicode encoding in non-UTF locales on Solaris #25096

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vstinner merged 12 commits into from
Apr 30, 2021
Merged

bpo-43667: Fix broken Unicode encoding in non-UTF locales on Solaris #25096

Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
59548ca
fix wchar_t conversion on Solaris
kulikjak Mar 30, 2021
cb6452c
Add configure detection of Oracle Solaris and its uncommon wchar_t ha…
kulikjak Apr 1, 2021
c01b792
Rework wchar_t conversion based on suggestions
kulikjak Apr 1, 2021
0fc7f54
Incorporate further suggestions
kulikjak Apr 6, 2021
5fdd1a2
Incorporate additional suggestions and rework wchar_t conversion
kulikjak Apr 26, 2021
bad1eba
Minor fix
kulikjak Apr 28, 2021
e8dd8d1
Add reverse conversion from UCS-4 to native wchar_t
kulikjak Apr 28, 2021
4aebe1d
Fix one more issue introduced with bpo-35883
kulikjak Apr 28, 2021
afaeaa2
Fix include
kulikjak Apr 28, 2021
627e460
Clean unused includes
kulikjak Apr 29, 2021
02a37ee
Change return value of _Py_LocaleUsesNonUnicodeWchar to reflect its name
kulikjak Apr 29, 2021
00956fe
More improvements
kulikjak Apr 30, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix one more issue introduced with bpo-35883
  • Loading branch information
@kulikjak
kulikjak committed Apr 28, 2021
commit 4aebe1d9e6c06477e46ba69aa5380e10418046be
6 changes: 6 additions & 0 deletions Python/fileutils.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,12 @@ _Py_device_encoding(int fd)
static size_t
is_valid_wide_char(wchar_t ch)
{
#ifdef HAVE_NON_UNICODE_WCHAR_T_REPRESENTATION
/* Oracle Solaris doesn't use Unicode code points as wchar_t encoding
for non-Unicode locales, which makes values higher than MAX_UNICODE
possibly valid. */
return 1;
#endif
if (Py_UNICODE_IS_SURROGATE(ch)) {
// Reject lone surrogate characters
return 0;
Expand Down