Skip to content

strxfrm fails on macOS 15 with OSError: [Errno 22] Invalid argument #130567

New issue
New issue
Closed
Closed
strxfrm fails on macOS 15 with OSError: [Errno 22] Invalid argument#130567
Labels
OS-macextension-modulesC modules in the Modules dirtype-bugAn unexpected behavior, bug, or error

Description

@nijel
nijel
opened on Feb 26, 2025

Bug report

Bug description:

It appears that macOS 15 broke locale.strxfrm for letters outside the latin-1 set:

import locale
locale.setlocale(locale.LC_ALL, 'en_US.UTF-8')
locale.strxfrm('Français') # works
locale.strxfrm('zkouška') # crashes with OSError
locale.strxfrm('中文') # crashes with OSError

This works on Linux and macOS 14 and crashes on macOS 15, tested with Python 3.9–3.14: https://github.com/nijel/python-strxfrm/actions/runs/13541306384/job/37842825809

There are several projects which run into this already:

In most cases, this is being addressed by completely removing locale-aware sorting.

CPython versions tested on:

3.14

Operating systems tested on:

macOS

Linked PRs

Activity

  1. nijel
    added
    type-bugAn unexpected behavior, bug, or error
    on Feb 26, 2025

  2. nijel commented on Feb 26, 2025

    @nijel
    nijel
    on Feb 26, 2025
    ContributorAuthor

    Looking more into that I've found https://swi-prolog.discourse.group/t/compiling-swi-prolog-from-git-source-on-next-macos-15-beta/7590/1. They figured out that wcsxfrm does not actually return desired string size in case of the failure, so they ended up allocating double space in that case, see SWI-Prolog/swipl-devel@f0bbfb0.

    The following code would have to be changed:

    cpython/Modules/_localemodule.c

    Line 426 in 263d56e

    wchar_t * new_buf = PyMem_Realloc(buf, (n2+1)*sizeof(wchar_t));

  3. tomasr8
    added this to Locale issues 🗺on Feb 26, 2025
  4. github-project-automation
    moved this to Todo in Locale issues 🗺on Feb 26, 2025
  5. tomasr8
    added
    OS-mac
    on Feb 26, 2025
  6. nijel
    added a commit that references this issue on Feb 27, 2025

    pythongh-130567: fix strxfrm memory allocation

    f346b03

  7. nijel commented on Feb 27, 2025

    @nijel
    nijel
    on Feb 27, 2025
    ContributorAuthor

    I've tried to address this #130619.

  8. nijel
    added 5 commits that reference this issue on Feb 27, 2025

    pythongh-130567: fix strxfrm memory allocation

    8937cbb

    pythongh-130567: fix strxfrm memory allocation

    75f83f4

    pythongh-130567: fix strxfrm memory allocation

    94ba382

    pythongh-130567: fix strxfrm memory allocation

    cbdc389

    pythongh-130567: fix strxfrm memory allocation

    a6c5843

  9. serhiy-storchaka commented on Feb 28, 2025

    @serhiy-storchaka
    serhiy-storchaka
    on Feb 28, 2025
    Member

    What error do you get? What code?

  10. nijel
    added a commit that references this issue on Feb 28, 2025

    pythongh-130567: fix strxfrm memory allocation

    e3c5009

  11. serhiy-storchaka commented on Feb 28, 2025

    @serhiy-storchaka
    serhiy-storchaka
    on Feb 28, 2025
    Member

    Other reports show "[Errno 22] Invalid argument". This is EINVAL. But the re-allocation code is only executed if errno is 0 or ERANGE (34), so your fix cannot change the outcome.

  12. 32 remaining items

  13. miss-islington
    added 2 commits that reference this issue on Sep 23, 2025

    pythongh-130567: Fix possible crash in locale.strxfrm() (pythonGH-138940

    d456311

    pythongh-130567: Fix possible crash in locale.strxfrm() (pythonGH-138940

    1a92e15
  14. encukou
    added 2 commits that reference this issue on Sep 24, 2025

    [3.13] gh-130567: Fix possible crash in locale.strxfrm() (GH-138940) (G…

    d1f6b39

    gh-130567: Enable previously skipped locale tests on FreeBSD and macOS (

    1b8dcda
  15. miss-islington
    added 2 commits that reference this issue on Sep 29, 2025

    pythongh-130567: Enable previously skipped locale tests on FreeBSD an…

    b749800

    pythongh-130567: Enable previously skipped locale tests on FreeBSD an…

    0660d22
  16. encukou
    added a commit that references this issue on Sep 30, 2025

    [3.13] gh-130567: Enable previously skipped locale tests on FreeBSD a…

    0ef302e
  17. encukou
    added 2 commits that reference this issue on Oct 7, 2025

    [3.14] gh-130567: Fix possible crash in locale.strxfrm() (GH-138940) (G…

    fff2650

    [3.14] gh-130567: Enable previously skipped locale tests on FreeBSD a…

    f0f0566
  18. encukou
    added a commit that references this issue on Oct 16, 2025

    gh-130567: Remove optimistic allocation in locale.strxfrm() (GH-137143)

    2a2bc82
  19. encukou
    closed this as completedon Oct 16, 2025
  20. github-project-automation
    moved this from Todo to Done in Locale issues 🗺on Oct 16, 2025
  21. StanFromIreland
    added a commit that references this issue on Dec 6, 2025

    pythongh-130567: Remove optimistic allocation in locale.strxfrm() (py…

    6b3af77
  22. wkentaro
    added a commit that references this issue on Mar 28, 2026

    fix: handle natsort.os_sorted OSError on macOS with Unicode paths

    cf0b371
  23. StanFromIreland
    marked locale.strxfrm() can fail on FreeBSD #149603 as a duplicate of this issue on May 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    OS-macextension-modulesC modules in the Modules dirtype-bugAn unexpected behavior, bug, or error

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions