Skip to content

Private APIs used in Pyodide #79

Open

Description

@hoodmane

Here is the list of the private C APIs used by Pyodide. I tried to drop ones that have public equivalents on the main branch, but I may have accidentally kept some.

cc @vstinner

Argument parsing

  • _PyArg_CheckPositional -- argument clinic uses it, probably should be public?

Argument parsing for vectorcall, would be nice to have argument clinic variant
for these I guess?

  • _PyArg_Parser
  • _PyArg_ParseStack
  • _PyArg_ParseStackAndKeywords

Errors and tracebacks

  • _Py_DumpTraceback -- used for fatal errors
  • _PyTraceback_Add -- add JS frames to a Python traceback
  • _PyErr_FormatFromCause -- Chain exceptions conveniently

Generators

  • _PyGen_FetchStopIterationValue -- used to consume generators
  • _PyGen_SetStopIterationValue -- used to implement generators

Hashes

  • _Py_HashBytes -- use Python utility function for hashing my stuff rather than reimplement
  • _PyDict_GetItem_KnownHash -- several lookups in a row on same key

Numbers

  • _PyLong_AsByteArray -- For converting a Python int to a JS bigint
  • _PyLong_FromByteArray -- For converting a JS bigint to a Python int
  • _PyLong_NumBits -- Work out how much space to allocate for the bytearray passed to PyLong_FromByteArray
  • _PyNumber_Index -- much more common in internal Python code than
    PyNumber_Index. PyNumber_Index forces the return type to be exactly PyLong
    rather than a subtype. We can handle subtypes so we use the underscore version.

Other data structures

  • _PySet_Update
  • _PyUnicode_EQ

Miscellaneous

  • _PyObject_GetMethod -- caching for lookups on Python objects from JavaScript
  • _PyObject_NextNotImplemented -- checking if a Python object supports
    iteration, could be replaced with PyIter_Check

PyID functions

I could easily shim these; Pyodide is single threaded so they are nice.

  • _Py_IDENTIFIER
  • _PyObject_CallMethodIdNoArgs
  • _PyObject_CallMethodIdObjArgs
  • _PyObject_CallMethodIdOneArg
  • _PyObject_GetAttrId
  • _PyObject_SetAttrId
  • _PyUnicode_FromId

Activity

  1. vstinner commented on Jul 20, 2024

    @vstinner
    Contributor

    _Py_HashBytes -- use Python utility function for hashing my stuff rather than reimplement

    The API was approved but not implemented: capi-workgroup/decisions#13

  2. encukou commented on Jul 22, 2024

    @encukou
    Contributor

    Some of these should have a public alternative.
    (Maybe already evaluated some of these and found a reason to not use them?)

    _PyArg_CheckPositional

    Argument Clinic is private itself. But, if you run it with --limited, it'll keep to the limited (and thus public) API. (And the replacement for _PyArg_CheckPositional even hardcodes some things that the function determines at runtime!)

    _PyLong_AsByteArray / _PyLong_FromByteArray / _PyLong_NumBits

    There is now PyLong_AsNativeBytes/PyLong_FromNativeBytes.
    The As also serve the _PyLong_NumBits use case (and if you can optimistically give it a small buffer, only huge integers will need two calls)

    _PyNumber_Index

    You can use tp_as_number->nb_index, and type-check the result yourself.
    Note that returning int subtypes from nb_index (or __index__) is deprecated.

    _PySet_Update

    Call PyNumber_InPlaceOr, or tp_as_number->nb_inplace_or. (Assuming you don't mind that subclasses can override this.)

  3. vstinner commented on Sep 25, 2024

    @vstinner
    Contributor

    _PyUnicode_EQ

    I created python/cpython#124502 to add PyUnicode_Equal() function.

  4. vstinner commented on Sep 25, 2024

    @vstinner
    Contributor

    _Py_HashBytes -- use Python utility function for hashing my stuff rather than reimplement

    Py_HashBuffer() was added to Python 3.14: https://docs.python.org/dev/c-api/hash.html#c.Py_HashBuffer

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions