Skip to content

builtins: generate accurate __text_signature__ - #8512

Open
leehanjeong wants to merge 6 commits into
RustPython:mainfrom
leehanjeong:8383-text-signature-fix
Open

builtins: generate accurate __text_signature__#8512
leehanjeong wants to merge 6 commits into
RustPython:mainfrom
leehanjeong:8383-text-signature-fix

Conversation

@leehanjeong

@leehanjeong leehanjeong commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Problem

RustPython's #[pyfunction]s generate a __text_signature__ that isn't comparable to CPython's. Of the 45 builtin functions RustPython shares with CPython 3.14, none reported an identical inspect.signature() before this PR:

  • 43 carried a phantom module parameter that doesn't exist.
  • None marked their true positional-only parameters as such.
  • 2 (round, sum) emitted a signature string that isn't valid Python, so inspect.signature() raised ValueError: builtin has invalid signature. The same applies to os.pathconf, binascii.b2a_base64 and binascii.b2a_uu in the wider stdlib.

Tools that rely on inspect.signature() (unittest.mock.autospec, pydoc, IDE completion) act on this bad metadata.

Scope

This PR fixes everything reachable by changing crates/derive-impl/src/util.rs's signature generator alone, without touching FromArgs or adding new types. The #[derive(FromArgs)] struct case is deliberately left out, so #8383 stays open after this merges.

What changed

  • Drop the $module marker. CPython's C functions receive the module as __self__, so inspect strips a leading $module. RustPython's #[pyfunction]s take no module argument, so __self__ is always None and there was nothing to strip. The marker surfaced as a parameter that doesn't exist (len(module, /, obj) instead of len(obj)).
  • Mark parameters positional-only. Plain arguments bind through FuncArgs::take_positional, which never consults the keyword map: len(obj=[1, 2]) already raised TypeError, but inspect reported obj as POSITIONAL_OR_KEYWORD. Generated signatures now carry the / marker, except for *args/**kwargs and empty parameter lists, which cannot have one.
  • Give up cleanly on unnamed parameters. Some functions bind through a destructuring pattern (e.g. fn round(RoundArgs { number, ndigits }: RoundArgs, ..)). The generator used to stringify the Rust pattern verbatim, producing text that isn't valid Python. inspect.signature() still raises ValueError for these, but now with no signature found for builtin, matching how CPython reports a builtin it has no signature for, rather than builtin has invalid signature.
  • Rename 9 parameters to match CPython (bin, ord, divmod, setattr, delattr, hasattr, isinstance, issubclass, aiter). All positional-only, so the name is documentation only and renaming doesn't change behavior.
  • Supporting changes. test_module_level_callable_noargs in Lib/test/test_pydoc/test_pydoc.py now passes for real (the phantom module parameter was the cause), so its expectedFailure marker is dropped. The new snippet's "no signature" assertions are guarded to RustPython, since CPython has real Argument Clinic signatures for round/sum.

Before / after

Measured across the 45 builtin functions RustPython and CPython 3.14 share:

exact match ValueError phantom module
before 0 / 45 2 (invalid signature) 43
after 23 / 45 2 (no signature found) 0

The 22 that still differ fall into two groups that need different work.

CPython documents a signature we could match (13). Twelve of these hold their arguments in a type the signature generator can't see into: a #[derive(FromArgs)] struct (__import__, compile, eval, exec, open, pow, print, sorted, and round/sum, which now report no signature at all) or an OptionalArg whose default lives in the function body (format, input). The thirteenth, breakpoint, differs only because CPython names its keyword catch-all **kws and the generator hardcodes **kwargs for every FuncArgs function.

CPython has no signature at all, and RustPython reports one (9). __build_class__, anext, dir, getattr, iter, max, min, next and vars. Teaching FromArgs to report its parameters would not close this gap, since the generated signature is not what's wrong: matching CPython here means deciding to suppress a signature we are able to produce. test_autospec_on_bound_builtin_function stays expectedFailure for exactly this reason, via time.ctime.

Not in this PR

Two follow-ups would close most of the first group above:

  • FromArgs reporting its own parameters. Implementors would report the parameters they consume, mirroring how arity() already self-reports parameter count. The information is already there in each field's #[pyarg(...)] attribute; it just isn't reachable from the signature generator today.
  • OptionalArg's hidden defaults, which the same mechanism would have to carry.

The second group needs a separate decision about whether RustPython should suppress signatures CPython doesn't publish, which seems worth settling before either follow-up.

Two unrelated problems turned up while investigating, both out of scope here:

  • __doc__ carries the raw signature prefix. len.__doc__ is 'len(obj, /)\n--\n\nReturn the number of items in a container.'. get_doc_from_internal_doc (type.rs) strips this, but is only wired to PyType.__doc__, not to builtin_func.rs or descriptor.rs.
  • Methods have almost no signatures. pyclass.rs only attaches a generated signature when the method already has a doc comment, so list.append, str.split and dict.get all report __text_signature__ of None. This PR touches the #[pymethod] path only enough to keep it compiling.

Test plan

  • New extra_tests/snippets/builtin_signature.py, run under both CPython and RustPython.
  • cargo run --release -- -m test test_inspect test_pydoc test_unittest
  • cargo test -p rustpython-derive-impl
  • prek run --all-files

Developed with assistance from Claude Code (claude-opus-5)

Summary by CodeRabbit

  • Bug Fixes

    • Improved builtin function signatures for clearer, more accurate display.
    • Removed phantom module parameters from displayed signatures.
    • Added correct positional-only and variadic parameter formatting.
    • Omitted signatures when they cannot be generated reliably.
    • Preserved documentation accurately when signatures are unavailable or incomplete.
  • Tests

    • Added coverage for builtin signature formatting, parameter names, and edge cases.

CPython's C functions receive the module as their first argument, so
PyCFunction.__self__ is the module and inspect strips the $module
parameter when building a Signature. A #[pyfunction] takes no such
argument, PyNativeFunction::zelf is None, and inspect has nothing to
strip, so the marker surfaced as a parameter that does not exist:

    inspect.signature(len)
    (module, /, obj)     # was
    (obj)                # now

All 45 builtins shared with CPython carried it. Methods are unaffected;
their $self marker comes from func_sig and both branches now produce the
same string.

Assisted-by: Claude Code:claude-opus-5
Arguments bind through `FuncArgs::take_positional`, which pops from the
positional list and never consults the keyword map, so a #[pyfunction]
argument cannot be passed by name:

    >>> len(obj=[1, 2])
    TypeError

The generated signature omitted the `/` marker, so inspect reported those
parameters as POSITIONAL_OR_KEYWORD, contradicting the call above. Emit
the marker, except for `*args`/`**kwargs`, which cannot be followed by
`/`, and for empty parameter lists.

14 of the 45 builtins shared with CPython now report an identical
signature, up from 0.

Assisted-by: Claude Code:claude-opus-5
Arguments bound by a destructuring pattern, e.g.

    fn round(RoundArgs { number, ndigits }: RoundArgs, ..)

have no name to report, and func_sig stringified the pattern verbatim:

    >>> round.__text_signature__
    '($module, RoundArgs { number, ndigits })'

That is not valid Python, so inspect.signature() raised "builtin has
invalid signature". Return None instead, which leaves
__text_signature__ unset and makes inspect raise "no signature found",
the same as for a CPython builtin that has no signature.

Affects round, sum, os.pathconf, binascii.b2a_base64 and
binascii.b2a_uu. Their docstrings are unchanged; only the signature
prefix is dropped.

Assisted-by: Claude Code:claude-opus-5
These parameters are positional-only, so their names only ever appear in
__text_signature__ and cannot be used at a call site. Naming them after
CPython makes the generated signatures directly comparable:

    bin        x            -> number
    ord        string       -> character
    divmod     a, b         -> x, y
    setattr    attr         -> name
    delattr    attr         -> name
    hasattr    attr         -> name
    isinstance typ          -> class_or_tuple
    issubclass subclass,typ -> cls, class_or_tuple
    aiter      iter_target  -> async_iterable

23 of the 45 builtins shared with CPython now report an identical
signature, up from 0 before this branch. The remainder need FromArgs to
report the parameters of its own structs, which is left for a follow-up.

Add extra_tests/snippets/builtin_signature.py covering the phantom
module parameter, the positional-only marker, the names above, and the
signature-less builtins.

Assisted-by: Claude Code:claude-opus-5
pydoc's summary line for time.time was "time(module)" because the
generated signature carried a $module parameter that inspect could not
strip. It now reads "time()", as the test expects.

Assisted-by: Claude Code:claude-opus-5
test_snippets runs every snippet under CPython as well, and CPython does
have Argument Clinic signatures for round and sum, so that block only
holds for RustPython.

Assisted-by: Claude Code:claude-opus-5
@coderabbitai

coderabbitai Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro Plus

Run ID: 7bd94d27-47bf-458b-8358-1e41ec17bafe

📥 Commits

Reviewing files that changed from the base of the PR and between 525ba8c and 4e41ace.

⛔ Files ignored due to path filters (1)
  • Lib/test/test_pydoc/test_pydoc.py is excluded by !Lib/**
📒 Files selected for processing (5)
  • crates/derive-impl/src/pyclass.rs
  • crates/derive-impl/src/pymodule.rs
  • crates/derive-impl/src/util.rs
  • crates/vm/src/stdlib/builtins.rs
  • extra_tests/snippets/builtin_signature.py

📝 Walkthrough

Walkthrough

Builtin signature generation now supports missing signatures and positional-only formatting. Method generation now preserves available documentation and emits no documentation when none exists. Builtin parameter names and signature tests were updated.

Changes

Builtin signature and documentation handling

Layer / File(s) Summary
Optional builtin signature generation
crates/derive-impl/src/util.rs, crates/vm/src/stdlib/builtins.rs, extra_tests/snippets/builtin_signature.py
func_sig and text_signature now return optional signatures, omit $module, add positional-only syntax, and reject destructuring patterns. Builtin parameter names and signature assertions were updated.
Optional method documentation
crates/derive-impl/src/pyclass.rs, crates/derive-impl/src/pymodule.rs
Generated methods now preserve signature-only or source-only documentation and emit None when no documentation exists.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Mergeability Score: ⚪ Minimal · up to 4e41a

This PR improves builtin signature metadata and related tests without any identified merge-blocking risk; it is merge-ready after normal checks and review.

Suggested reviewers: youknowone

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely summarizes the main change: generating more accurate builtin text_signature metadata.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

[ ] test: cpython/Lib/test/test_set.py (TODO: 4)

dependencies:

dependent tests: (no tests depend on set)

[x] test: cpython/Lib/test/test_marshal.py (TODO: 15)

dependencies:

dependent tests: (25 tests)

  • marshal: test_bool test_exceptions test_importlib test_inspect test_marshal test_zipimport
    • importlib._bootstrap_external: test_importlib test_unittest
      • modulefinder: test_importlib test_modulefinder
      • py_compile: test_argparse test_cmd_line_script test_compileall test_importlib test_multiprocessing_main_handling test_py_compile test_pydoc test_runpy
      • pydoc: test_enum
    • pkgutil: test_pkgutil test_pyrepl
    • profile: test_profile
    • pstats: test_pstats
    • zipimport: test_importlib test_zipimport_support

[ ] lib: cpython/Lib/typing.py
[ ] test: cpython/Lib/test/test_typing.py (TODO: 2)
[x] test: cpython/Lib/test/test_type_aliases.py
[x] test: cpython/Lib/test/test_type_annotations.py (TODO: 1)
[ ] test: cpython/Lib/test/test_type_params.py (TODO: 1)
[x] test: cpython/Lib/test/test_genericalias.py

dependencies:

  • typing (native: _typing, collections.abc, sys)
    • collections (native: _collections, _weakref, itertools, sys)
    • inspect (native: builtins, collections.abc, importlib.machinery, itertools, sys)
    • warnings (native: _contextvars, _thread, _warnings, builtins, sys)
    • abc, annotationlib, contextlib, copyreg, functools, operator, re, types

dependent tests: (19 tests)

  • typing: test_annotationlib test_builtin test_copy test_enum test_fractions test_funcattrs test_functools test_genericalias test_grammar test_inspect test_isinstance test_patma test_peg_generator test_pydoc test_pyrepl test_type_aliases test_type_params test_types test_typing

[ ] lib: cpython/Lib/pydoc.py
[ ] lib: cpython/Lib/pydoc_data
[ ] test: cpython/Lib/test/test_pydoc (TODO: 31)

dependencies:

  • pydoc (native: _pyrepl.pager, builtins, email.message, http.server, importlib._bootstrap, importlib._bootstrap_external, importlib.machinery, importlib.util, pydoc_data.topics, select, sys, time, urllib.parse)
    • pydoc_data
    • io (native: _io, _thread, errno, msvcrt, sys)
    • platform (native: _wmi, itertools, java.lang, sys, vms_lib, winreg)
    • pydoc_data
    • sysconfig (native: _sysconfig, _winapi, importlib.machinery, importlib.util, os.path, sys)
    • collections, inspect, warnings
    • future, annotationlib, ast, getopt, os, pkgutil, re, reprlib, textwrap, threading, tokenize, traceback, webbrowser

dependent tests: (5 tests)

  • pydoc: test_enum test_pydoc
    • pdb: test_pdb
    • xmlrpc.server: test_docxmlrpc test_xmlrpc

[x] test: cpython/Lib/test/test_itertools.py (TODO: 6)

dependencies:

dependent tests: (56 tests)

  • itertools: test_annotationlib test_ast test_asyncio test_bdb test_buffer test_builtin test_call test_codeccallbacks test_collections test_compile test_concurrent_futures test_csv test_ctypes test_descr test_dis test_email test_exceptions test_functools test_genericalias test_hashlib test_heapq test_httplib test_importlib test_inspect test_io test_iterlen test_itertools test_launcher test_logging test_math test_memoryview test_mmap test_os test_peepholer test_platform test_pprint test_pyrepl test_queue test_range test_set test_shlex test_slice test_socket test_sort test_statistics test_str test_struct test_subprocess test_tokenize test_tuple test_typing test_unittest test_uuid test_winreg test_xml_etree test_zipfile

Legend:

  • [+] path exists in CPython
  • [x] up-to-date, [ ] outdated

@ShaharNaveh

Copy link
Copy Markdown
Contributor

@moreal the OSCCA job is failing

@youknowone youknowone added the z-ca-2026 Tag to track Contribution Academy 2026 label Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

z-ca-2026 Tag to track Contribution Academy 2026

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants