Skip to content

extmod/moductypes: Follow the attr protocol for unknown attributes.#19345

Open
cnadler86 wants to merge 1 commit into
micropython:masterfrom
cnadler86:fix/uctypes-attr-protocol
Open

extmod/moductypes: Follow the attr protocol for unknown attributes.#19345
cnadler86 wants to merge 1 commit into
micropython:masterfrom
cnadler86:fix/uctypes-attr-protocol

Conversation

@cnadler86

@cnadler86 cnadler86 commented Jun 16, 2026

Copy link
Copy Markdown

extmod/moductypes: Make the attr handler follow the native attr protocol

Summary

The attr handler of uctypes.struct (uctypes_struct_attr in extmod/moductypes.c)
doesn't follow the contract every other native attr handler honours: for an attribute
that is not a field of the struct it raises KeyError (via mp_obj_dict_get) instead
of leaving the result slot untouched so the caller can decide what to do.

This is invisible on a plain uctypes.struct, but leaks out as soon as a Python class
subclasses a native type with an attr handler — the scenario behind #18592.

The native attr protocol

mp_store_attr and mp_load_method_maybe (py/runtime.c) define the contract. The handler
gets a 2-element dest array:

  • Store/deletedest = { MP_OBJ_SENTINEL, value }:
    • owns the attribute → store it and set dest[0] = MP_OBJ_NULL (success);
    • does not own it → leave dest[0] == MP_OBJ_SENTINEL (caller raises AttributeError,
      or for a subclass falls back to instance __dict__);
    • operation genuinely invalid (e.g. wrong value type) → raise.
  • Loaddest[0] == MP_OBJ_NULL: found → set dest[0]; not found → leave it NULL.

The key point: "attribute not recognised" is signalled in-band (sentinel / NULL), not via an
exception — exceptions are reserved for genuine errors. mp_obj_exception_attr,
complex_attr, range_attr, etc. all follow this.

What uctypes does instead

mp_obj_t deref = mp_obj_dict_get(self->desc, MP_OBJ_NEW_QSTR(attr));  // raises KeyError

For an unknown attribute this raises KeyError, conflating "not one of my fields" with a
genuine error.

Why it matters

  1. Subclassing (Python subclass of native type with attr handler cannot properly store attributes #18592). Stores on a native subclass must be offered to the native
    handler first (else native fields are shadowed by __dict__ entries). With a conforming
    handler the generic code in py/objtype.c just checks the sentinel and falls through.
    Because uctypes raises, that code is otherwise forced to wrap the call in nlr_push/
    nlr_pop and guess from the exception type whether the attribute was unknown or a real
    failure — which adds an nlr frame to the hot path of every native-subclass store, is a
    heuristic (a setter legitimately failing with KeyError/AttributeError gets
    misread as "unknown" and silently redirected to __dict__), and is asymmetric with the
    load path (which never catches).

  2. Inconsistent error type. A missing attribute on a uctypes.struct raises KeyError,
    whereas every other object (and CPython) raises AttributeError:

import uctypes, array
data = array.array("I", [0])
s = uctypes.struct(uctypes.addressof(data), {"value": uctypes.UINT32 | 0}, uctypes.NATIVE)
s.nonexistent          # -> KeyError      (expected: AttributeError)
s.nonexistent = 1      # -> KeyError      (expected: AttributeError)

Fix

Do a non-raising field lookup and return MP_OBJ_NULL when the field is absent, so the
standard machinery falls back / raises AttributeError:

mp_map_elem_t *e = mp_map_lookup(mp_obj_dict_get_map(self->desc),
                                 MP_OBJ_NEW_QSTR(attr), MP_MAP_LOOKUP);
if (e == NULL) {
    return MP_OBJ_NULL;   // not a field: let the caller fall back / raise AttributeError
}
mp_obj_t deref = e->value;

This lets py/objtype.c (the #18592 fix) drop the nlr_push workaround and just honour the
sentinel, makes store symmetric with load, lets genuine setter errors propagate as in
CPython, and works for any native type with an attr handler — not just uctypes.

Behaviour change

A missing attribute on a uctypes.struct now raises AttributeError instead of KeyError
(load and store). This is more CPython-like and consistent with all other objects; no test
in the tree depends on the old KeyError (checked against tests/extmod/uctypes_*).

The struct attr handler raised KeyError for an attribute that is not a
field, instead of leaving dest[0] as MP_OBJ_SENTINEL to signal "not
handled" like every other native attr handler does (see mp_store_attr).
This leaks a KeyError on a plain struct, and breaks subclassing of native
types (micropython#18592), where a store must be offered to the native handler before
falling back to the instance dict.

Use a non-raising field lookup and return MP_OBJ_NULL when the attribute
is not a field, so the standard machinery falls back or raises
AttributeError.  A missing attribute now raises AttributeError instead of
KeyError, consistent with all other objects and CPython.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Christopher Nadler <christopher.nadler@gmail.com>
@codecov

codecov Bot commented Jun 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.51%. Comparing base (d901e98) to head (3d44761).

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #19345   +/-   ##
=======================================
  Coverage   98.51%   98.51%           
=======================================
  Files         176      176           
  Lines       22903    22907    +4     
=======================================
+ Hits        22562    22566    +4     
  Misses        341      341           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

Copy link
Copy Markdown

Code size report:

Reference:  unix/README: Update the supported targets list. [d901e98]
Comparison: extmod/moductypes: Follow the attr protocol for unknown attributes. [merge of 3d44761]
  mpy-cross:    +0 +0.000% 
   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:   +48 +0.006% standard
      stm32:    +8 +0.002% PYBV10
      esp32:   +12 +0.001% ESP32_GENERIC
     mimxrt:    +8 +0.002% TEENSY40
        rp2:    +8 +0.001% RPI_PICO_W
       samd:    +8 +0.003% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:   +24 +0.005% VIRT_RV32

@dpgeorge

Copy link
Copy Markdown
Member

Do you think you could add a test for this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants