Skip to content

Reuse stored hashes when building a set from a set/frozenset/dict - #8491

Draft
fregataa wants to merge 1 commit into
RustPython:mainfrom
fregataa:reuse-stored-hashes-in-set-ops
Draft

Reuse stored hashes when building a set from a set/frozenset/dict#8491
fregataa wants to merge 1 commit into
RustPython:mainfrom
fregataa:reuse-stored-hashes-in-set-ops

Conversation

@fregataa

@fregataa fregataa commented Aug 11, 2026

Copy link
Copy Markdown

Closes #8489.

Changes

dict_inner.rs — each entry point computed the hash on its first line and passed only the hash variable downward, so the split is mechanical: the body moves to a *_with_hash variant and the original becomes a wrapper. No logic is duplicated.

  • insert_with_hash, contains_with_hash, remove_if_with_hash, delete_if_exists_with_hash, delete_or_insert_with_hash
  • keys_with_hashes() — yields (key, hash) from the entries, next to the existing keys()/values()/items()

These mirror CPython's _key/_entry pairs (set_add_key/set_add_entry, set_contains_key/set_contains_entry).

function/protocol.rsArgIterable::as_object() returns the object before any __iter__ call. The set operations take ArgIterable, which already held the original PyObjectRef in a private field; exposing it lets them dispatch on the source type without changing a single public signature (they are passed around as fn(&PySetInner, ArgIterable, &VirtualMachine) -> ... function pointers, so changing them would have rippled widely).

builtins/set.rs

  • PySetInner::cached_hashes()Some(Vec<(key, hash)>) for a set/frozenset (subclasses included, via the existing extract_set) or an exact dict, None otherwise. Same predicates as CPython.
  • add_with_hash / contains_with_hash
  • merge_set / merge_dict now forward the stored hashes
  • Fast path added to union, intersection, difference, symmetric_difference, difference_update, symmetric_difference_update, intersection_updatethe existing generic loop stays as the fallback in each
  • PyFrozenSet's Constructor::Args becomes OptionalArg<PyObjectRef> so py_new receives the source object and can hand its stored hashes over via PySetInner::from_object, instead of flattening to Vec<PyObjectRef> first; slot_new keeps only argument parsing and the empty-singleton check

Result

after dict.fromkeys(map):  10
after frozenset(d):        10
after set(d):              10
after s.difference(d):     10
after s.symmetric_difference_update(d): 10

Testing

  • test_setRan 630 tests ... OK (expected failures=10), no new failures
  • test_dict, test_dictviews, test_dictcomps, test_ordered_dict, test_defaultdict, test_userdict, test_setcomps, test_weakset, test_collections, test_copy — all pass
  • test_types, test_builtin, test_iter, test_pickle, test_marshal, test_descr, test_class, test_functools, test_typing — all pass
  • extra_tests/snippets/: builtin_set.py, builtin_dict.py, builtin_dict_union.py, frozen.py
  • Ad-hoc: empty-frozenset singleton, frozenset(f) is f, distinct ids for empty subclass instances, dict subclasses correctly excluded from the exact-dict path, self-referential ops (s.update(s), s.symmetric_difference_update(s)), results not aliasing their operands, error messages unchanged vs CPython 3.14.6
  • cargo fmt --check clean, cargo clippy -p rustpython-vm --all-targets no new warnings

Review note

insert_with_hash depends on a contract the callee cannot verify: pass a hash that isn't key.key_hash(vm) and the entry lands in a bucket no lookup will probe, so the key silently disappears. This is documented on the function, and every call site forwards a hash that came from keys_with_hashes() on a container holding that same key object. Worth a careful look.

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro Plus

Run ID: 775aec25-74c4-48e7-b4e4-5a7111b1b961

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

set and frozenset recomputed __hash__ for every element even when the
source object already stored a hash per entry. CPython's
set_update_internal branches on PyAnySet_Check / PyDict_CheckExact and
feeds set_add_entry the hash read from the source table; RustPython
always iterated generically.

Split the hash computation out of the Dict entry points so callers can
supply a hash they already hold, add keys_with_hashes() to hand out
(key, hash) pairs, and take the fast path in the set constructors and
in the set operations whose argument is a set/frozenset/exact dict.

ArgIterable::as_object() exposes the pre-__iter__ object so the set
operations can dispatch on the source type without changing any of
their signatures.

Closes RustPython#8489. dict.fromkeys() is the dict-target counterpart and is
tracked in RustPython#8490, so test_do_not_rehash_dict_keys keeps its
expectedFailure marker until that lands too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@fregataa
fregataa force-pushed the reuse-stored-hashes-in-set-ops branch from ba69646 to 2924f86 Compare August 11, 2026 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

set/frozenset re-hash elements when constructed from a set, frozenset, or dict

1 participant