Skip to content

gh-153229: Optimize dict.fromkeys() fast-clone path for combined dicts#153234

Open
rajat315315 wants to merge 3 commits into
python:mainfrom
rajat315315:dict_fromkeys_opt
Open

gh-153229: Optimize dict.fromkeys() fast-clone path for combined dicts#153234
rajat315315 wants to merge 3 commits into
python:mainfrom
rajat315315:dict_fromkeys_opt

Conversation

@rajat315315

Copy link
Copy Markdown

Summary

Optimize dict.fromkeys(d, val) when d is a combined, compact dict by cloning the source PyDictKeysObject in one memcpy instead of inserting each key individually.

Fixes gh-153229

Motivation

Previously dict_dict_fromkeys() always called insertdict() for every key in the source, performing a full hash probe on the (empty) target dict N times. This is wasteful: the target is always empty, so every probe is a guaranteed miss.

dict.copy() and {}.update(other) already use clone_combined_dict_keys() to fast-clone the PyDictKeysObject in one shot. dict.fromkeys() lacked an equivalent path.

Changes

Objects/dictobject.c

  • Added clone_combined_dict_keys_with_value(orig, fill_value): a variant of clone_combined_dict_keys() that replaces every active entry's value with a new reference to fill_value instead of copying from the source.

  • Added fast path in dict_dict_fromkeys(): when the source dict satisfies:

    • ma_values == NULL (combined, not split)
    • ma_used == dk_nentries (compact, no deleted entries)
    • Is a plain dict (not a subclass with custom tp_iter)
    • Source is non-empty

    …call clone_combined_dict_keys_with_value() and install the resulting keys object directly, bypassing the insertdict loop entirely.

Benchmark

1000-key string dict, 10,000 iterations of dict.fromkeys(d, None), Python 3.16.0a0, GCC 15.2.0:

Benchmark Before After Speedup
dict.fromkeys (1000 str keys) ~423 ms ~90 ms +79%

Tests

./python -m test test_dict test_dictviews
== Tests result: SUCCESS ==   (157 tests passed)

…d dicts

When the source iterable is a combined, compact dict (no deleted entries),
skip the per-key insertdict() loop entirely. Instead, clone the source
PyDictKeysObject in one memcpy and replace every active entry's value
with a new reference to the fill value.

This avoids O(n) hash lookups and collision probing on the target dict.

Benchmark (1000-key string dict, 10k iterations):
  Before: ~365-423 ms
  After:  ~90 ms  (+~79% faster)

All test_dict and test_dictviews tests pass.
@bedevere-app

bedevere-app Bot commented Jul 6, 2026

Copy link
Copy Markdown

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@rajat315315 rajat315315 changed the title gh-153199: Optimize dict.fromkeys() fast-clone path for combined dicts gh-153229: Optimize dict.fromkeys() fast-clone path for combined dicts Jul 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Optimize dict.fromkeys() fast-clone path for combined dicts

1 participant