gh-153229: Optimize dict.fromkeys() fast-clone path for combined dicts#153234
Open
rajat315315 wants to merge 3 commits into
Open
gh-153229: Optimize dict.fromkeys() fast-clone path for combined dicts#153234rajat315315 wants to merge 3 commits into
rajat315315 wants to merge 3 commits into
Conversation
…d dicts When the source iterable is a combined, compact dict (no deleted entries), skip the per-key insertdict() loop entirely. Instead, clone the source PyDictKeysObject in one memcpy and replace every active entry's value with a new reference to the fill value. This avoids O(n) hash lookups and collision probing on the target dict. Benchmark (1000-key string dict, 10k iterations): Before: ~365-423 ms After: ~90 ms (+~79% faster) All test_dict and test_dictviews tests pass.
|
Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool. If this change has little impact on Python users, wait for a maintainer to apply the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Optimize
dict.fromkeys(d, val)whendis a combined, compact dict by cloning the sourcePyDictKeysObjectin onememcpyinstead of inserting each key individually.Fixes gh-153229
Motivation
Previously
dict_dict_fromkeys()always calledinsertdict()for every key in the source, performing a full hash probe on the (empty) target dict N times. This is wasteful: the target is always empty, so every probe is a guaranteed miss.dict.copy()and{}.update(other)already useclone_combined_dict_keys()to fast-clone thePyDictKeysObjectin one shot.dict.fromkeys()lacked an equivalent path.Changes
Objects/dictobject.cAdded
clone_combined_dict_keys_with_value(orig, fill_value): a variant ofclone_combined_dict_keys()that replaces every active entry's value with a new reference tofill_valueinstead of copying from the source.Added fast path in
dict_dict_fromkeys(): when the source dict satisfies:ma_values == NULL(combined, not split)ma_used == dk_nentries(compact, no deleted entries)dict(not a subclass with customtp_iter)…call
clone_combined_dict_keys_with_value()and install the resulting keys object directly, bypassing theinsertdictloop entirely.Benchmark
1000-key string dict, 10,000 iterations of
dict.fromkeys(d, None), Python 3.16.0a0, GCC 15.2.0:dict.fromkeys(1000 str keys)Tests