perf: defer building model validators until first use - #2784
Open
dmontagu wants to merge 1 commit into
Open
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
google.genai.types defines ~770 models, and importing the package builds a pydantic validator and serializer for every one of them. Any single application uses a small fraction: a client doing text generation never touches the Live, Batch, Tuning or Caching families. Setting defer_build=True on the shared BaseModel builds each model's validator and serializer the first time that model is actually used instead. Measured on Python 3.12 with google-genai 1.72.0, RSS attributable to `import google.genai` drops from 84.1MB to 78.2MB (-7.0%) and import time from 0.73s to 0.57s (-22%), against a one-off cost of ~5ms for the first model used and ~16ms for the first GenerateContentConfig. Steady-state validation throughput is unchanged.
dmontagu
force-pushed
the
defer-model-build
branch
from
July 30, 2026 15:33
022f428 to
162cdaf
Compare
Author
|
@Venkaiahbabuneelam please feel free to take this over or close it and tweak in any way you like. Obviously it's only a one-line change besides the comment, my point is just I have zero need for attribution or anything like that. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
defer_build=Trueto themodel_configofgoogle.genai._common.BaseModel, so that each model's pydantic validator and serializer is built the first time the model is actually used rather than whengoogle.genaiis imported.google.genai.typesdefines 424 pydantic models (each with aTypedDictmirror), 418 of which get a validator and a serializer built at import time today. Any single application uses a small fraction of them — a client doing text generation never touches the Live, Batch, Tuning or Caching families.Measurements
macOS/arm64,
google-genai1.72.0, releasedpydantic2.13.4 /pydantic-core2.46.4 from PyPI, no pydantic plugins installed. RSS attributable toimport google.genai, measured in-process as anru_maxrssdelta; fresh interpreter per sample, warm bytecode caches, both variants interleaved round-robin, median of 12 samples each:defer_build=Truedefer_build=TrueThe deferred work comes back only for the models actually used, once each (Python 3.12):
defer_build=Truetypes.Part(...)types.Content(...)+types.GenerateContentConfig(...)types.Part(...)content.model_dump(exclude_none=True)So an application pays roughly 13 ms once, at first use, for the model families it actually touches, and keeps most of the memory saving permanently. Validation and serialization throughput are unaffected —
defer_buildonly moves when the validator is built, not what it does.Compatibility
defer_buildis a standard pydanticConfigDictoption, available since pydantic 2.1 (this package requires ≥ 2.9), and it is inherited by everyBaseModelsubclass intypes.py.model_validate,model_dumpandmodel_json_schemaall trigger the build transparently on first use;__pydantic_core_schema__is regenerated on demand if anything asks for it.The full test suite passes with the change.