Skip to content

fix: validate non-BaseModel schemas in validate_schema - #7285

Open
chelsealong wants to merge 2 commits into
google:mainfrom
chelsealong:fix-validate-schema-non-basemodel-7280
Open

chelsealong wants to merge 2 commits into
google:mainfrom
chelsealong:fix-validate-schema-non-basemodel-7280

Conversation

@chelsealong

Copy link
Copy Markdown
Contributor

Summary

Fixes #7280.

validate_schema() (src/google/adk/utils/_schema_utils.py) only ran real
validation for BaseModel and list[BaseModel] schemas. Every other
supported SchemaType — generic aliases like list[str] / dict[str, int],
raw JSON Schema dicts, and types.Schema — fell through to a bare
json.loads with no validation, so malformed values were silently accepted:

validate_schema(list[str], "[1, 2, 3]")        # -> [1, 2, 3]  (should reject)
validate_schema(dict[str, int], '{"a": "x"}')  # -> {'a': 'x'} (should reject)
validate_schema(
    {"type": "object", "properties": {"a": {"type": "string"}}, "required": ["a"]},
    '{"a": 1}',
)  # -> {'a': 1} (should reject)

This matters because output_schema accepts all of these types, and an
unvalidated result gets written straight into session state
(output_key) or returned to a parent agent via AgentTool.

Fix

  • Generic alias schemas (list[str], dict[str, int], etc.) now go through
    TypeAdapter(schema).validate_json(...), the same mechanism already used
    for the list[BaseModel] branch.
  • Raw JSON Schema dicts and types.Schema instances are now validated with
    jsonschema.validate (jsonschema is already a direct dependency, and
    already used elsewhere in the codebase, e.g.
    cli/built_in_agents/tools/write_config_files.py). types.Schema uses
    uppercase type names ("STRING", "OBJECT"), so it's dumped to a JSON
    Schema dict and passed through the existing lowercase_schema_types
    helper before validation.
  • types.Schema.model_dump() is called with by_alias=True. types.Schema's
    Python fields are snake_case (any_of, additional_properties,
    max_items, ...) with JSON Schema keyword aliases (anyOf,
    additionalProperties, maxItems, ...). Without by_alias=True,
    model_dump() emitted the snake_case names, which jsonschema.validate
    silently ignores as unknown properties — so any_of, additionalProperties,
    and the min/max constraints were never actually enforced, even though the
    rest of the schema validated correctly. This was caught in review; added
    regression tests for any_of and additional_properties below.

No changes to pyproject.toml / lockfiles — jsonschema was already a
declared dependency.

Test plan

Added 8 tests to tests/unittests/utils/test_schema_utils.py::TestValidateSchema
covering the reject case for each affected schema kind (list[str],
dict[str, int], raw JSON Schema dict, types.Schema), the accept case
for the two new schema kinds that weren't exercised before, and two
aliased-field regression cases (any_of, additional_properties) on
types.Schema.

Confirmed the new reject-case tests fail on the pre-fix code
(git stash the source file, keep the tests, run again):

$ git stash push -- src/google/adk/utils/_schema_utils.py
$ pytest tests/unittests/utils/test_schema_utils.py -q
...
FAILED ...::test_list_of_str_schema_rejects_wrong_element_type - Failed: DID NOT RAISE ValidationError
FAILED ...::test_dict_schema_rejects_wrong_value_type - Failed: DID NOT RAISE ValidationError
FAILED ...::test_raw_json_schema_dict_rejects_invalid_value - Failed: DID NOT RAISE ValidationError
FAILED ...::test_types_schema_rejects_invalid_value - Failed: DID NOT RAISE ValidationError
FAILED ...::test_types_schema_any_of_rejects_invalid_value - Failed: DID NOT RAISE ValidationError
FAILED ...::test_types_schema_additional_properties_rejects_extra_field - Failed: DID NOT RAISE ValidationError
6 failed, 65 passed
$ git stash pop

With the fix restored:

$ pytest tests/unittests/utils/test_schema_utils.py -q
.......................................................................    [100%]
71 passed in 0.80s

Also ran the broader suite for callers of validate_schema plus the rest of
tests/unittests/utils/, all green:

$ pytest tests/unittests/agents/test_llm_agent_output_save.py tests/unittests/tools/test_agent_tool.py tests/unittests/utils/ -q
552 passed, 209 warnings in 7.31s

pre-commit run passes on the changed files (ruff, isort, pyink, addlicense,
codespell). mypy reports the same 6 pre-existing errors in this file with
and without the change (unrelated to this fix, present on unmodified main).

AI disclosure

This change was written with the assistance of Claude Code (Anthropic).

🤖 Generated with Claude Code

validate_schema() only ran Pydantic validation for BaseModel and
list[BaseModel] schemas. Every other supported schema kind (generic
aliases like list[str]/dict[str, int], raw JSON Schema dicts, and
types.Schema) fell through to a bare JSON parse with no validation,
so malformed values (e.g. list[str] receiving [1, 2, 3]) were
silently accepted into session state and tool results.

Generic aliases now go through TypeAdapter.validate_json, matching
the existing list[BaseModel] path. Raw dict schemas and types.Schema
are validated with jsonschema (already a direct dependency), with
types.Schema's uppercase type names lowercased first via the
existing lowercase_schema_types helper.

Fixes google#7280
model_dump() without by_alias=True emitted snake_case field names
(any_of, additional_properties, max_items, etc.) instead of their
JSON Schema keyword aliases (anyOf, additionalProperties, maxItems).
jsonschema.validate() silently ignores unknown keywords, so any
types.Schema using these aliased fields was not actually validated,
reproducing the same "validation appears to run but doesn't" bug the
prior commit fixed for other constraints.

Adds tests covering any_of and additional_properties enforcement.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

validate_schema skips validation for non-BaseModel schemas (list[str], dict[…], raw JSON Schema)

2 participants