Skip to content

[FEATURE] Fan out JSON() structured values in SELECT WHERE clauses through query param transposition (AWS query/ec2 protocols) #700

Closed

Description

@jeffreyaven

Summary

A required structured parameter supplied as JSON('{...}') in a SELECT WHERE clause is dropped at parameter binding time and never reaches the request. The AWSCanonical transpose fan-out (dotted, .N-indexed form keys) already works for INSERT values and UPDATE SET; SELECT WHERE bypasses it, so any AWS query/ec2 operation with a required structure parameter cannot be called via SELECT.

Repro

Using the AWS provider (aws.ec2.instance_types, method get_instance_types_from_instance_requirements, ec2 protocol, queryParamTranspose: AWSCanonical + requestTranslate: get_query_to_post_form_utf_8):

SELECT instance_type
FROM aws.ec2.instance_types
WHERE region = 'us-east-1'
AND ArchitectureType = 'x86_64'
AND VirtualizationType = 'hvm'
AND InstanceRequirements = JSON('{"VCpuCount":{"Min":2,"Max":8},"MemoryMiB":{"Min":4096}}');

Routing selects the correct method (the scalar params serialize fine), but AWS returns:

http response status code: 400
<Code>MissingParameter</Code>
<Message>The request must include the InstanceRequirements parameter. Add the required parameter and retry the request.</Message>

The InstanceRequirements.* keys are absent from the outgoing form body.

Expected wire shape

The botocore query serialization for this operation, expressed as curl:

curl "https://ec2.us-east-1.amazonaws.com/" \
  --user "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY" \
  --aws-sigv4 "aws:amz:us-east-1:ec2" \
  -H "Content-Type: application/x-www-form-urlencoded; charset=utf-8" \
  -d 'Action=GetInstanceTypesFromInstanceRequirements' \
  -d 'Version=2016-11-15' \
  -d 'ArchitectureType.1=x86_64' \
  -d 'VirtualizationType.1=hvm' \
  -d 'InstanceRequirements.VCpuCount.Min=2' \
  -d 'InstanceRequirements.VCpuCount.Max=8' \
  -d 'InstanceRequirements.MemoryMiB.Min=4096'

Root cause

The any-sdk side is already complete - no any-sdk changes needed:

  • pkg/querytranspose/aws_canonical_querytranspose.go recursively expands maps/slices/scalars into the dotted .N-indexed form.
  • internal/anysdk/http.go processFuncHTTPParam() detects a *sqlparser.FuncExpr named JSON in a query param binding and runs the transposer; it is invoked from both ToFlatMap() and GetRemainingQueryParamsFlatMap(), and parameterize() in internal/anysdk/operation_store.go carries the fanned-out keys into the URL query, which get_query_to_post_form_utf_8 then moves into the signed form body.

The drop is in stackql core:

  • The WHERE visitor captures col = FuncExpr comparisons into parameter metadata (internal/stackql/astvisit/param_extract.go, ComparisonExpr -> *sqlparser.FuncExpr case), which is why routing works.
  • But when the SELECT annotation context builds the armoury params (internal/stackql/taxonomy/annotation_context.go, the WithGetHTTPArmoury closure), it first calls util.TransformSQLRawParameters() - and that function explicitly discards every FuncExpr value (internal/stackql/util/plan_utility.go, case *sqlparser.FuncExpr: log; continue, both bare and wrapped in ParameterMetadata). The value never reaches the stream that feeds NewHTTPPreparator.

Proposed fix

In TransformSQLRawParameters(), pass FuncExprs named JSON through as-is (keep the *sqlparser.FuncExpr as the param value) instead of dropping them. any-sdk imports the same sqlparser types and handles the rest.

Caveats:

  1. Scope the pass-through to JSON() only - other FuncExprs in WHERE (e.g. strftime) are evaluated via the feat: mcp credential (re)sourcing via server.env_file + reload_credentials tool (#688) #689 path and must not regress.
  2. After the request succeeds, the local SQLite re-filter may evaluate InstanceRequirements = JSON('...') against the parameter column and filter out every row unless the param value is echoed into rows (the [FEATURE] SQL functions in WHERE clause not evaluated for required parameters (start_time), and projected parameter/response columns return null #686 echo behavior) or the predicate is neutralized for consumed params - likely a second latent bug behind the first.
  3. DELETE WHERE / EXEC params pass JSON strings verbatim ([FEATURE] Extended support for batch mutation operations in the planner #683) - a separate code path, not fixed by this.

Related

Activity

  1. self-assigned this
    on Jul 29, 2026
  2. changed the title [-][FEATURE] [FEATURE] Fan out JSON() structured values in SELECT WHERE clauses through query param transposition (AWS query/ec2 protocols)[/-] [+][FEATURE] Fan out JSON() structured values in SELECT WHERE clauses through query param transposition (AWS query/ec2 protocols)[/+] on Jul 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions