Repository navigation
[FEATURE] Fan out JSON() structured values in SELECT WHERE clauses through query param transposition (AWS query/ec2 protocols) #700
Description
Summary
A required structured parameter supplied as JSON('{...}') in a SELECT WHERE clause is dropped at parameter binding time and never reaches the request. The AWSCanonical transpose fan-out (dotted, .N-indexed form keys) already works for INSERT values and UPDATE SET; SELECT WHERE bypasses it, so any AWS query/ec2 operation with a required structure parameter cannot be called via SELECT.
Repro
Using the AWS provider (aws.ec2.instance_types, method get_instance_types_from_instance_requirements, ec2 protocol, queryParamTranspose: AWSCanonical + requestTranslate: get_query_to_post_form_utf_8):
SELECT instance_type
FROM aws.ec2.instance_types
WHERE region = 'us-east-1'
AND ArchitectureType = 'x86_64'
AND VirtualizationType = 'hvm'
AND InstanceRequirements = JSON('{"VCpuCount":{"Min":2,"Max":8},"MemoryMiB":{"Min":4096}}');Routing selects the correct method (the scalar params serialize fine), but AWS returns:
http response status code: 400
<Code>MissingParameter</Code>
<Message>The request must include the InstanceRequirements parameter. Add the required parameter and retry the request.</Message>
The InstanceRequirements.* keys are absent from the outgoing form body.
Expected wire shape
The botocore query serialization for this operation, expressed as curl:
curl "https://ec2.us-east-1.amazonaws.com/" \
--user "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY" \
--aws-sigv4 "aws:amz:us-east-1:ec2" \
-H "Content-Type: application/x-www-form-urlencoded; charset=utf-8" \
-d 'Action=GetInstanceTypesFromInstanceRequirements' \
-d 'Version=2016-11-15' \
-d 'ArchitectureType.1=x86_64' \
-d 'VirtualizationType.1=hvm' \
-d 'InstanceRequirements.VCpuCount.Min=2' \
-d 'InstanceRequirements.VCpuCount.Max=8' \
-d 'InstanceRequirements.MemoryMiB.Min=4096'Root cause
The any-sdk side is already complete - no any-sdk changes needed:
pkg/querytranspose/aws_canonical_querytranspose.gorecursively expands maps/slices/scalars into the dotted.N-indexed form.internal/anysdk/http.goprocessFuncHTTPParam()detects a*sqlparser.FuncExprnamedJSONin a query param binding and runs the transposer; it is invoked from bothToFlatMap()andGetRemainingQueryParamsFlatMap(), andparameterize()ininternal/anysdk/operation_store.gocarries the fanned-out keys into the URL query, whichget_query_to_post_form_utf_8then moves into the signed form body.
The drop is in stackql core:
- The WHERE visitor captures
col = FuncExprcomparisons into parameter metadata (internal/stackql/astvisit/param_extract.go,ComparisonExpr->*sqlparser.FuncExprcase), which is why routing works. - But when the SELECT annotation context builds the armoury params (
internal/stackql/taxonomy/annotation_context.go, theWithGetHTTPArmouryclosure), it first callsutil.TransformSQLRawParameters()- and that function explicitly discards every FuncExpr value (internal/stackql/util/plan_utility.go,case *sqlparser.FuncExpr: log; continue, both bare and wrapped inParameterMetadata). The value never reaches the stream that feedsNewHTTPPreparator.
Proposed fix
In TransformSQLRawParameters(), pass FuncExprs named JSON through as-is (keep the *sqlparser.FuncExpr as the param value) instead of dropping them. any-sdk imports the same sqlparser types and handles the rest.
Caveats:
- Scope the pass-through to
JSON()only - other FuncExprs in WHERE (e.g.strftime) are evaluated via the feat: mcp credential (re)sourcing via server.env_file + reload_credentials tool (#688) #689 path and must not regress. - After the request succeeds, the local SQLite re-filter may evaluate
InstanceRequirements = JSON('...')against the parameter column and filter out every row unless the param value is echoed into rows (the [FEATURE] SQL functions in WHERE clause not evaluated for required parameters (start_time), and projected parameter/response columns return null #686 echo behavior) or the predicate is neutralized for consumed params - likely a second latent bug behind the first. - DELETE WHERE / EXEC params pass JSON strings verbatim ([FEATURE] Extended support for batch mutation operations in the planner #683) - a separate code path, not fixed by this.
Related
- [FEATURE] SQL functions in WHERE clause not evaluated for required parameters (start_time), and projected parameter/response columns return null #686 (scalar function evaluation in WHERE, closed via feat: mcp credential (re)sourcing via server.env_file + reload_credentials tool (#688) #689) - sibling: covered flat scalar params, not structured fan-out
- [FEATURE] Extended support for batch mutation operations in the planner #683 (JSON verbatim in DELETE/EXEC params) - same family, mutation/lifecycle scope
- Unblocks attribute-based instance type selection (
GetInstanceTypesFromInstanceRequirements) and any query/ec2 op with required structure params
Activity
[-][FEATURE] [FEATURE] Fan out JSON() structured values in SELECT WHERE clauses through query param transposition (AWS query/ec2 protocols)[/-][+][FEATURE] Fan out JSON() structured values in SELECT WHERE clauses through query param transposition (AWS query/ec2 protocols)[/+]fix: stdio malformed-frame resilience, JSON() param fan-out, window/C…