Repository navigation
desc-alias-and-otlp-push-exporter - #795
Merged
Merged
Conversation
- MCP: class DESC as read-only like DESCRIBE (#773); the parser already accepts DESC and DESC EXTENDED on every other path - --output otel: OTLP/HTTP push exporter behind the standard OTEL_EXPORTER_OTLP_* variables, batched per statement, chunked, retried with backoff (#755 phase 2) - robot: DESC alias scenarios (CLI and MCP read_only), OTLP mock plus push, chunk and retry scenarios Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The Linux runners start the mocks via docker compose whenever docker is present, regardless of EXECUTION_PLATFORM, so the OTLP mock needs a compose service as well as the native launcher; the arm64 job failed the three push scenarios with connection refused on 127.0.0.1:1200. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Robot's Run Process captures stdout through a pipe it reads only after the child exits; the 11 KB /requests response exceeds the 4 KB pipe buffer on Windows and curl blocked forever, hanging the Windows job. Route stdout to a file and cap the mock calls with --max-time. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
| // OTEL_EXPORTER_OTLP_* environment variables; a logs-specific variable | ||
| // takes precedence over its generic counterpart. | ||
| const ( | ||
| otlpLogsEndpointEnv = "OTEL_EXPORTER_OTLP_LOGS_ENDPOINT" |
Contributor
There was a problem hiding this comment.
This should not be. None of this functionality should be driven by env vars. Configuration for stackql belongs in idiomatic DTO structures. There are countless existing ones to folllow. For a groupling like this, they probably go in a dedicated struct which is an optional member of some higher level DTO
Review: exporter settings belong in a DTO, not environment variables. The exporter is now sink.OTLPHTTPConfig (json/yaml tagged), the optional exporter member of output.OTelConfig, parsed once from the new --otel.config flag; no OTEL_EXPORTER_OTLP_* variables are read and a malformed value fails startup. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
general-kroll-4-life
approved these changes
Sep 26, 2026
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #773. Implements phase 2 of #755; the phase 3 backend recipes are website docs and are listed under follow-ups.
DESCas an alias forDESCRIBE(#773)stackql-parserv0.0.16-alpha02) already listsDESCas an explain synonym, soDESC <table>andDESC EXTENDED <table>parse and execute asDESCRIBEon the CLI, shell and pg wire paths. No engine change was needed there.pkg/mcp_server/policy.ClassifyQuery) switches on the literal first token and did not knowDESC, sorun_select_querywithDESC ...was classed unknown: refused onread_onlyservers and gated onsafeones.DESCis now classed read-only alongsideDESCRIBE.DESCcannot clash with theORDER BY ... DESCdirection keyword, which never starts a statement; the grammar already distinguishes the two.DESC METHOD .... TheDESCRIBE METHODrule instackql-parser(sql.y,explain_statement) namesDESCRIBEonly; extending it needs a parser release and ago.modbump, so it is a follow-up in that repo.--output otelOTLP/HTTP push exporter (#755 phase 2)Configured through a new
--otel.configflag (a JSON / YAML string, like the other raw-config flags), parsed once at startup into anOTelConfigDTO whose optionalexportermember is asink.OTLPHTTPConfig(json/yaml-tagged, alongsidesink.FileConfig). No environment variables are read; this replaced theOTEL_EXPORTER_OTLP_*design after review.{ "exporter": { "endpoint": "http://collector:4318/v1/logs", "headers": { "authorization": "<token>" }, "timeout_ms": 10000, "batch_size": 512 } }exporter.endpointis required to enable the exporter and is used verbatim;timeout_msandbatch_sizedefault to 10000 and 512. A malformed value, or an exporter without an endpoint, fails startup withinvalid --otel.config: ....Behaviour:
LogsData(oneresourceLogs/scopeLogsenvelope, NlogRecords), chunked by the batch size. The last chunk ships after the completion record, so the process does not exit before the push completes.Retry-After(delay-seconds) when sent. Client errors are not retried. A failed export is written to stderr after the rows have streamed, because the response handler discards writer errors.Content-Type: application/json);OTEL_EXPORTER_OTLP_PROTOCOLis not consulted.Design choices:
pkg/sinkasotlpHTTPSinkbehindNewOTLPHTTPSink(OTLPHTTPConfig) (Sink, error); the DTO is exported only because it is deserialised configuration, the sink itself stays unexported.output.InitOTelConfigmirrorsintrinsic.Initfor the--previewflag. The MCP auditotelformat can reuse the same sink and DTO later.sink.NewMultiSinkis the fan-out (stdout + exporter);OTelWriternow closes its sink at the end ofWrite/WriteErrorto flush the batch.test/python/stackql_test_tooling/flask/otlp/app.py, port 1200) captures requests for assertion and has a keyed flaky route. Only the port and base URL were added tostackql_context.py(config, mirroring the retry mock); queries are inline in the scenarios. The/requestsfetch writes curl's stdout to a file: robot'sRun Processcaptures stdout through a pipe it reads only after exit, and the 11 KB response deadlocks against the 4 KB pipe buffer on Windows (Linux and macOS have 64 KB), which is what hung the Windows job on the first run.web_service_keywords.pyanddocker-compose-testing.yml, which the Linux runners use for the mocks whenever docker is present (regardless ofEXECUTION_PLATFORM). The push scenarios skip onEXECUTION_PLATFORM=docker, like the--env.filescenario, because the exec container neither receives the exporter environment nor reaches the host mock.Test plan
pkg/sink(batching and envelope merge, retry/backoff/Retry-After, no retry on 4xx, max attempts, env precedence, multi sink),internal/stackql/output(push through the writer via env, chunking, failure on stderr),pkg/mcp_server/policy(desc,DESC EXTENDED).stackql_mocked_from_cmd_line.robot:Desc Extended Is Accepted As An Alias For Describe Extended(exactDESCRIBE EXTENDEDoutput forDESC EXTENDED, plus plaindesc),OTel Output Pushes Statement Batch To OTLP Endpoint,OTel Output Chunks Pushed Batches By Configured Batch Size,OTel Output Retries OTLP Push On Server Error. All pass locally with the cross-compiled binary.mcp.robot:MCP HTTP Mode Read Only Allows Desc Alias(DESC EXTENDED google.storage.bucketsviarun_select_queryon the 9920 read_only server). Verified by hand against a read_only server: main refuses, this branch returns rows.golangci-lintv2.5.0: 0 issues.Follow-ups
stackql-parser: acceptDESC METHODin the grammar, then bump here.--output otelphase 2 and 3: OTLP/HTTP push exporter and backend recipes #755 phase 2 docs (Output Modes, global flags) and phase 3 ClickHouse / Datadog recipes on the website.otelformat could adopt the same OTLP/HTTP sink.🤖 Generated with Claude Code