Skip to content

feat(gateway): sign S3-compatible endpoints via an aws secret type - #418

Open
tenequm wants to merge 3 commits into
onecli:mainfrom
tenequm:feat/s3-compatible-sigv4-secrets
Open

feat(gateway): sign S3-compatible endpoints via an aws secret type#418
tenequm wants to merge 3 commits into
onecli:mainfrom
tenequm:feat/s3-compatible-sigv4-secrets

Conversation

@tenequm

@tenequm tenequm commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

I have read the CONTRIBUTING.md file.

YES

What kind of change does this PR introduce?

Feature - gateway request signing. Adds an aws secret type so the gateway can SigV4-sign requests to S3-compatible endpoints (Hetzner Object Storage, Cloudflare R2, Backblaze B2, MinIO), not just *.amazonaws.com. Credentials stay in the vault; the agent's machine never holds them.

Closes #419. Related but separate: #417 (non-HTTP credential flows). Discussed on the 2026-07-07 call with @johnnyfish and @guyb1, who invited this PR. Opening as a draft for design feedback before the dashboard UI (see Additional context).

What is the current behavior?

The gateway has a working SigV4 finalizer (apps/gateway/src/gateway/finalizers/aws_sigv4.rs), but it only fires for the compiled-in aws provider whose host rules match .amazonaws.com / .api.aws. For a non-AWS S3 host two things block it:

  1. Nothing routes a non-AWS host to the finalizer.
  2. Even if it did, parse_service_region("nbg1.your-objectstorage.com") returns ("unknown", ...), so the signature scope is wrong and S3 rejects it.

A generic secret can't fill the gap: it injects a static header, while S3 auth is a computed per-request signature.

What is the new behavior?

An aws secret type, resolved through the secret layer (user-configured hosts) rather than the compiled-in provider registry - a Hetzner bucket isn't "AWS", so modeling it as a user secret with a free-form hostPattern is the honest fit and gives multi-bucket / wildcard support for free.

  • secret_inject.rs - new "aws" arm. Value is JSON { accessKeyId, secretAccessKey, sessionToken? }; injectionConfig is { region, service? } (service defaults to s3). Emits the internal x-onecli-aws-* headers plus a new x-onecli-aws-service.
  • aws_sigv4.rs - AwsCredentials gains an optional service; when set it overrides hostname parsing so any S3-compatible host signs correctly. AWS hosts are unchanged (no service header -> parsed from hostname as today).
  • forward.rs - if no connection/host finalizer matched but the request carries the internal AWS access-key header (an aws secret injected it), run the SigV4 finalizer. finalize_request is a no-op without those headers, so non-AWS traffic is untouched.
  • API - aws added to the secret type enum and the injectionConfig union ({ region, service? }), with validation that the value is JSON carrying both keys and that a region is present, on both the create and update paths.

Additional context

Design note (for reviewers). I considered adding an S3-compatible host rule to the aws app connection with credential_host_field (the JFrog *.jfrog.io idiom) instead. I went with the secret layer because a non-AWS endpoint isn't conceptually an "AWS" connection, and the secret path already models per-host user credentials with wildcard support. Happy to move it to the connection side if you'd rather keep all AWS-signing under the aws provider - it's your vocabulary. Likewise, forward.rs dispatch uses header-presence detection (smallest change, no resolve-cache format change); I can thread an explicit finalizer through the secret resolve result instead if you prefer. No new trust surface either way: an agent can already set x-onecli-aws-* toward *.amazonaws.com and only ever signs with its own supplied creds - no vault material is exposed.

Deliberately out of scope (follow-up). No dashboard dialog yet - the web SecretType union is local to the dialog, so this PR keeps the UI compiling untouched and the feature is reachable via the API/SDK/CLI. If the design lands, the UI add is a small follow-up: an "AWS / S3-compatible" option with Access Key ID / Secret Access Key / Region / optional Service fields that packs the two keys into the JSON value and region/service into injectionConfig.

Verification.

  • pnpm check (lint + types + format) clean across the monorepo.
  • Gateway: cargo fmt --check and cargo clippy -- -D warnings clean; cargo test green (unit + integration), including the service override, a Hetzner-style host signing as s3, and the aws injection shape.
  • API: vitest green (284), including aws create/value validation and the shared value validator. Create and update both reject a non-JSON value or a missing region, so a secret can't be edited into a silently-broken state.

Try it. Create an aws secret (via API) with hostPattern: nbg1.your-objectstorage.com, value {"accessKeyId":"...","secretAccessKey":"..."}, injectionConfig: {"region":"eu-central-1"}, assign it to an agent, and point an S3 client at the bucket through the gateway - it signs on the wire, the key never lands on the sandbox.

tenequm added 2 commits July 7, 2026 20:08
Adds an "aws" secret type whose value carries S3 credentials and whose
injectionConfig carries { region, service? }. The gateway emits the
existing internal x-onecli-aws-* headers plus a service header, and the
SigV4 finalizer signs with the explicit service when set - so
S3-compatible stores (Hetzner, R2, B2, MinIO) work, not just
*.amazonaws.com hosts. Real AWS hosts are unchanged (no service header
-> hostname parsing as before).
createSecret enforced JSON-with-both-keys and a present region for aws
secrets, but updateSecret did not, so an edit could set injectionConfig
to null or the value to non-JSON and the secret would silently inject
nothing at request time. Share one isValidAwsValueJson validator between
the create schema and updateSecret, and reject an aws update that drops
the region or supplies a malformed value.
@tenequm
tenequm marked this pull request as ready for review July 7, 2026 19:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Sign S3-compatible object storage (Hetzner, R2, B2, MinIO), not only AWS

1 participant