From 53ca097e4c29ddf39bfeb2c65f8a94288443225d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Vedran=20Bilopavlovi=C4=87?= Date: Sat, 13 Jun 2026 13:11:23 +0200 Subject: [PATCH 01/25] =?UTF-8?q?fix(docker):=20JIT=20image=20=E2=80=94=20?= =?UTF-8?q?copy=20NpgsqlRest.Common=20shared=20source;=20CI:=20stop=20publ?= =?UTF-8?q?ishing=20on=20PRs?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The JIT Docker build (docker/Dockerfile.jit) compiles from source inside the container, and NpgsqlRest.csproj now compiles + global-uses the shared source in NpgsqlRest.Common/ (). That folder was never copied into the build context, so the publish failed with CS0234 'NpgsqlRest.Common'. Added the COPY; validated by building the image locally. (The other Docker variants copy a prebuilt binary, so they were fine.) CI: removed the pull_request trigger so the publish/release pipeline no longer runs on every PR (dependabot PRs were flooding Actions with failing publish runs — PRs can't publish anyway, no secrets). TEMP (this commit only): build-test-publish is if:false and build-docker-jit is detached from create-release, so this push runs ONLY build-docker-jit to push the missing v3.17.0-jit / latest-jit image. To be reverted once green. --- .github/workflows/build-test-publish.yml | 8 +++++--- docker/Dockerfile.jit | 3 +++ 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/.github/workflows/build-test-publish.yml b/.github/workflows/build-test-publish.yml index 3e5b8e75..fb10a49c 100644 --- a/.github/workflows/build-test-publish.yml +++ b/.github/workflows/build-test-publish.yml @@ -3,8 +3,6 @@ name: Build, Test, Publish and Release on: push: branches: [ master ] - pull_request: - branches: [ master ] workflow_dispatch: # Add workflow-level permissions @@ -14,6 +12,9 @@ permissions: jobs: build-test-publish: + # TEMP (jit-only republish of v3.17.0): disabled so the whole release chain is skipped — 3.17.0 is + # already published, we only need the missing -jit Docker image. RESTORE: delete this `if: false`. + if: false runs-on: ubuntu-22.04 services: postgres: @@ -275,7 +276,8 @@ jobs: vbilopav/npgsqlrest:latest-aot build-docker-jit: - needs: create-release + # TEMP (jit-only republish): `needs: create-release` removed so this runs standalone while the rest of + # the chain is disabled. It reads the version from version.txt itself. RESTORE: re-add `needs: create-release`. runs-on: ubuntu-22.04 permissions: contents: read diff --git a/docker/Dockerfile.jit b/docker/Dockerfile.jit index f5c86336..5b0ce475 100644 --- a/docker/Dockerfile.jit +++ b/docker/Dockerfile.jit @@ -21,6 +21,9 @@ RUN dotnet restore NpgsqlRestClient/NpgsqlRestClient.csproj COPY LICENSE LICENSE COPY README.md README.MD COPY NpgsqlRest/ NpgsqlRest/ +# Shared source compiled into NpgsqlRest.csproj ( + global +# usings). Without this the JIT build fails: NpgsqlRest.GlobalUsings.g.cs -> CS0234 'NpgsqlRest.Common'. +COPY NpgsqlRest.Common/ NpgsqlRest.Common/ COPY NpgsqlRestClient/*.cs NpgsqlRestClient/ COPY NpgsqlRestClient/*.csproj NpgsqlRestClient/ COPY NpgsqlRestClient/appsettings.json NpgsqlRestClient/ From 1f6e4074620f85bcec8a073c8683ddfec2e7d78e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Vedran=20Bilopavlovi=C4=87?= Date: Sat, 13 Jun 2026 13:12:25 +0200 Subject: [PATCH 02/25] docs(changelog): mark v3.17.0 released (2026-06-13); CI: restore full release workflow Re-enables the full release chain (build-test-publish no longer if:false; build-docker-jit needs create-release again) now that the v3.17.0-jit image is published. The pull_request trigger stays removed, so the publish/release pipeline runs only on push to master + manual dispatch (no PR/dependabot noise). --- .github/workflows/build-test-publish.yml | 6 +----- changelog/v3.17.0.md | 2 +- 2 files changed, 2 insertions(+), 6 deletions(-) diff --git a/.github/workflows/build-test-publish.yml b/.github/workflows/build-test-publish.yml index fb10a49c..e9d337b2 100644 --- a/.github/workflows/build-test-publish.yml +++ b/.github/workflows/build-test-publish.yml @@ -12,9 +12,6 @@ permissions: jobs: build-test-publish: - # TEMP (jit-only republish of v3.17.0): disabled so the whole release chain is skipped — 3.17.0 is - # already published, we only need the missing -jit Docker image. RESTORE: delete this `if: false`. - if: false runs-on: ubuntu-22.04 services: postgres: @@ -276,8 +273,7 @@ jobs: vbilopav/npgsqlrest:latest-aot build-docker-jit: - # TEMP (jit-only republish): `needs: create-release` removed so this runs standalone while the rest of - # the chain is disabled. It reads the version from version.txt itself. RESTORE: re-add `needs: create-release`. + needs: create-release runs-on: ubuntu-22.04 permissions: contents: read diff --git a/changelog/v3.17.0.md b/changelog/v3.17.0.md index f5b52704..4ee8ac7d 100644 --- a/changelog/v3.17.0.md +++ b/changelog/v3.17.0.md @@ -1,6 +1,6 @@ # Changelog v3.17.0 -## Version [3.17.0](https://github.com/NpgsqlRest/NpgsqlRest/tree/3.17.0) (unreleased) +## Version [3.17.0](https://github.com/NpgsqlRest/NpgsqlRest/tree/3.17.0) (2026-06-13) [Full Changelog](https://github.com/NpgsqlRest/NpgsqlRest/compare/3.16.3...3.17.0) From bf78434dc6c95436d34f710d11851ae36e164d0a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Vedran=20Bilopavlovi=C4=87?= Date: Tue, 23 Jun 2026 11:47:02 +0200 Subject: [PATCH 03/25] fix(http-types): fire one outbound call per distinct HTTP type (v3.17.1) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A DB-function composite parameter is expanded into one parameter per field, each carrying the same TypeDescriptor.CustomType. The per-request list of HTTP types therefore held the type name once per field, and InvokeAllAsync fired InvokeAsync once per entry — making N identical outbound calls (N = field count) while the fill loop resolved handlers by distinct type name. SQL-file endpoints kept the type as a single composite parameter and were unaffected. Guard the firing loop to request each distinct HTTP type once, reusing the dictionary the fill loop already keys on. Preserves the established one-call-per-distinct-type contract. Adds two regression tests that count actual outbound calls via a WireMock callback: a 6-field type fires exactly 1 call (was 6), and two distinct types fire 1 call each. Verified failing pre-fix (found 6). See changelog/v3.17.1.md. --- .../HttpClientType/HttpClientTypeHandler.cs | 8 +++ .../HttpClientTypeTests.cs | 72 +++++++++++++++++++ changelog/v3.17.1.md | 45 ++++++++++++ npm/package.json | 2 +- version.txt | 2 +- 5 files changed, 127 insertions(+), 2 deletions(-) create mode 100644 changelog/v3.17.1.md diff --git a/NpgsqlRest/HttpClientType/HttpClientTypeHandler.cs b/NpgsqlRest/HttpClientType/HttpClientTypeHandler.cs index 41a126c1..96add620 100644 --- a/NpgsqlRest/HttpClientType/HttpClientTypeHandler.cs +++ b/NpgsqlRest/HttpClientType/HttpClientTypeHandler.cs @@ -284,6 +284,14 @@ public static async Task InvokeAllAsync( foreach (var typeName in typeNames) { + // A DB-function composite parameter is expanded into one parameter per field, + // each carrying the same CustomType, so typeNames can contain the same type + // multiple times. Fire exactly one outbound call per distinct HTTP type - the + // fill loop below resolves handlers by distinct type name as well. + if (handlers.ContainsKey(typeName)) + { + continue; + } if (HttpClientTypes.Definitions.TryGetValue(typeName, out var definition)) { var handler = new HttpClientTypeHandler(definition, replacements); diff --git a/NpgsqlRestTests/HttpClientTypeTests/HttpClientTypeTests.cs b/NpgsqlRestTests/HttpClientTypeTests/HttpClientTypeTests.cs index d4e6bfad..9cd232b7 100644 --- a/NpgsqlRestTests/HttpClientTypeTests/HttpClientTypeTests.cs +++ b/NpgsqlRestTests/HttpClientTypeTests/HttpClientTypeTests.cs @@ -2,6 +2,9 @@ using WireMock.RequestBuilders; using WireMock.ResponseBuilders; using WireMock.Settings; +using WireMock; +using WireMock.Types; +using WireMock.Util; namespace NpgsqlRestTests; @@ -879,4 +882,73 @@ public async Task Test_get_http_api_multi_types_one_fails() content.Should().Contain("\"users_status\" : 200"); content.Should().Contain("\"products_status\" : 500"); } + + // Regression: a DB-function composite parameter is expanded into one parameter per field, + // each carrying the same HTTP CustomType. Before the InvokeAllAsync dedup guard, the request + // fired once per field (6 outbound calls for the 6-field all-fields type). It must fire exactly + // once per distinct HTTP type. Counter-based to count actual outbound calls (same pattern as + // the retry tests). + [Fact] + public async Task Test_multi_field_type_fires_exactly_one_outbound_call() + { + int calls = 0; + _server + .Given(Request.Create().WithPath("/api/all-fields").UsingGet()) + .RespondWith(Response.Create().WithCallback(req => + { + Interlocked.Increment(ref calls); + return new ResponseMessage + { + StatusCode = 200, + BodyData = new BodyData { BodyAsString = "response body", DetectedBodyType = BodyType.String } + }; + })); + + using var response = await _test.Client.GetAsync("/api/get-http-api-all-fields/"); + var content = await response.Content.ReadAsStringAsync(); + + response?.StatusCode.Should().Be(HttpStatusCode.OK); + content.Should().Contain("\"body\" : \"response body\""); + calls.Should().Be(1); + } + + // Regression: two distinct HTTP types referenced by a single function must each fire exactly + // once - one outbound call per distinct type, not per expanded field. + [Fact] + public async Task Test_multi_types_each_fire_exactly_one_outbound_call() + { + int usersCalls = 0; + int productsCalls = 0; + _server + .Given(Request.Create().WithPath("/api/users").UsingGet()) + .RespondWith(Response.Create().WithCallback(req => + { + Interlocked.Increment(ref usersCalls); + return new ResponseMessage + { + StatusCode = 200, + BodyData = new BodyData { BodyAsString = "[{\"id\": 1}]", DetectedBodyType = BodyType.String } + }; + })); + _server + .Given(Request.Create().WithPath("/api/products").UsingGet()) + .RespondWith(Response.Create().WithCallback(req => + { + Interlocked.Increment(ref productsCalls); + return new ResponseMessage + { + StatusCode = 200, + BodyData = new BodyData { BodyAsString = "[{\"id\": 101}]", DetectedBodyType = BodyType.String } + }; + })); + + using var response = await _test.Client.GetAsync("/api/get-http-api-multi-types/"); + var content = await response.Content.ReadAsStringAsync(); + + response?.StatusCode.Should().Be(HttpStatusCode.OK); + content.Should().Contain("\"users_status\" : 200"); + content.Should().Contain("\"products_status\" : 200"); + usersCalls.Should().Be(1); + productsCalls.Should().Be(1); + } } diff --git a/changelog/v3.17.1.md b/changelog/v3.17.1.md new file mode 100644 index 00000000..3d659752 --- /dev/null +++ b/changelog/v3.17.1.md @@ -0,0 +1,45 @@ +# Changelog v3.17.1 + +## Version [3.17.1](https://github.com/NpgsqlRest/NpgsqlRest/tree/3.17.1) + +[Full Changelog](https://github.com/NpgsqlRest/NpgsqlRest/compare/3.17.0...3.17.1) + +Patch release that fixes **duplicate outbound HTTP calls for HTTP Custom Types on database-function endpoints**. + +## The bug + +An endpoint backed by a **database function/procedure** whose parameter is an HTTP Custom Type fired **one outbound HTTP call per field of the type** on every inbound request. A 4-field type made 4 identical calls; a 6-field type made 6 — multiplying latency and load on the target service. SQL-file endpoints were not affected. + +## Cause + +A composite function parameter is expanded into one parameter **per field**, and each of those expanded parameters carries the same `TypeDescriptor.CustomType` (the HTTP type name). The type name was therefore collected once per field, so the per-request list of HTTP types to invoke contained the same name N times. The firing loop in `HttpClientTypeHandler.InvokeAllAsync` then called `InvokeAsync` once per entry, with no dedup — while the fill loop immediately below resolves handlers by **distinct** type name out of a dictionary. The design already assumes exactly one call per distinct type; the firing loop just failed to match, so the extra calls were made and their handlers discarded. + +SQL-file endpoints keep the HTTP type as a single composite parameter (`CustomTypeName` null), so the name was collected once → exactly one call, which is why they were unaffected. + +## What changed + +A one-line guard in `HttpClientTypeHandler.InvokeAllAsync` makes the firing loop request each distinct HTTP type once: + +```csharp +foreach (var typeName in typeNames) +{ + if (handlers.ContainsKey(typeName)) continue; // fire once per distinct type + if (HttpClientTypes.Definitions.TryGetValue(typeName, out var definition)) + { + var handler = new HttpClientTypeHandler(definition, replacements); + handlers[typeName] = handler; + tasks.Add((typeName, handler, handler.InvokeAsync(cancellationToken))); + } +} +``` + +This is the single choke point for firing outbound calls and reuses the same dictionary the fill loop already keys on, so it is robust regardless of how the parameter list is built. The established contract is preserved: one call per **distinct** HTTP type, shared from one response — two parameters referencing the same type still share a single call, and two different types remain two separate calls. + +## Tests + +Two regression tests added (`NpgsqlRestTests/HttpClientTypeTests/HttpClientTypeTests.cs`) that count **actual** outbound calls via a WireMock response callback (the prior suite asserted response content but never call counts): + +- A 6-field HTTP type fires **exactly one** outbound call (was 6 before the fix). +- Two distinct HTTP types in one function each fire **exactly one** call. + +Verified against the pre-fix code: the multi-field test failed with `Expected calls to be 1, but found 6`, reproducing the reported behavior. All 46 HTTP-client-type tests pass with the fix. diff --git a/npm/package.json b/npm/package.json index 484cd0f8..859024b3 100644 --- a/npm/package.json +++ b/npm/package.json @@ -1,6 +1,6 @@ { "name": "npgsqlrest", - "version": "3.17.0", + "version": "3.17.1", "description": "Automatic REST API for PostgreSQL Databases Client Build", "scripts": { "postinstall": "node postinstall.js", diff --git a/version.txt b/version.txt index f85bf6e3..0caba260 100644 --- a/version.txt +++ b/version.txt @@ -1 +1 @@ -3.17.0 \ No newline at end of file +3.17.1 \ No newline at end of file From 9f67d878a885fb1a4731f57b040efbd545bf1b6b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Vedran=20Bilopavlovi=C4=87?= Date: Tue, 23 Jun 2026 12:27:35 +0200 Subject: [PATCH 04/25] feat(http-types): response caching via @cache; parse directives after headers (v3.18.0) Add opt-in HTTP Custom Type response caching with a @cache type-comment directive (alongside @timeout/@retry_delay): - GET-only (warn + ignore on non-GET); success-only storage so a transient upstream failure is never pinned for the TTL. - New HttpResponseCache: in-memory store with Lazy stampede coalescing (one outbound call per key), max-entries cap, prune timer. - Cache key = method + resolved URL + content-type + headers + body. - HttpClientOptions.CacheEnabled (global kill switch), MaxCacheEntries, CachePruneIntervalSeconds; full config round-trip wired (Builder, appsettings.json, ConfigTemplate, ConfigDefaults, ConfigSchemaGenerator). Also fix: @timeout/@retry_delay/@cache are now parsed both before the request line AND after the headers. Previously only the leading position was recognized, so a directive after the headers (as the docs showed) was silently ignored. Real headers (e.g. Cache-Control) are unaffected. Folds in the v3.17.1 duplicate-outbound-call fix: version.txt and npm/package.json bumped to 3.18.0, changelog renamed v3.17.1 -> v3.18.0 covering both the feature and the fix. Tests: parse-level @cache forms + both-placement directive tests; integration tests for cache hit, 6-field dedup+cache, error-not-cached, POST-ignored, TTL expiry. Full suite green (2284). --- .../HttpClientType/HttpClientTypeHandler.cs | 72 ++++- NpgsqlRest/HttpClientType/HttpClientTypes.cs | 126 +++++++- .../HttpClientType/HttpResponseCache.cs | 160 ++++++++++ .../HttpClientType/HttpTypeDefinition.cs | 13 + NpgsqlRest/NpgsqlRestBuilder.cs | 9 + NpgsqlRest/Options/HttpClientOptions.cs | 20 ++ NpgsqlRestClient/Builder.cs | 5 +- NpgsqlRestClient/ConfigDefaults.cs | 5 +- NpgsqlRestClient/ConfigSchemaGenerator.cs | 3 + NpgsqlRestClient/ConfigTemplate.cs | 17 +- NpgsqlRestClient/appsettings.json | 17 +- .../HttpClientTypeCacheTests.cs | 289 ++++++++++++++++++ .../ParseHttpTypeDefinitionTests.cs | 144 +++++++++ changelog/v3.17.1.md | 45 --- changelog/v3.18.0.md | 53 ++++ npm/package.json | 2 +- version.txt | 2 +- 17 files changed, 929 insertions(+), 53 deletions(-) create mode 100644 NpgsqlRest/HttpClientType/HttpResponseCache.cs create mode 100644 NpgsqlRestTests/HttpClientTypeTests/HttpClientTypeCacheTests.cs delete mode 100644 changelog/v3.17.1.md create mode 100644 changelog/v3.18.0.md diff --git a/NpgsqlRest/HttpClientType/HttpClientTypeHandler.cs b/NpgsqlRest/HttpClientType/HttpClientTypeHandler.cs index 96add620..76e6127b 100644 --- a/NpgsqlRest/HttpClientType/HttpClientTypeHandler.cs +++ b/NpgsqlRest/HttpClientType/HttpClientTypeHandler.cs @@ -160,6 +160,76 @@ public async Task InvokeAsync(CancellationToken cancellationToken = default) } } + /// + /// Runs the outbound call, routing it through when the type opts in + /// via @cache and caching is enabled globally. On a cache hit (or a coalesced concurrent call) + /// the cached response is applied to this handler so the fill loop reads it exactly as a live response. + /// + public async Task InvokeWithCacheAsync(CancellationToken cancellationToken = default) + { + if (!Options.HttpClientOptions.CacheEnabled || !definition.CacheEnabled) + { + await InvokeAsync(cancellationToken); + return; + } + + var key = ComputeCacheKey(); + var cached = await HttpResponseCache.GetOrCreateAsync( + key, + definition.CacheDuration, + async ct => + { + await InvokeAsync(ct); + return SnapshotResponse(); + }, + cancellationToken); + + ApplyResponse(cached); + } + + /// + /// Cache key for this request: method + resolved URL + resolved content-type + resolved headers + /// (sorted) + resolved body. Placeholders are resolved so per-request values vary the key; a type + /// with no placeholders produces a constant key (one shared cached response). + /// + private string ComputeCacheKey() + { + var sb = new StringBuilder(); + sb.Append(definition.Method).Append('\n'); + sb.Append(ResolveValue(definition.Url)).Append('\n'); + if (definition.ContentType is not null) + { + sb.Append(ResolveValue(definition.ContentType)); + } + sb.Append('\n'); + if (definition.Headers is { Count: > 0 }) + { + foreach (var header in definition.Headers.OrderBy(h => h.Key, StringComparer.Ordinal)) + { + sb.Append(header.Key).Append(':').Append(ResolveValue(header.Value)).Append('\n'); + } + } + sb.Append('\n'); + if (definition.Body is not null) + { + sb.Append(ResolveValue(definition.Body)); + } + return sb.ToString(); + } + + private CachedHttpResponse SnapshotResponse() => + new(StatusCode, Body, ResponseHeaders, ContentType, IsSuccess, ErrorMessage); + + private void ApplyResponse(CachedHttpResponse r) + { + StatusCode = r.StatusCode; + Body = r.Body; + ResponseHeaders = r.ResponseHeaders; + ContentType = r.ContentType; + IsSuccess = r.IsSuccess; + ErrorMessage = r.ErrorMessage; + } + private bool ShouldRetry(int statusCode) { if (definition.RetryOnStatusCodes is null) @@ -296,7 +366,7 @@ public static async Task InvokeAllAsync( { var handler = new HttpClientTypeHandler(definition, replacements); handlers[typeName] = handler; - tasks.Add((typeName, handler, handler.InvokeAsync(cancellationToken))); + tasks.Add((typeName, handler, handler.InvokeWithCacheAsync(cancellationToken))); } } diff --git a/NpgsqlRest/HttpClientType/HttpClientTypes.cs b/NpgsqlRest/HttpClientType/HttpClientTypes.cs index 7debf487..49c40c25 100644 --- a/NpgsqlRest/HttpClientType/HttpClientTypes.cs +++ b/NpgsqlRest/HttpClientType/HttpClientTypes.cs @@ -91,6 +91,8 @@ public HttpClientTypes(IApplicationBuilder? builder, RetryStrategy? retryStrateg TimeSpan? timeout = null; TimeSpan[]? retryDelays = null; HashSet? retryOnStatusCodes = null; + bool cacheEnabled = false; + TimeSpan? cacheDuration = null; // Parse directives before the request line while (pos < span.Length) @@ -129,6 +131,17 @@ public HttpClientTypes(IApplicationBuilder? builder, RetryStrategy? retryStrateg continue; } + // Check for cache directive + if (TryParseCacheDirective(trimmedLine, typeName, out var parsedCacheDuration)) + { + cacheEnabled = true; + cacheDuration = parsedCacheDuration; + pos += lineEnd == -1 ? line.Length : lineEnd; + if (pos < span.Length && span[pos] == '\r') pos++; + if (pos < span.Length && span[pos] == '\n') pos++; + continue; + } + // Check for # comment (non-timeout directive) if (trimmedLine[0] == '#') { @@ -198,12 +211,15 @@ public HttpClientTypes(IApplicationBuilder? builder, RetryStrategy? retryStrateg Url = new string(urlSpan), Timeout = timeout, RetryDelays = retryDelays, - RetryOnStatusCodes = retryOnStatusCodes + RetryOnStatusCodes = retryOnStatusCodes, + CacheEnabled = cacheEnabled, + CacheDuration = cacheDuration }; // Move past first line if (firstLineEnd == -1) { + NormalizeCacheDirective(result, typeName); result.NeedsParsing = needsParsing; return result; } @@ -233,6 +249,36 @@ public HttpClientTypes(IApplicationBuilder? builder, RetryStrategy? retryStrateg break; } + // Directives may also appear in the header section (after the request line), not only + // before it. Check for them before treating the line as an HTTP header. A directive's + // separator/value shape means real headers (which carry a 'Name-Word: value') don't match. + if (TryParseTimeoutDirective(line, typeName, out var hdrTimeout)) + { + result.Timeout = hdrTimeout; + pos += lineEnd == -1 ? line.Length : lineEnd; + if (pos < span.Length && span[pos] == '\r') pos++; + if (pos < span.Length && span[pos] == '\n') pos++; + continue; + } + if (TryParseRetryDirective(line, typeName, out var hdrDelays, out var hdrCodes)) + { + result.RetryDelays = hdrDelays; + result.RetryOnStatusCodes = hdrCodes; + pos += lineEnd == -1 ? line.Length : lineEnd; + if (pos < span.Length && span[pos] == '\r') pos++; + if (pos < span.Length && span[pos] == '\n') pos++; + continue; + } + if (TryParseCacheDirective(line, typeName, out var hdrCacheDuration)) + { + result.CacheEnabled = true; + result.CacheDuration = hdrCacheDuration; + pos += lineEnd == -1 ? line.Length : lineEnd; + if (pos < span.Length && span[pos] == '\r') pos++; + if (pos < span.Length && span[pos] == '\n') pos++; + continue; + } + int colonIndex = line.IndexOf(':'); if (colonIndex > 0) { @@ -285,10 +331,33 @@ public HttpClientTypes(IApplicationBuilder? builder, RetryStrategy? retryStrateg } } + NormalizeCacheDirective(result, typeName); result.NeedsParsing = needsParsing; return result; } + // Caching is only safe for GET (idempotent reads). A @cache directive on any other method is + // almost always a mistake; warn and ignore it rather than caching a mutating call. Applied after + // the whole comment is parsed so it sees @cache whether it appeared before the request line or + // among the headers, and after the method is known. + private static void NormalizeCacheDirective(HttpTypeDefinition result, string? typeName) + { + if (!result.CacheEnabled) + { + return; + } + if (!string.Equals(result.Method, "GET", StringComparison.Ordinal)) + { + Logger?.LogWarning("Type '{TypeName}': @cache is only supported for GET requests; ignoring it for '{Method}'", typeName, result.Method); + result.CacheEnabled = false; + result.CacheDuration = null; + } + else if (result.CacheDuration is null) + { + Logger?.LogWarning("Type '{TypeName}': @cache has no expiration interval; responses will be cached until the process restarts", typeName); + } + } + private static ReadOnlySpan TrimSpan(ReadOnlySpan span) { int start = 0; @@ -548,4 +617,59 @@ private static bool TryParseRetryDirective( return true; } + + private static bool TryParseCacheDirective(ReadOnlySpan line, string? typeName, out TimeSpan? duration) + { + duration = null; + var trimmed = TrimSpan(line); + + // Remove leading # if present + if (!trimmed.IsEmpty && trimmed[0] == '#') + { + trimmed = TrimSpan(trimmed[1..]); + } + + // Remove leading @ if present + if (!trimmed.IsEmpty && trimmed[0] == '@') + { + trimmed = TrimSpan(trimmed[1..]); + } + + // Check for "cache" keyword (case-insensitive). Must be the whole token, so "cache_profile" + // or other future "cache*" directives are not swallowed here. + if (!trimmed.StartsWith("cache", StringComparison.OrdinalIgnoreCase)) + { + return false; + } + + var afterKeyword = trimmed[5..]; // Skip "cache" + + // Bare "@cache" (no value) → enabled with no expiration. + if (afterKeyword.IsEmpty) + { + return true; + } + + // The next char must be a separator (space, '=', ':'); otherwise this is a different keyword. + char sep = afterKeyword[0]; + if (sep != ' ' && sep != '\t' && sep != '=' && sep != ':') + { + return false; + } + + if (sep == '=' || sep == ':') + { + afterKeyword = afterKeyword[1..]; + } + afterKeyword = TrimSpan(afterKeyword); + + // "@cache" followed only by separator → enabled with no expiration. + if (afterKeyword.IsEmpty) + { + return true; + } + + duration = ParseTimeoutValue(afterKeyword, typeName); + return true; + } } \ No newline at end of file diff --git a/NpgsqlRest/HttpClientType/HttpResponseCache.cs b/NpgsqlRest/HttpClientType/HttpResponseCache.cs new file mode 100644 index 00000000..df1d5099 --- /dev/null +++ b/NpgsqlRest/HttpClientType/HttpResponseCache.cs @@ -0,0 +1,160 @@ +using System.Collections.Concurrent; + +namespace NpgsqlRest.HttpClientType; + +/// +/// Immutable snapshot of an outbound HTTP type response, suitable for caching and reuse. +/// +public sealed record CachedHttpResponse( + int StatusCode, + string? Body, + string? ResponseHeaders, + string? ContentType, + bool IsSuccess, + string? ErrorMessage); + +/// +/// In-memory cache of HTTP type responses with stampede protection. A burst of concurrent requests +/// for the same key coalesce into a single outbound call; the rest await the in-flight result. +/// Only successful responses are stored, so a transient upstream failure is never pinned for the TTL. +/// +public static class HttpResponseCache +{ + private sealed class CacheEntry + { + public required CachedHttpResponse Value { get; init; } + public DateTime? ExpirationTime { get; init; } + public bool IsExpired => ExpirationTime.HasValue && DateTime.UtcNow > ExpirationTime.Value; + } + + private static readonly ConcurrentDictionary _cache = new(StringComparer.Ordinal); + // In-flight factory invocations keyed by cache key. The value is a Lazy so that even if + // ConcurrentDictionary.GetOrAdd runs its value-factory more than once under contention, only the + // stored Lazy's task is ever started — guaranteeing a single outbound call per key. + private static readonly ConcurrentDictionary>> _inflight = new(StringComparer.Ordinal); + + private static Timer? _cleanupTimer; + private static int _maxEntries = 10_000; + + public static void Start(NpgsqlRestOptions options) + { + _maxEntries = options.HttpClientOptions.MaxCacheEntries; + var interval = TimeSpan.FromSeconds(options.HttpClientOptions.CachePruneIntervalSeconds); + _cleanupTimer?.Dispose(); + _cleanupTimer = new Timer(_ => CleanupExpiredEntries(), null, interval, interval); + } + + public static void Shutdown() + { + _cleanupTimer?.Dispose(); + _cleanupTimer = null; + _cache.Clear(); + _inflight.Clear(); + } + + private static void CleanupExpiredEntries() + { + foreach (var kvp in _cache) + { + if (kvp.Value.IsExpired) + { + _cache.TryRemove(kvp.Key, out _); + } + } + } + + private static bool TryGet(string key, out CachedHttpResponse value) + { + if (_cache.TryGetValue(key, out var entry)) + { + if (entry.IsExpired) + { + _cache.TryRemove(key, out _); + value = null!; + return false; + } + value = entry.Value; + return true; + } + value = null!; + return false; + } + + private static void Store(string key, CachedHttpResponse value, TimeSpan? ttl) + { + // Bound memory: once full, don't admit new keys (existing entries still serve and expire). + // Updates to an already-cached key are always allowed. + if (_cache.Count >= _maxEntries && !_cache.ContainsKey(key)) + { + return; + } + + _cache[key] = new CacheEntry + { + Value = value, + ExpirationTime = ttl.HasValue ? DateTime.UtcNow + ttl.Value : null + }; + } + + /// + /// Returns the cached response for if present and unexpired; otherwise runs + /// (the outbound call), stores the result when it is successful, and + /// returns it. Concurrent callers for the same key coalesce into one factory invocation. + /// + public static async Task GetOrCreateAsync( + string key, + TimeSpan? ttl, + Func> factory, + CancellationToken cancellationToken) + { + while (true) + { + cancellationToken.ThrowIfCancellationRequested(); + + if (TryGet(key, out var cached)) + { + return cached; + } + + var lazy = _inflight.GetOrAdd( + key, + _ => new Lazy>(() => RunFactoryAsync(key, ttl, factory, cancellationToken))); + + try + { + return await lazy.Value.WaitAsync(cancellationToken); + } + catch (OperationCanceledException) + { + if (cancellationToken.IsCancellationRequested) + { + throw; + } + // The lead caller (whose token drove the shared run) was cancelled before this waiter. + // Loop and retry — the value may now be cached, or this caller becomes the new lead. + } + } + } + + private static async Task RunFactoryAsync( + string key, + TimeSpan? ttl, + Func> factory, + CancellationToken cancellationToken) + { + try + { + var result = await factory(cancellationToken).ConfigureAwait(false); + // Success-only: never pin a transient upstream failure for the whole TTL. + if (result.IsSuccess) + { + Store(key, result, ttl); + } + return result; + } + finally + { + _inflight.TryRemove(key, out _); + } + } +} diff --git a/NpgsqlRest/HttpClientType/HttpTypeDefinition.cs b/NpgsqlRest/HttpClientType/HttpTypeDefinition.cs index 6cb1ffb8..045a3960 100644 --- a/NpgsqlRest/HttpClientType/HttpTypeDefinition.cs +++ b/NpgsqlRest/HttpClientType/HttpTypeDefinition.cs @@ -11,4 +11,17 @@ public class HttpTypeDefinition public TimeSpan[]? RetryDelays { get; set; } public HashSet? RetryOnStatusCodes { get; set; } public bool NeedsParsing { get; set; } + + /// + /// True when the type comment carries a @cache directive, opting this HTTP type into + /// response caching. Successful responses are cached and reused for matching requests + /// (same method + resolved URL + headers + body) until elapses. + /// + public bool CacheEnabled { get; set; } + + /// + /// Time-to-live for cached responses. null when is true means + /// the response is cached with no expiration (until the process restarts). + /// + public TimeSpan? CacheDuration { get; set; } } \ No newline at end of file diff --git a/NpgsqlRest/NpgsqlRestBuilder.cs b/NpgsqlRest/NpgsqlRestBuilder.cs index a7b87d30..42b5fb15 100644 --- a/NpgsqlRest/NpgsqlRestBuilder.cs +++ b/NpgsqlRest/NpgsqlRestBuilder.cs @@ -58,6 +58,15 @@ public static IApplicationBuilder UseNpgsqlRest(this WebApplication builder, Npg if (Options.HttpClientOptions.Enabled is true) { new HttpClientTypes(builder, defaultStrategy); + + if (Options.HttpClientOptions.CacheEnabled) + { + HttpClientType.HttpResponseCache.Start(Options); + if (builder is WebApplication httpCacheApp) + { + httpCacheApp.Lifetime.ApplicationStopping.Register(HttpClientType.HttpResponseCache.Shutdown); + } + } } var ( diff --git a/NpgsqlRest/Options/HttpClientOptions.cs b/NpgsqlRest/Options/HttpClientOptions.cs index 6cbfd0fe..a72ebd18 100644 --- a/NpgsqlRest/Options/HttpClientOptions.cs +++ b/NpgsqlRest/Options/HttpClientOptions.cs @@ -44,4 +44,24 @@ public class HttpClientOptions /// Example: "http://localhost:5000" /// public string? SelfBaseUrl { get; set; } + + /// + /// Global kill switch for HTTP type response caching. When false, the @cache directive on + /// individual types is ignored and every request fires a fresh outbound call. Default is true, + /// so caching is opt-in per type via @cache and globally disable-able here. + /// + public bool CacheEnabled { get; set; } = true; + + /// + /// Maximum number of distinct cached HTTP responses held in memory. Once the cache is full, new + /// responses are not cached (existing entries are still served and expire normally). Bounds memory + /// for types whose URL/body/headers contain per-request placeholders. Default is 10000. + /// + public int MaxCacheEntries { get; set; } = 10_000; + + /// + /// Interval in seconds at which expired cached HTTP responses are pruned from memory. + /// Default is 60 seconds. + /// + public int CachePruneIntervalSeconds { get; set; } = 60; } diff --git a/NpgsqlRestClient/Builder.cs b/NpgsqlRestClient/Builder.cs index 8fb0b747..d38119aa 100644 --- a/NpgsqlRestClient/Builder.cs +++ b/NpgsqlRestClient/Builder.cs @@ -2806,7 +2806,10 @@ public HttpClientOptions BuildHttpClientOptions() ResponseHeadersField = _config.GetConfigStr("ResponseHeadersField", cfg) ?? "headers", ResponseContentTypeField = _config.GetConfigStr("ResponseContentTypeField", cfg) ?? "content_type", ResponseSuccessField = _config.GetConfigStr("ResponseSuccessField", cfg) ?? "success", - ResponseErrorMessageField = _config.GetConfigStr("ResponseErrorMessageField", cfg) ?? "error_message" + ResponseErrorMessageField = _config.GetConfigStr("ResponseErrorMessageField", cfg) ?? "error_message", + CacheEnabled = _config.GetConfigBool("CacheEnabled", cfg, true), + MaxCacheEntries = _config.GetConfigInt("MaxCacheEntries", cfg) ?? 10_000, + CachePruneIntervalSeconds = _config.GetConfigInt("CachePruneIntervalSeconds", cfg) ?? 60 }; if (options.Enabled) diff --git a/NpgsqlRestClient/ConfigDefaults.cs b/NpgsqlRestClient/ConfigDefaults.cs index 4c4f30c9..cbdeff67 100644 --- a/NpgsqlRestClient/ConfigDefaults.cs +++ b/NpgsqlRestClient/ConfigDefaults.cs @@ -1044,7 +1044,10 @@ private static JsonObject GetHttpClientOptionsDefaults() ["ResponseHeadersField"] = "headers", ["ResponseContentTypeField"] = "content_type", ["ResponseSuccessField"] = "success", - ["ResponseErrorMessageField"] = "error_message" + ["ResponseErrorMessageField"] = "error_message", + ["CacheEnabled"] = true, + ["MaxCacheEntries"] = 10000, + ["CachePruneIntervalSeconds"] = 60 }; } diff --git a/NpgsqlRestClient/ConfigSchemaGenerator.cs b/NpgsqlRestClient/ConfigSchemaGenerator.cs index d58b0882..114082cb 100644 --- a/NpgsqlRestClient/ConfigSchemaGenerator.cs +++ b/NpgsqlRestClient/ConfigSchemaGenerator.cs @@ -493,6 +493,9 @@ public static partial class ConfigSchemaGenerator ["NpgsqlRest:HttpClientOptions:ResponseContentTypeField"] = "Default name for the response content type field within annotated types.", ["NpgsqlRest:HttpClientOptions:ResponseSuccessField"] = "Default name for the response success field within annotated types.", ["NpgsqlRest:HttpClientOptions:ResponseErrorMessageField"] = "Default name for the response error message field within annotated types.", + ["NpgsqlRest:HttpClientOptions:CacheEnabled"] = "Global kill switch for HTTP type response caching. When false, the '@cache' directive on individual types is ignored and every request fires a fresh outbound call. Caching is opt-in per type via the '@cache ' type-comment directive.", + ["NpgsqlRest:HttpClientOptions:MaxCacheEntries"] = "Maximum number of distinct cached HTTP responses held in memory. Once full, new responses are not cached (existing entries are still served and expire normally).", + ["NpgsqlRest:HttpClientOptions:CachePruneIntervalSeconds"] = "Interval in seconds at which expired cached HTTP responses are pruned from memory.", ["NpgsqlRest:ProxyOptions"] = "Reverse proxy functionality for NpgsqlRest endpoints.\nWhen an endpoint is marked with 'proxy' annotation, incoming requests are forwarded to another URL.", ["NpgsqlRest:ProxyOptions:Enabled"] = "Enable proxy functionality for annotated endpoints.", ["NpgsqlRest:ProxyOptions:Host"] = "Base URL (host) for proxy requests (e.g., \"https://api.example.com\").\nWhen set, proxy endpoints will forward requests to this host + the original path.", diff --git a/NpgsqlRestClient/ConfigTemplate.cs b/NpgsqlRestClient/ConfigTemplate.cs index aaeafd15..df4b51c6 100644 --- a/NpgsqlRestClient/ConfigTemplate.cs +++ b/NpgsqlRestClient/ConfigTemplate.cs @@ -2803,7 +2803,22 @@ public static partial class ConfigSchemaGenerator // // Default name for the response error message field within annotated types. // - "ResponseErrorMessageField": "error_message" + "ResponseErrorMessageField": "error_message", + // + // Global kill switch for HTTP type response caching. When false, the '@cache' directive on + // individual types is ignored and every request fires a fresh outbound call. Caching is opt-in + // per type via the '@cache ' type-comment directive. + // + "CacheEnabled": true, + // + // Maximum number of distinct cached HTTP responses held in memory. Once full, new responses are + // not cached (existing entries are still served and expire normally). + // + "MaxCacheEntries": 10000, + // + // Interval in seconds at which expired cached HTTP responses are pruned from memory. + // + "CachePruneIntervalSeconds": 60 }, // diff --git a/NpgsqlRestClient/appsettings.json b/NpgsqlRestClient/appsettings.json index 14ba99a4..9b19fb5d 100644 --- a/NpgsqlRestClient/appsettings.json +++ b/NpgsqlRestClient/appsettings.json @@ -2794,7 +2794,22 @@ // // Default name for the response error message field within annotated types. // - "ResponseErrorMessageField": "error_message" + "ResponseErrorMessageField": "error_message", + // + // Global kill switch for HTTP type response caching. When false, the '@cache' directive on + // individual types is ignored and every request fires a fresh outbound call. Caching is opt-in + // per type via the '@cache ' type-comment directive. + // + "CacheEnabled": true, + // + // Maximum number of distinct cached HTTP responses held in memory. Once full, new responses are + // not cached (existing entries are still served and expire normally). + // + "MaxCacheEntries": 10000, + // + // Interval in seconds at which expired cached HTTP responses are pruned from memory. + // + "CachePruneIntervalSeconds": 60 }, // diff --git a/NpgsqlRestTests/HttpClientTypeTests/HttpClientTypeCacheTests.cs b/NpgsqlRestTests/HttpClientTypeTests/HttpClientTypeCacheTests.cs new file mode 100644 index 00000000..d204e052 --- /dev/null +++ b/NpgsqlRestTests/HttpClientTypeTests/HttpClientTypeCacheTests.cs @@ -0,0 +1,289 @@ +using WireMock.Server; +using WireMock.RequestBuilders; +using WireMock.ResponseBuilders; +using WireMock; +using WireMock.Types; +using WireMock.Util; + +namespace NpgsqlRestTests; + +public static partial class Database +{ + public static void HttpClientTypeCacheTests() + { + script.Append($@" + -- Cache test 1: basic GET with @cache, body response + create type cache_basic_api as ( + body text + ); + comment on type cache_basic_api is '@cache 60s +GET http://localhost:{WireMockFixture.Port}/api/cache-basic'; + + create function get_cache_basic( + req cache_basic_api + ) + returns text + language plpgsql + as + $$ + begin + return (req).body; + end; + $$; + + -- Cache test 2: multi-field (6) type with @cache - dedup + cache combined + create type cache_sixfield_api as ( + body text, + status_code int, + content_type text, + headers json, + success boolean, + error_message text + ); + comment on type cache_sixfield_api is '@cache 60s +GET http://localhost:{WireMockFixture.Port}/api/cache-sixfield'; + + create function get_cache_sixfield( + req cache_sixfield_api + ) + returns text + language plpgsql + as + $$ + begin + return (req).body; + end; + $$; + + -- Cache test 3: error responses must NOT be cached + create type cache_error_api as ( + body text, + status_code int, + success boolean + ); + comment on type cache_error_api is '@cache 60s +GET http://localhost:{WireMockFixture.Port}/api/cache-error'; + + create function get_cache_error( + req cache_error_api + ) + returns json + language plpgsql + as + $$ + begin + return json_build_object( + 'body', (req).body, + 'status_code', (req).status_code, + 'success', (req).success + ); + end; + $$; + + -- Cache test 4: @cache on POST is ignored (only GET is cacheable) + create type cache_post_api as ( + body text, + status_code int + ); + comment on type cache_post_api is '@cache 60s +POST http://localhost:{WireMockFixture.Port}/api/cache-post +Content-Type: application/json + +{{""ping"": true}}'; + + create function get_cache_post( + req cache_post_api + ) + returns text + language plpgsql + as + $$ + begin + return (req).body; + end; + $$; + + -- Cache test 5: short TTL for expiry verification + create type cache_ttl_api as ( + body text + ); + comment on type cache_ttl_api is '@cache 1s +GET http://localhost:{WireMockFixture.Port}/api/cache-ttl'; + + create function get_cache_ttl( + req cache_ttl_api + ) + returns text + language plpgsql + as + $$ + begin + return (req).body; + end; + $$; +"); + } +} + +[Collection("TestFixture")] +public class HttpClientTypeCacheTests : IClassFixture +{ + private readonly TestFixture _test; + private readonly WireMockServer _server; + + public HttpClientTypeCacheTests(TestFixture test, WireMockFixture wireMock) + { + _test = test; + _server = wireMock.Server; + _server.Reset(); + } + + // A cached GET fires the outbound call once; the second request is served from the cache. The + // callback embeds its invocation count in the body, so a cache hit returns the SAME body the + // first call produced - proving the second response is the cached one, not a fresh fetch. + [Fact] + public async Task Test_cached_get_fires_outbound_call_once() + { + int calls = 0; + _server + .Given(Request.Create().WithPath("/api/cache-basic").UsingGet()) + .RespondWith(Response.Create().WithCallback(req => + { + int n = Interlocked.Increment(ref calls); + return new ResponseMessage + { + StatusCode = 200, + BodyData = new BodyData { BodyAsString = $"cache-basic-call-{n}", DetectedBodyType = BodyType.String } + }; + })); + + using var first = await _test.Client.GetAsync("/api/get-cache-basic/"); + var firstContent = await first.Content.ReadAsStringAsync(); + using var second = await _test.Client.GetAsync("/api/get-cache-basic/"); + var secondContent = await second.Content.ReadAsStringAsync(); + + first.StatusCode.Should().Be(HttpStatusCode.OK); + second.StatusCode.Should().Be(HttpStatusCode.OK); + firstContent.Should().Be("cache-basic-call-1"); + secondContent.Should().Be("cache-basic-call-1"); // served from cache, not "call-2" + calls.Should().Be(1); + } + + // A 6-field composite type combines the dedup fix (one call per request, not 6) with caching + // (one call across requests). Two requests against a 6-field cached type → exactly one call. + [Fact] + public async Task Test_cached_multi_field_type_fires_one_call_across_requests() + { + int calls = 0; + _server + .Given(Request.Create().WithPath("/api/cache-sixfield").UsingGet()) + .RespondWith(Response.Create().WithCallback(req => + { + Interlocked.Increment(ref calls); + return new ResponseMessage + { + StatusCode = 200, + BodyData = new BodyData { BodyAsString = "sixfield-body", DetectedBodyType = BodyType.String } + }; + })); + + using var first = await _test.Client.GetAsync("/api/get-cache-sixfield/"); + var firstContent = await first.Content.ReadAsStringAsync(); + using var second = await _test.Client.GetAsync("/api/get-cache-sixfield/"); + + first.StatusCode.Should().Be(HttpStatusCode.OK); + second.StatusCode.Should().Be(HttpStatusCode.OK); + firstContent.Should().Be("sixfield-body"); + calls.Should().Be(1); + } + + // A failed (non-2xx) response must not be cached, so a transient upstream error is not pinned for + // the whole TTL. First call returns 500, second returns 200 - the second must reach upstream and + // observe the 200, and both calls must hit the server. + [Fact] + public async Task Test_error_response_is_not_cached() + { + int calls = 0; + _server + .Given(Request.Create().WithPath("/api/cache-error").UsingGet()) + .RespondWith(Response.Create().WithCallback(req => + { + int n = Interlocked.Increment(ref calls); + return n == 1 + ? new ResponseMessage + { + StatusCode = 500, + BodyData = new BodyData { BodyAsString = "boom", DetectedBodyType = BodyType.String } + } + : new ResponseMessage + { + StatusCode = 200, + BodyData = new BodyData { BodyAsString = "recovered", DetectedBodyType = BodyType.String } + }; + })); + + using var first = await _test.Client.GetAsync("/api/get-cache-error/"); + var firstContent = await first.Content.ReadAsStringAsync(); + using var second = await _test.Client.GetAsync("/api/get-cache-error/"); + var secondContent = await second.Content.ReadAsStringAsync(); + + firstContent.Should().Contain("\"status_code\" : 500"); + firstContent.Should().Contain("\"success\" : false"); + secondContent.Should().Contain("\"status_code\" : 200"); + secondContent.Should().Contain("\"success\" : true"); + calls.Should().Be(2); // the 500 was not cached + } + + // @cache on a non-GET method is ignored (warned at startup), so POST fires every request. + [Fact] + public async Task Test_cache_directive_ignored_for_post() + { + int calls = 0; + _server + .Given(Request.Create().WithPath("/api/cache-post").UsingPost()) + .RespondWith(Response.Create().WithCallback(req => + { + Interlocked.Increment(ref calls); + return new ResponseMessage + { + StatusCode = 200, + BodyData = new BodyData { BodyAsString = "posted", DetectedBodyType = BodyType.String } + }; + })); + + using var first = await _test.Client.GetAsync("/api/get-cache-post/"); + using var second = await _test.Client.GetAsync("/api/get-cache-post/"); + + first.StatusCode.Should().Be(HttpStatusCode.OK); + second.StatusCode.Should().Be(HttpStatusCode.OK); + calls.Should().Be(2); // POST is never cached + } + + // After the TTL elapses, the entry expires and the next request re-fetches. + [Fact] + public async Task Test_cached_response_expires_after_ttl() + { + int calls = 0; + _server + .Given(Request.Create().WithPath("/api/cache-ttl").UsingGet()) + .RespondWith(Response.Create().WithCallback(req => + { + Interlocked.Increment(ref calls); + return new ResponseMessage + { + StatusCode = 200, + BodyData = new BodyData { BodyAsString = "ttl-body", DetectedBodyType = BodyType.String } + }; + })); + + using var first = await _test.Client.GetAsync("/api/get-cache-ttl/"); // miss -> 1 call + using var second = await _test.Client.GetAsync("/api/get-cache-ttl/"); // hit -> still 1 + calls.Should().Be(1); + + await Task.Delay(TimeSpan.FromSeconds(2)); // TTL is 1s; generous margin + + using var third = await _test.Client.GetAsync("/api/get-cache-ttl/"); // expired -> 2 calls + third.StatusCode.Should().Be(HttpStatusCode.OK); + calls.Should().Be(2); + } +} diff --git a/NpgsqlRestTests/HttpClientTypeTests/ParseHttpTypeDefinitionTests.cs b/NpgsqlRestTests/HttpClientTypeTests/ParseHttpTypeDefinitionTests.cs index 9a05221b..5cb76b3d 100644 --- a/NpgsqlRestTests/HttpClientTypeTests/ParseHttpTypeDefinitionTests.cs +++ b/NpgsqlRestTests/HttpClientTypeTests/ParseHttpTypeDefinitionTests.cs @@ -1016,4 +1016,148 @@ public void NeedsParsing_handles_nested_json_with_placeholder() result.Should().NotBeNull(); result!.NeedsParsing.Should().BeTrue(); } + + // No @cache directive: caching is off and duration is null. + [Fact] + public void Cache_disabled_by_default() + { + var result = _parser.ParseHttpTypeDefinition("GET https://api.example.com/data"); + + result.Should().NotBeNull(); + result!.CacheEnabled.Should().BeFalse(); + result.CacheDuration.Should().BeNull(); + } + + [Theory] + [InlineData("cache 60s")] + [InlineData("cache=60s")] + [InlineData("cache: 60s")] + [InlineData("@cache 60s")] + public void Parses_cache_directive_with_interval(string directive) + { + var input = $"{directive}\nGET https://api.example.com/data"; + + var result = _parser.ParseHttpTypeDefinition(input); + + result.Should().NotBeNull(); + result!.CacheEnabled.Should().BeTrue(); + result.CacheDuration.Should().Be(TimeSpan.FromSeconds(60)); + } + + [Fact] + public void Parses_cache_directive_with_timespan_format() + { + var input = """ + @cache 00:05:00 + GET https://api.example.com/data + """; + + var result = _parser.ParseHttpTypeDefinition(input); + + result.Should().NotBeNull(); + result!.CacheEnabled.Should().BeTrue(); + result.CacheDuration.Should().Be(TimeSpan.FromMinutes(5)); + } + + // Bare @cache (no interval): enabled, no expiration. + [Fact] + public void Parses_bare_cache_directive_as_enabled_no_expiry() + { + var input = """ + @cache + GET https://api.example.com/data + """; + + var result = _parser.ParseHttpTypeDefinition(input); + + result.Should().NotBeNull(); + result!.CacheEnabled.Should().BeTrue(); + result.CacheDuration.Should().BeNull(); + } + + // @cache on a non-GET method is ignored (only GET is cacheable). + [Theory] + [InlineData("POST")] + [InlineData("PUT")] + [InlineData("PATCH")] + [InlineData("DELETE")] + public void Cache_directive_ignored_for_non_get(string method) + { + var input = $"@cache 60s\n{method} https://api.example.com/data"; + + var result = _parser.ParseHttpTypeDefinition(input); + + result.Should().NotBeNull(); + result!.CacheEnabled.Should().BeFalse(); + result.CacheDuration.Should().BeNull(); + } + + // Directives may appear AFTER the request line and headers, not only before it. The docs show + // them in this position; the parser recognizes them in the header section too. + [Fact] + public void Parses_timeout_directive_after_headers() + { + var input = """ + GET https://api.example.com/data + Accept: application/json + @timeout 30s + """; + + var result = _parser.ParseHttpTypeDefinition(input); + + result.Should().NotBeNull(); + result!.Timeout.Should().Be(TimeSpan.FromSeconds(30)); + result.Headers.Should().ContainKey("Accept"); + } + + [Fact] + public void Parses_retry_directive_after_headers() + { + var input = """ + GET https://api.example.com/data + Accept: application/json + @retry_delay 1s, 2s on 429, 503 + """; + + var result = _parser.ParseHttpTypeDefinition(input); + + result.Should().NotBeNull(); + result!.RetryDelays.Should().Equal(TimeSpan.FromSeconds(1), TimeSpan.FromSeconds(2)); + result.RetryOnStatusCodes.Should().BeEquivalentTo(new[] { 429, 503 }); + result.Headers.Should().ContainKey("Accept"); + } + + [Fact] + public void Parses_cache_directive_after_headers() + { + var input = """ + GET https://api.example.com/data + Accept: application/json + @cache 5m + """; + + var result = _parser.ParseHttpTypeDefinition(input); + + result.Should().NotBeNull(); + result!.CacheEnabled.Should().BeTrue(); + result.CacheDuration.Should().Be(TimeSpan.FromMinutes(5)); + result.Headers.Should().ContainKey("Accept"); + } + + // A real header whose name merely starts with a directive keyword but isn't one (e.g. + // "Cache-Control") is still treated as a header, not a directive. + [Fact] + public void Cache_control_header_is_not_treated_as_cache_directive() + { + var input = """ + GET https://api.example.com/data + Cache-Control: no-cache + """; + + var result = _parser.ParseHttpTypeDefinition(input); + + result.Should().NotBeNull(); + result!.CacheEnabled.Should().BeFalse(); + result.Headers.Should().ContainKey("Cache-Control"); + } } diff --git a/changelog/v3.17.1.md b/changelog/v3.17.1.md deleted file mode 100644 index 3d659752..00000000 --- a/changelog/v3.17.1.md +++ /dev/null @@ -1,45 +0,0 @@ -# Changelog v3.17.1 - -## Version [3.17.1](https://github.com/NpgsqlRest/NpgsqlRest/tree/3.17.1) - -[Full Changelog](https://github.com/NpgsqlRest/NpgsqlRest/compare/3.17.0...3.17.1) - -Patch release that fixes **duplicate outbound HTTP calls for HTTP Custom Types on database-function endpoints**. - -## The bug - -An endpoint backed by a **database function/procedure** whose parameter is an HTTP Custom Type fired **one outbound HTTP call per field of the type** on every inbound request. A 4-field type made 4 identical calls; a 6-field type made 6 — multiplying latency and load on the target service. SQL-file endpoints were not affected. - -## Cause - -A composite function parameter is expanded into one parameter **per field**, and each of those expanded parameters carries the same `TypeDescriptor.CustomType` (the HTTP type name). The type name was therefore collected once per field, so the per-request list of HTTP types to invoke contained the same name N times. The firing loop in `HttpClientTypeHandler.InvokeAllAsync` then called `InvokeAsync` once per entry, with no dedup — while the fill loop immediately below resolves handlers by **distinct** type name out of a dictionary. The design already assumes exactly one call per distinct type; the firing loop just failed to match, so the extra calls were made and their handlers discarded. - -SQL-file endpoints keep the HTTP type as a single composite parameter (`CustomTypeName` null), so the name was collected once → exactly one call, which is why they were unaffected. - -## What changed - -A one-line guard in `HttpClientTypeHandler.InvokeAllAsync` makes the firing loop request each distinct HTTP type once: - -```csharp -foreach (var typeName in typeNames) -{ - if (handlers.ContainsKey(typeName)) continue; // fire once per distinct type - if (HttpClientTypes.Definitions.TryGetValue(typeName, out var definition)) - { - var handler = new HttpClientTypeHandler(definition, replacements); - handlers[typeName] = handler; - tasks.Add((typeName, handler, handler.InvokeAsync(cancellationToken))); - } -} -``` - -This is the single choke point for firing outbound calls and reuses the same dictionary the fill loop already keys on, so it is robust regardless of how the parameter list is built. The established contract is preserved: one call per **distinct** HTTP type, shared from one response — two parameters referencing the same type still share a single call, and two different types remain two separate calls. - -## Tests - -Two regression tests added (`NpgsqlRestTests/HttpClientTypeTests/HttpClientTypeTests.cs`) that count **actual** outbound calls via a WireMock response callback (the prior suite asserted response content but never call counts): - -- A 6-field HTTP type fires **exactly one** outbound call (was 6 before the fix). -- Two distinct HTTP types in one function each fire **exactly one** call. - -Verified against the pre-fix code: the multi-field test failed with `Expected calls to be 1, but found 6`, reproducing the reported behavior. All 46 HTTP-client-type tests pass with the fix. diff --git a/changelog/v3.18.0.md b/changelog/v3.18.0.md new file mode 100644 index 00000000..33042a0a --- /dev/null +++ b/changelog/v3.18.0.md @@ -0,0 +1,53 @@ +# Changelog v3.18.0 + +## Version [3.18.0](https://github.com/NpgsqlRest/NpgsqlRest/tree/3.18.0) + +[Full Changelog](https://github.com/NpgsqlRest/NpgsqlRest/compare/3.17.0...3.18.0) + +The headline of this release is **HTTP Custom Type response caching** — outbound HTTP calls made by HTTP Custom Types can now be cached and reused, eliminating repeated calls to the same upstream within a configurable time window. The release also fixes a duplicate-outbound-call bug for HTTP types on database-function endpoints. + +## New Features + +### HTTP Custom Type response caching — `@cache` directive + +An HTTP Custom Type can now opt into response caching with a `@cache` directive in its type comment, alongside the existing `@timeout` and `@retry_delay` directives. Directives appear **before** the request line: + +```sql +comment on type books_api is '@cache 5m +GET https://books.toscrape.com/'; +``` + +A cached type fires **one outbound call** for a given request shape; subsequent matching requests are served from the in-memory cache until the TTL elapses. For a type with no per-request placeholders (a constant URL/headers/body), that means a single shared upstream call per TTL window across the whole application — instead of one call per inbound request. + +**Behavior and safety rules:** + +- **Opt-in, GET-only.** Caching is enabled per type by `@cache`. A `@cache` directive on any non-GET method is ignored with a startup warning — caching a mutating call is almost always a mistake. +- **TTL.** `@cache ` accepts the same formats as `@timeout` (`5m`, `30s`, `1h`, `00:05:00`, or a bare number of seconds). A bare `@cache` (no interval) caches with no expiration (until the process restarts) and warns. +- **Success-only.** Only successful (2xx) responses are cached, so a transient upstream failure is never pinned for the whole TTL — the next request re-fetches. +- **Stampede protection.** A burst of concurrent requests for the same cache key coalesces into a **single** outbound call; the rest await the in-flight result (same `Lazy` coalescing model as the routine cache). +- **Cache key** = HTTP method + resolved URL + resolved content-type + resolved headers + resolved body. Placeholders are resolved first, so per-request values vary the key naturally. + +**Configuration** (`HttpClientOptions`): + +- `CacheEnabled` (default `true`) — global kill switch. When `false`, `@cache` directives are ignored and every request fires a fresh call. +- `MaxCacheEntries` (default `10000`) — bounds memory; once full, new responses are not cached (existing entries still serve and expire normally). +- `CachePruneIntervalSeconds` (default `60`) — how often expired entries are pruned. + +## Fixes + +### HTTP Custom Type request fired once per composite field on database-function endpoints + +An endpoint backed by a **database function/procedure** whose parameter is an HTTP Custom Type fired **one outbound HTTP call per field of the type** on every inbound request (a 4-field type → 4 identical calls; a 6-field type → 6), multiplying latency and load on the target. SQL-file endpoints were not affected. + +**Cause.** A composite function parameter is expanded into one parameter per field, each carrying the same `TypeDescriptor.CustomType` (the HTTP type name). The per-request list of HTTP types therefore held the same name N times, and the firing loop in `HttpClientTypeHandler.InvokeAllAsync` called `InvokeAsync` once per entry — while the fill loop immediately below resolves handlers by **distinct** type name. The design already assumes one call per distinct type; the firing loop just failed to match. + +**Fix.** A guard in the firing loop requests each distinct HTTP type once, reusing the dictionary the fill loop already keys on. The established contract is preserved: one call per **distinct** HTTP type, shared from one response — two parameters referencing the same type still share a single call, and two different types remain two separate calls. + +### HTTP type directives after the headers were silently ignored + +The `@timeout`, `@retry_delay`, and `@cache` directives are now parsed both **before** the request line and **after** the headers. Previously only the leading position (before the request line) was recognized, so a directive placed after the headers — as the documentation and examples showed — was silently dropped (e.g. a `@timeout` that never applied). Both placements are now equivalent. Real HTTP headers are unaffected: a header whose name merely starts with a directive keyword (e.g. `Cache-Control`) is still treated as a header. + +## Tests + +- Regression tests count **actual** outbound calls via WireMock response callbacks (the prior suite asserted content but never call counts): a 6-field type fires exactly one call (was 6), and two distinct types fire one call each. +- Caching tests cover: cache hit reduces to one call, 6-field dedup + caching combined, error responses not cached, `@cache` ignored on POST, and TTL expiry. Parse-level tests cover the `@cache` directive forms and GET-only enforcement. diff --git a/npm/package.json b/npm/package.json index 859024b3..621ecdbe 100644 --- a/npm/package.json +++ b/npm/package.json @@ -1,6 +1,6 @@ { "name": "npgsqlrest", - "version": "3.17.1", + "version": "3.18.0", "description": "Automatic REST API for PostgreSQL Databases Client Build", "scripts": { "postinstall": "node postinstall.js", diff --git a/version.txt b/version.txt index 0caba260..ae561550 100644 --- a/version.txt +++ b/version.txt @@ -1 +1 @@ -3.17.1 \ No newline at end of file +3.18.0 \ No newline at end of file From e036f3d322e900e7a36c5d65ae96f4f48b32602e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Vedran=20Bilopavlovi=C4=87?= Date: Tue, 23 Jun 2026 12:36:52 +0200 Subject: [PATCH 05/25] upgrade references --- NpgsqlRestClient/NpgsqlRestClient.csproj | 4 ++-- NpgsqlRestTests/NpgsqlRestTests.csproj | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/NpgsqlRestClient/NpgsqlRestClient.csproj b/NpgsqlRestClient/NpgsqlRestClient.csproj index 03e0a02b..6beaecf6 100644 --- a/NpgsqlRestClient/NpgsqlRestClient.csproj +++ b/NpgsqlRestClient/NpgsqlRestClient.csproj @@ -15,14 +15,14 @@ - + - + diff --git a/NpgsqlRestTests/NpgsqlRestTests.csproj b/NpgsqlRestTests/NpgsqlRestTests.csproj index fcd52b6b..a2a10468 100644 --- a/NpgsqlRestTests/NpgsqlRestTests.csproj +++ b/NpgsqlRestTests/NpgsqlRestTests.csproj @@ -16,9 +16,9 @@ - + - + runtime; build; native; contentfiles; analyzers; buildtransitive From c74a4b5db331f47f62f3707138ca2e40bae0b8a1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Vedran=20Bilopavlovi=C4=87?= Date: Tue, 23 Jun 2026 14:06:57 +0200 Subject: [PATCH 06/25] feat(proxy): forward all automatic parameters to proxy upstream consistently (v3.18.1) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit When an endpoint is a proxy, all server-filled parameters now forward to the upstream through one unified path: user claims, IP address, HTTP Custom Type fields, and resolved-parameter expressions. Placement mirrors the endpoint's own signature, NOT the HTTP verb: - a @body_parameter_name param carries the raw request body; - otherwise RequestParamType decides — QueryString appends to the proxy query string, BodyJson merges into the proxy JSON body (typed: numbers / booleans / embedded json / strings) when the method carries a body. Additive: the verbatim incoming request is still forwarded; the automatic params are added on top so the upstream receives the same parameter set the routine would. The previous claim/IP-only query append in BuildTargetUrl is removed and folded into the unified mechanism. Behavior change: claims/IP now follow RequestParamType (unchanged for GET/QueryString — still query; for BodyJson endpoints they now go to the body). Notes: body merge only for JSON content types (multipart/non-JSON forwarded verbatim); only expanded per-field HTTP-type params forwarded. Tests cover GET-query, POST-body (typed), param_type-query on POST, and resolved-param forwarding; existing claim/IP proxy tests pass unchanged. Full suite green (2288). See changelog/v3.18.1.md. Also adds a Claude Code skill bundle (.claude/skills/npgsqlrest/) — a usage guide plus full annotation and configuration references generated from --annotations/--config — and a README section on installing it. --- .claude/skills/npgsqlrest/SKILL.md | 238 ++ .../npgsqlrest/annotations-reference.md | 355 ++ .../npgsqlrest/configuration-reference.jsonc | 2949 +++++++++++++++++ NpgsqlRest/Proxy/ProxyRequestHandler.cs | 226 +- .../ProxyTests/ProxyHttpTypeProbeTest.cs | 272 ++ README.md | 16 + changelog/v3.18.0.md | 2 +- changelog/v3.18.1.md | 40 + npm/package.json | 2 +- version.txt | 2 +- 10 files changed, 4066 insertions(+), 36 deletions(-) create mode 100644 .claude/skills/npgsqlrest/SKILL.md create mode 100644 .claude/skills/npgsqlrest/annotations-reference.md create mode 100644 .claude/skills/npgsqlrest/configuration-reference.jsonc create mode 100644 NpgsqlRestTests/ProxyTests/ProxyHttpTypeProbeTest.cs create mode 100644 changelog/v3.18.1.md diff --git a/.claude/skills/npgsqlrest/SKILL.md b/.claude/skills/npgsqlrest/SKILL.md new file mode 100644 index 00000000..2cb12cd5 --- /dev/null +++ b/.claude/skills/npgsqlrest/SKILL.md @@ -0,0 +1,238 @@ +--- +name: npgsqlrest +description: Build and modify REST APIs with NpgsqlRest — exposing PostgreSQL as HTTP endpoints from two sources (database functions/procedures/tables/views, and plain .sql files), driven by SQL comment annotations (no C# needed). Use when working in an NpgsqlRest project: writing or changing endpoint SQL (functions or .sql files), comment annotations (HTTP routing, authorize, cached, proxy, HTTP Custom Types, SSE, MCP, upload), appsettings.json config, or running/troubleshooting the `npgsqlrest` client. +--- + +# Working with NpgsqlRest + +NpgsqlRest auto-generates a REST API from PostgreSQL. You write SQL and **annotate it with comments**; NpgsqlRest turns each annotated object into an HTTP endpoint at startup. There is **no controller/model/mapping layer and no C# to write** — behavior is declared in SQL comments and `appsettings.json`. + +## Mental model (read this first) + +- An endpoint comes from one of **two sources** (see below): a **database routine** (function/procedure/table/view) or a **`.sql` file**. Both use the **same annotation vocabulary**; only where you write the annotation differs. +- An endpoint is created when its comment/file carries an `HTTP` tag (or a plugin tag like `mcp`), under the default `CommentsMode: OnlyAnnotated`. +- The result is serialized to JSON automatically (a set → array; `@single` → one object; `@raw` → plain text; multi-statement `.sql` → object keyed per statement). +- Auth, caching, rate limiting, headers, validation, proxy, outbound HTTP, SSE, MCP — all are **comment annotations**. +- `appsettings.json` configures the server, connection, auth, and which sources/schemas/objects are exposed. + +## Authoritative sources — prefer these over guessing + +The annotation/config surface is large and version-dependent. In order of preference: + +1. **Bundled with this skill** (offline, complete — these files sit next to this `SKILL.md`): + - `annotations-reference.md` — every comment annotation (name, aliases, syntax, description). + - `configuration-reference.jsonc` — the full `appsettings.json` with every option and inline comments. +2. **The installed binary** (authoritative for the exact version in use): + ```bash + npgsqlrest --annotations # every supported annotation, as JSON + npgsqlrest --config # full default appsettings.json (every option, commented) + npgsqlrest --validate # test DB connectivity + that endpoints build (incl. .sql Describe) + npgsqlrest --version + ``` +3. **Online:** docs at (e.g. `/guide/annotations`, `/guide/sql-files`, `/annotations/`, `/config/
`); source & issues at . + +The two bundled files were generated from `--annotations` / `--config`. If your installed `npgsqlrest` version differs, trust the binary (or regenerate them with those commands). + +## The two endpoint sources + +Both are independently enabled; you can use either or **both together** (a `.sql` file can call a function; an HTTP Custom Type can reference any endpoint regardless of source). + +### 1. Database routines (functions / procedures / tables / views) + +Annotations live in the PostgreSQL **object comment**. NpgsqlRest reads the catalog at startup. Nothing extra to enable beyond schema selection. + +```sql +create function get_users(_active boolean default true) +returns setof users language sql as $$ + select * from users where active = _active; +$$; + +comment on function get_users(boolean) is ' +HTTP GET /api/users +@authorize +@cached _active +@cache_expires_in 5m'; +``` + +- Default path derives from the routine name (kebab-cased, under `UrlPathPrefix`, default `/api`). +- Parameters auto-map by name; `@param` renames/retypes/defaults them. +- Param location (query vs JSON body) is governed by `@request_param_type` / defaults, **not** the HTTP verb (a POST can use query params). +- **Project helper pattern:** some projects wrap the comment in a helper like `call myschema.annotate('schema.func', 'HTTP POST', 'authorize manager', ...)` that just builds the `comment on ...` string. Follow the project's convention — the mechanism is always the routine comment. + +### 2. SQL files (`.sql`) + +Annotations live in **leading line comments** (`-- ...` or `/* ... */`) in a `.sql` file. No `CREATE FUNCTION`, no `COMMENT ON`. Enable the source and point it at a glob: + +```json +{ "NpgsqlRest": { "SqlFileSource": { "Enabled": true, "FilePattern": "sql/**/*.sql" } } } +``` + +```sql +-- sql/get-reports.sql +-- HTTP GET +-- @param $1 from_date +-- @param $2 to_date +-- @authorize +select id, title, created_at +from reports +where created_at between $1 and $2; +``` + +- **Filename → path**: `get-reports.sql` → `/api/get-reports`. +- **Startup Describe = static type checking.** Each statement is parsed/described against the live DB (no execution); SQL errors fail startup (`ErrorMode: Exit`; set `Skip` to log-and-continue). This catches `column does not exist` etc. before serving. +- **Parameters are positional** (`$1, $2`). `@param $N name [type] [default ...]` names/retypes/defaults them (positional params have no native DEFAULT). `@define_param name [type]` creates a *virtual* param (for placeholders/claims; not bound to SQL). +- **Verb inference** when no `HTTP` tag: `SELECT`→GET, `INSERT`→PUT, `UPDATE`→POST, `DELETE`→DELETE, `DO`→POST (most destructive wins). Explicit `-- HTTP POST` overrides. +- **Multi-command files** (statements split on `;`) run in **one batch / round-trip** and return a JSON object keyed per statement. Positional annotations apply to the *next* statement (or inline after `;`): + - `@result name` — name the result key (default `result1`, `result2`, …) + - `@single` — that statement returns one object instead of an array + - `@skip` — run it but exclude from the response + - `@void` — whole endpoint returns **204 No Content** (all statements run for side effects only) +- **`@returns `** — skip Describe for a statement and resolve columns from a type instead. Needed when a statement references objects that don't exist at startup (e.g. a temp table built in a `DO` block). +- **`DO` block limits** (PostgreSQL, not NpgsqlRest): `DO` blocks can't take `$N` params or return values. Bridge with `set_config(key, $1, true)` + `current_setting()`, or a temp table + `@returns`. For real procedural logic, prefer a function. +- Config knobs: `CommentScope` (`All` | `Header`), `UnnamedSingleColumnSet` (single-column → flat array), `ResultPrefix`, `SkipNonQueryCommands`. + +### Functions vs SQL files — when to use which + +- **SQL files**: declarative queries, multi-statement workflows, teams that prefer plain `.sql` over DDL. +- **Functions**: procedural logic, native params/returns, `assert`-based tests in repeatable migrations, optimizer hints (`STABLE`/`COST`/`ROWS`), overloading. + +Everything below (annotations, HTTP types, proxy, caching, auth, SSE, MCP) works **identically in both sources**. + +## Annotation cheat-sheet (grouped) + +Same annotations apply to functions and `.sql` files. Confirm exact syntax/aliases with `npgsqlrest --annotations`. + +**Routing / exposure** +- `HTTP [METHOD] [/path]` — expose. `@path /x` — override path. `@disabled`/`@enabled`. `@internal` — exists but no public route (reachable via proxy / HTTP-type self-call). `@tags a,b`, `@openapi …`. + +**Auth** +- `@authorize` (any authenticated) / `@authorize role1, role2`. `@allow_anonymous` (aliases `@anonymous`, `@anon`). `@login` / `@logout`. `@user_context` (claims → PG context + headers), `@user_parameters` (claims → params). `@security_sensitive` (keep param values out of logs). + +**Params / request** +- `@param $1 name [type] [default …]` (rename/retype/default). `@define_param name [type]` (virtual, SQL-file). `@request_param_type query_string | body_json`. `@body_parameter_name _x` (one param = raw body). `@request_headers_mode parameter` + `@request_headers_parameter_name _headers`. Resolved param: a line `_token = select api_token from tokens where user_name = {_name}` fills `_token` server-side (client can't override). + +**Response shaping** +- `@single`, `@raw` (+ `@separator`, `@new_line`, `@columns`), `@result ` (multi-command key), `@skip`, `@void` (204), `@nested` (keep composites nested), `@returns ` (SQL-file, skip Describe). Header lines like `Content-Type: text/csv`. + +**Caching** (server-side response cache) +- `@cached [p1, p2, …]` — **always list the key params explicitly** (a bare `@cached` keys on the endpoint only). `@cache_expires_in 30s|5m|1h` (alias `@cache_expires`). `@cache_profile name` (backend + defaults from `CacheOptions:Profiles`). + +**Other** +- `@rate_limiter_policy name`. `@command_timeout 30s`. `@connection Name` (multi-connection). `@buffer_rows N`. `@validate _email using required, email`. `@error_code_policy 23505 -> 409`. `@upload [for csv|excel|file_system|large_object]`. + +## HTTP Custom Types (outbound HTTP from SQL) + +A **composite type** whose comment defines an outbound HTTP request. When a routine (or `.sql` param) is of that type, NpgsqlRest performs the call **before** running the endpoint and fills the composite fields. + +```sql +create type books_api as (body text, status_code int, success boolean, error_message text); + +-- directives go BEFORE the request line +comment on type books_api is '@timeout 30s +@retry_delay 1s, 2s on 429, 503 +@cache 5m +GET https://books.toscrape.com/ +Accept: text/html'; +``` + +- Directives `@timeout` / `@retry_delay` / `@cache` may appear before the request line OR after the headers (both work since 3.18.0; before is safest). +- Response fields (names configurable in `HttpClientOptions`): `body`, `status_code`, `content_type`, `headers` (json), `success`, `error_message`. +- `@cache ` (3.18.0+): caches the outbound response — **GET only, 2xx only**, stampede-coalesced; globally toggled by `HttpClientOptions.CacheEnabled`. +- Placeholders `{name}` in URL/headers/body substitute from params, resolved-param expressions, allow-listed env vars. +- A relative URL (`GET /api/other`) is a **self-call** to another endpoint with no HTTP round-trip — give a function several HTTP-type params and they fire concurrently. +- Requires `HttpClientOptions.Enabled = true`. + +## Proxy (reverse proxy endpoints) + +`@proxy [METHOD] [host]` forwards upstream (target = `host + incoming path + query`; host from annotation or `ProxyOptions.Host`). + +- **Passthrough** (no proxy-response params): function body is **not executed**; upstream response streamed back; no DB connection. +- **Transform** (routine declares `_proxy_status_code int`, `_proxy_body text`, `_proxy_success boolean`, `_proxy_headers json`, `_proxy_content_type text`, `_proxy_error_message text`): NpgsqlRest proxies, binds the response into those params, runs the function, returns its result. +- **Automatic params forwarded upstream** (3.18.1+): user claims, IP, HTTP-Custom-Type fields, resolved-param expressions are forwarded in the endpoint's native shape (query for `QueryString` endpoints, merged JSON body for `BodyJson`), honoring `@body_parameter_name`. +- Requires `ProxyOptions.Enabled = true`. + +## SSE (server-sent events) + +A long-running routine streams progress via `RAISE INFO/NOTICE`. Annotate with `@sse` (or a project's `sse_publish`/`sse_subscribe` split) and a scope (`all` | `authorize` | `matching`). Per-recipient targeting: `raise info 'msg' using hint = format('authorize %s', _user_id)`. A subscribe-only endpoint's body never runs. A cache hit skips execution → no RAISE → no broadcast (correct). + +## MCP (Model Context Protocol) + +`@mcp [text]` exposes a routine as an MCP tool. A bare `@mcp` with **no** HTTP tag = MCP-only (no public route). `@mcp_description` / `@mcp_name` refine it. Served at `/mcp` when the `NpgsqlRest.Mcp` plugin is loaded. + +## Auth pattern (login → claims) + +A login endpoint is a routine annotated `@login` returning a status/claims row; NpgsqlRest reads configured columns and issues the cookie/token: + +```sql +-- returns (status, scheme, user_id, user_name, user_roles, message) by convention +comment on function auth_login(text, text) is 'HTTP POST +@login +@allow_anonymous +@rate_limiter_policy login_throttle +@security_sensitive'; +``` + +- `status` 200 → success (other codes → that HTTP status); `scheme` picks the auth scheme/cookie. +- Column→claim mapping in `AuthenticationOptions` (`StatusColumnName`, `SchemeColumnName`, role/name/id claim columns). +- **Inject identity from claims, never trust client-supplied IDs.** Declare params like `_user_id text = null`, `_user_roles text[] = '{}'` and map them via `AuthenticationOptions.ParameterNameClaimsMapping` / `IpAddressParameterName`. NpgsqlRest fills them from the authenticated principal. +- Use `security definer` on API functions and still permission-check inside (`assert _user_roles && array['admin']` …). + +## Configuration (appsettings.json) + +`npgsqlrest --config` prints the full annotated default. Most-used sections: + +```jsonc +{ + "ConnectionStrings": { "Default": "Host={PGHOST};Port={PGPORT};Database={PGDATABASE};Username={APP_USER};Password={APP_PASSWORD}" }, + "Urls": "http://0.0.0.0:8080", + "Auth": { "CookieAuth": true, "CookieName": "app", "CookieValid": "365 days" /* + Jwt/Bearer/Passkey/External */ }, + "NpgsqlRest": { + "IncludeSchemas": ["myapi"], + "CommentsMode": "OnlyAnnotated", + "UrlPathPrefix": "/api", + "RequiresAuthorization": true, + "KebabCaseUrls": true, "CamelCaseNames": true, + "AuthenticationOptions": { /* status/scheme/claim columns, ParameterNameClaimsMapping, IpAddressParameterName */ }, + "SqlFileSource": { "Enabled": false, "FilePattern": "sql/**/*.sql", "ErrorMode": "Exit", "CommentScope": "All" } + }, + "CacheOptions": { "Enabled": true, "Type": "Memory" /* or Redis / Hybrid */, "Profiles": { } }, + "HttpClientOptions": { "Enabled": false /* + CacheEnabled, response field names */ }, + "ProxyOptions": { "Enabled": false, "Host": null }, + "RateLimiterOptions": { "Enabled": false, "Policies": { } }, + "Log": { "MinimalLevels": { "NpgsqlRest": "Information" } } +} +``` + +- **Two sources, independently enabled:** database routines (always available via the catalog, filtered by `IncludeSchemas`/`SchemaSimilarTo`/`NameSimilarTo`/`CommentsMode`) and `NpgsqlRest:SqlFileSource` (off by default; needs `Enabled` + `FilePattern`). +- `{ENV_VAR}` placeholders are substituted from environment variables at startup. +- `--config ` loads a specific file; multiple files overlay (later wins): `npgsqlrest ./appsettings.json ./appsettings.development.json`. +- Dev override file commonly enables `Debug` logging, TypeScript client codegen, and `.http` export. + +## Running + +```bash +npgsqlrest # appsettings.json in cwd +npgsqlrest ./config/appsettings.json ./config/appsettings.development.json +npgsqlrest --connectionstrings:default="Host=localhost;Database=db;Username=postgres;Password=postgres" +npgsqlrest --log:minimallevels:npgsqlrest=debug # see every annotation parsed +``` + +Install via the GitHub release binary, `npm install -g npgsqlrest`, or the `vbilopav/npgsqlrest` Docker image. + +## Project conventions worth copying (from real projects) + +- **One schema for the API** (`IncludeSchemas: ["myapi"]`); internal helpers in another schema. +- **Organize by feature/domain**, not by migration type. Define endpoints as **repeatable migrations** (`R___.sql`) so re-running is idempotent — or as plain `.sql` files under the source glob. +- **Dev codegen:** enable the TypeScript client generator + `.http` export in the dev override file only; commit the generated client. +- **Two-layer caching:** a `Cache-Control` header for the browser + server-side `@cached`/`@cache_profile` for cross-client dedup; match the windows. +- **Cache-key discipline:** include every param that changes the result (and `_user_id` for per-user data); omit `_user_id` for shared results so all users hit one entry. Add a `_cache_bust`/`_param_hash` param to force misses after writes. + +## Gotchas + +- **`@cached` needs an explicit param list** — bare `@cached` keys on the endpoint only (a common silent bug). +- **Verb ≠ param location** — use `@request_param_type`, not the method, to reason about where params come from / are forwarded. +- **HTTP-type directives** are safest **before the request line**. +- **Passthrough proxy doesn't run the function** — declare proxy-response params (transform mode) if you need the body executed. +- **`HttpClientOptions.Enabled` / `ProxyOptions.Enabled` / `SqlFileSource.Enabled`** must be true for those features. `@cache` on a non-GET HTTP type is ignored with a warning. +- **SQL files:** annotations are in `--`/`/* */` comments; params are positional `$N` (name via `@param`); a startup Describe error fails boot unless `ErrorMode: Skip`; use `@returns` for statements over not-yet-existing objects (temp tables); `DO` blocks can't take `$N` or return values. +- After changing an annotation, re-run with `--log:minimallevels:npgsqlrest=debug` (or `--validate`) to confirm it parsed as intended. diff --git a/.claude/skills/npgsqlrest/annotations-reference.md b/.claude/skills/npgsqlrest/annotations-reference.md new file mode 100644 index 00000000..d9c612ca --- /dev/null +++ b/.claude/skills/npgsqlrest/annotations-reference.md @@ -0,0 +1,355 @@ +# NpgsqlRest — Full Annotation Reference + +Every comment annotation, generated from `npgsqlrest --annotations` (NpgsqlRest v3.18.1). +Annotations apply to both endpoint sources (database routines via `comment on`, and `.sql` files via leading `--` comments). The `@` prefix is optional. Regenerate with `npgsqlrest --annotations`. + +## `http` + +- **Aliases:** http +- **Syntax:** `http [GET|POST|PUT|DELETE] [path]` + +Enable endpoint and configure HTTP method and/or path. Required (for HTTP exposure) when CommentsMode is OnlyAnnotated (or its alias OnlyWithHttpTag). + +## `path` + +- **Aliases:** path +- **Syntax:** `path ` + +Override the endpoint URL path. + +## `param_type` + +- **Aliases:** request_param_type, param_type +- **Syntax:** `param_type [query_string|query|body_json|body]` + +Set request parameter type to query string or JSON body. + +## `authorize` + +- **Aliases:** authorize, authorized, requires_authorization +- **Syntax:** `authorize [role1, role2, ...]` + +Require authorization, optionally restricting to specific roles. + +## `allow_anonymous` + +- **Aliases:** allow_anonymous, anonymous, allow_anon, anon +- **Syntax:** `allow_anonymous` + +Allow unauthenticated access to this endpoint. + +## `login` + +- **Aliases:** login, signin +- **Syntax:** `login` + +Mark endpoint as a login/authentication endpoint. + +## `logout` + +- **Aliases:** logout, signout +- **Syntax:** `logout` + +Mark endpoint as a logout endpoint. + +## `raw` + +- **Aliases:** raw, raw_mode, raw_results +- **Syntax:** `raw` + +Return raw results without JSON formatting. + +## `separator` + +- **Aliases:** separator, raw_separator +- **Syntax:** `separator ` + +Set the value separator for raw mode output. + +## `new_line` + +- **Aliases:** new_line, raw_new_line +- **Syntax:** `new_line ` + +Set the line separator for raw mode output. + +## `columns` + +- **Aliases:** columns, names, column_names +- **Syntax:** `columns` + +Include column names as the first row in raw output. + +## `buffer_rows` + +- **Aliases:** buffer_rows, buffer +- **Syntax:** `buffer_rows ` + +Set the number of rows to buffer before sending response. + +## `cached` + +- **Aliases:** cached +- **Syntax:** `cached [param1, param2, ...]` + +Enable response caching, optionally specifying cache key parameters. + +## `cache_expires` + +- **Aliases:** cache_expires, cache_expires_in +- **Syntax:** `cache_expires ` + +Set cache expiration time (PostgreSQL interval format). + +## `cache_profile` + +- **Aliases:** cache_profile +- **Syntax:** `cache_profile ` + +Select a named cache profile defined in CacheOptions.Profiles. The profile supplies the cache backend, default expiration, default key parameters, and per-parameter skip conditions. Implies caching even without @cached. Existing @cached and @cache_expires annotations override the profile's defaults. Unknown profile names cause startup to fail. + +## `connection_name` + +- **Aliases:** connection, connection_name +- **Syntax:** `connection_name ` + +Use a specific named connection string for this endpoint. + +## `timeout` + +- **Aliases:** command_timeout, timeout +- **Syntax:** `timeout ` + +Set command execution timeout (PostgreSQL interval format). + +## `request_headers_mode` + +- **Aliases:** request_headers_mode, request_headers +- **Syntax:** `request_headers [ignore|context|parameter]` + +Control how HTTP request headers are passed to the routine. + +## `request_headers_parameter_name` + +- **Aliases:** request_headers_parameter_name, request_headers_param_name, request-headers-param-name +- **Syntax:** `request_headers_parameter_name ` + +Set the parameter name for request headers when mode is parameter. + +## `body_parameter_name` + +- **Aliases:** body_parameter_name, body_param_name +- **Syntax:** `body_parameter_name ` + +Set the parameter name for the JSON body content. + +## `response_null_handling` + +- **Aliases:** response_null_handling, response_null +- **Syntax:** `response_null [empty_string|null_literal|no_content|204]` + +Control how NULL return values are rendered in responses. + +## `query_string_null_handling` + +- **Aliases:** query_string_null_handling, query_null_handling, query_string_null, query_null +- **Syntax:** `query_null [empty_string|null_literal|ignore]` + +Control how NULL query string parameters are handled. + +## `security_sensitive` + +- **Aliases:** sensitive, security, security_sensitive +- **Syntax:** `security_sensitive` + +Mark endpoint as security-sensitive (suppresses logging of parameters). + +## `user_context` + +- **Aliases:** user_context +- **Syntax:** `user_context` + +Pass authenticated user context to the routine via connection settings. + +## `user_parameters` + +- **Aliases:** user_parameters, user_params +- **Syntax:** `user_parameters` + +Map user claims to routine parameters. + +## `upload` + +- **Aliases:** upload +- **Syntax:** `upload [for handler1, handler2, ...]` + +Enable file upload for this endpoint, optionally specifying upload handlers. + +## `param` + +- **Aliases:** parameter, param +- **Syntax:** `param is hash of | param is upload metadata | param [type] | param is [type]` + +Configure parameter behavior: hash computation, upload metadata binding, or rename/retype. Rename forms: 'param $1 user_id', 'param $1 user_id integer', 'param $1 is user_id', 'param _old_name better_name'. Works on all endpoint types. + +## `sse_path` + +- **Aliases:** sse, sse_path, sse_events_path +- **Syntax:** `sse_path [path] [on info|notice|warning]` + +Enable Server-Sent Events streaming with optional path and notice level. + +## `sse_level` + +- **Aliases:** sse_level, sse_events_level +- **Syntax:** `sse_level [info|notice|warning]` + +Set the PostgreSQL notice level for SSE events. + +## `sse_scope` + +- **Aliases:** sse_scope, sse_events_scope +- **Syntax:** `sse_scope [all|authorize|matching] [role1, role2, ...]` + +Set the broadcast scope for SSE events. + +## `basic_auth` + +- **Aliases:** basic_authentication, basic_auth +- **Syntax:** `basic_auth [username] [password]` + +Enable HTTP Basic Authentication for this endpoint. + +## `basic_auth_realm` + +- **Aliases:** basic_authentication_realm, basic_auth_realm, realm +- **Syntax:** `basic_auth_realm ` + +Set the authentication realm for Basic Auth challenges. + +## `basic_auth_command` + +- **Aliases:** basic_authentication_command, basic_auth_command, challenge_command +- **Syntax:** `basic_auth_command ` + +Set a custom SQL command for Basic Auth credential validation. + +## `retry_strategy` + +- **Aliases:** retry_strategy_name, retry_strategy, retry +- **Syntax:** `retry_strategy ` + +Apply a named retry strategy for transient database errors. + +## `rate_limiter` + +- **Aliases:** rate_limiter_policy_name, rate_limiter_policy, rate_limiter +- **Syntax:** `rate_limiter ` + +Apply a named rate limiting policy to this endpoint. + +## `error_code_policy` + +- **Aliases:** error_code_policy_name, error_code_policy, error_code +- **Syntax:** `error_code_policy ` + +Apply a named error code mapping policy for PostgreSQL error codes. + +## `validate` + +- **Aliases:** validate, validation +- **Syntax:** `validate using ` + +Add parameter validation using a named validation rule. + +## `proxy` + +- **Aliases:** proxy, reverse_proxy +- **Syntax:** `proxy [GET|POST|PUT|DELETE|PATCH] [host_url]` + +Configure endpoint as a reverse proxy. + +## `proxy_out` + +- **Aliases:** proxy_out, forward_proxy +- **Syntax:** `proxy_out [GET|POST|PUT|DELETE|PATCH] [host_url]` + +Execute function first, then forward result body to upstream proxy service. + +## `nested_json` + +- **Aliases:** nested, nested_json, nested_composite +- **Syntax:** `nested_json` + +Serialize composite type columns as nested JSON objects. + +## `tags` + +- **Aliases:** for, tags, tag +- **Syntax:** `for tag1, tag2, ...` + +Filter endpoint availability by tags. + +## `disabled` + +- **Aliases:** disabled +- **Syntax:** `disabled [tag1, tag2, ...]` + +Disable this endpoint, optionally only for specific tags. + +## `enabled` + +- **Aliases:** enabled +- **Syntax:** `enabled [tag1, tag2, ...]` + +Enable this endpoint, optionally only for specific tags. + +## `internal` + +- **Aliases:** internal, internal_only +- **Syntax:** `internal` + +Mark endpoint as internal-only. Not exposed as an HTTP route, only accessible via self-referencing calls (proxy, HTTP client types). + +## `encrypt` + +- **Aliases:** encrypt, encrypted, protect, protected +- **Syntax:** `encrypt [param1, param2, ...]` + +Encrypt parameter values using the default data protector before sending to PostgreSQL. Without arguments, encrypts all text parameters. + +## `decrypt` + +- **Aliases:** decrypt, decrypted, unprotect, unprotected +- **Syntax:** `decrypt [column1, column2, ...]` + +Decrypt result column values using the default data protector before returning to the client. Without arguments, decrypts all text columns. + +## `custom_parameter` + +- **Aliases:** +- **Syntax:** `key = value` + +Define a custom parameter as key-value pair (separated by =). + +## `header` + +- **Aliases:** +- **Syntax:** `Header-Name: header-value` + +Add a response header (separated by :). + +## `resultN` + +- **Aliases:** result1, result2, result3 +- **Syntax:** `resultN | resultN is ` + +Rename a result key in multi-command SQL file responses. N is the 1-based command index. Example: '@result1 validate' renames the first result from 'result1' to 'validate'. SQL file source only. + +## `define_param` + +- **Aliases:** define_param +- **Syntax:** `define_param [type]` + +Define a virtual parameter that exists for HTTP matching and claim mapping but is NOT bound to the PostgreSQL command. Useful for SQL file endpoints where you need parameters for comment placeholders or claim mapping without referencing them in SQL. Default type is text. SQL file source only. + diff --git a/.claude/skills/npgsqlrest/configuration-reference.jsonc b/.claude/skills/npgsqlrest/configuration-reference.jsonc new file mode 100644 index 00000000..f986c9e0 --- /dev/null +++ b/.claude/skills/npgsqlrest/configuration-reference.jsonc @@ -0,0 +1,2949 @@ +// NpgsqlRest — FULL configuration reference (every option, with inline comments). +// Generated verbatim from: npgsqlrest --config (NpgsqlRest v3.18.1) +// This is the source of truth for configuration. Regenerate with: npgsqlrest --config + +{ + // + // The application name used to set the application name property in connection string by "NpgsqlRest.SetApplicationNameInConnection" or the "NpgsqlRest.UseJsonApplicationName" settings. + // It is the name of the top-level directory if set to null. + // + "ApplicationName": null, + + // + // Production or Development + // + "EnvironmentName": "Production", + + // + // Specify the urls the web host will listen on. See https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.hosting.hostingabstractionswebhostbuilderextensions.useurls?view=aspnetcore-8.0 + // + "Urls": "http://localhost:8080", + + // + // Logs at startup, format placeholders: + // {time} - startup time + // {urls} - listening on urls + // {version} - current version + // {environment} - EnvironmentName + // {application} - ApplicationName + // + // Note: This message is logged at Information level. To disable this message, set to empty string. + // + "StartupMessage": "Started in {time}, listening on {urls}, version {version}", + + // + // Configuration settings + // + "Config": { + // + // Add the environment variables to configuration. + // When enabled, environment variables will override the settings in this configuration file but can be overridden by command line arguments. + // Complex hierarchical keys can be defined using double underscore as a separator. + // For example, "ConnectionStrings__Default" environment variable will override the "ConnectionStrings.Default" setting in this configuration file. + // + "AddEnvironmentVariables": false, + // + // When set, configuration values will be parsed for environment variables in the format {ENV_VAR_NAME} + // and replaced with the value of the environment variable when available. + // + "ParseEnvironmentVariables": true, + // + // Path to a .env file containing environment variables. + // When AddEnvironmentVariables or ParseEnvironmentVariables is true and this file exists, + // variables from this file will be loaded and made available for configuration parsing. + // Format: KEY=VALUE (one per line) + // + "EnvFile": null, + // + // Validate configuration keys against known defaults at startup. + // "Ignore" - no validation + // "Warning" - log warnings for unknown keys, continue startup (default) + // "Error" - log errors for unknown keys and exit + // + "ValidateConfigKeys": "Warning" + }, + + // + // List of named connection strings to PostgreSQL databases. + // The "Default" connection string is used when no connection name is specified. + // For connection string definition see https://www.npgsql.org/doc/connection-string-parameters.html + // + "ConnectionStrings": { + "Default": "Host={PGHOST};Port=5432;Database={PGDATABASE};Username={PGUSER};Password={PGPASSWORD}" + }, + + // + // Additional connection settings and options. + // + "ConnectionSettings": { + // + // Sets the ApplicationName connection property in the connection string to the value of the ApplicationName configuration. + // Note: This option is ignored if the UseJsonApplicationName option is enabled. + // + "SetApplicationNameInConnection": true, + // + // Sets the ApplicationName connection property dynamically on every request in the following format: + // {"app":"","uid":"","id":""} + // Note: The ApplicationName connection property is limited to 64 characters. + // + "UseJsonApplicationName": false, + // + // Test any connection string before initializing the application and using it. The connection string is tested by opening and closing the connection. + // + "TestConnectionStrings": true, + // + // Connection open retry options. + // + "RetryOptions": { + "Enabled": true, + // + // Retry sequence in seconds. Accepts decimal numbers (0.25 is quarter of a second). The length of the array determines the maximum number of retries. + // + "RetrySequenceSeconds": [1, 3, 6, 12], + // + // Error codes that will trigger a retry when opening a connection. See https://www.postgresql.org/docs/current/errcodes-appendix.html + // + "ErrorCodes": [ + "08000", "08003", "08006", "08001", "08004", // Connection failure codes + "55P03", // Lock not available + "55006", // Object in use + "53300", // Too many connections + "57P03", // Cannot connect now + "40001" // Serialization failure (can be retried) + ] + }, + // + // The connection name in ConnectionStrings configuration that will be used to execute the metadata query. If this value is null, the default connection string will be used. + // + "MetadataQueryConnectionName": null, + // + // Set the search path to this schema before executing the metadata query function. + // When null (default), no search path is set and the server's default search path is used. + // + // This is needed when using non superuser connection roles with limited schema access and mapping the metadata function to a specific schema. + // If the connection string contains the same "Search Path=" it will be skipped. + // + "MetadataQuerySchema": null, + // Any: Any successful connection is acceptable. + // Primary: Server must not be in hot standby mode (pg_is_in_recovery() must return false). + // Standby: Server must be in hot standby mode (pg_is_in_recovery() must return true). + // PreferPrimary: First try to find a primary server, but if none of the listed hosts is a primary server, try again in Any mode. + // PreferStandby: First try to find a standby server, but if none of the listed hosts is a standby server, try again in Any mode. + // ReadWrite: Session must accept read-write transactions by default (that is, the server must not be in hot standby mode and the default_transaction_read_only parameter must be off). + // ReadOnly: Session must not accept read-write transactions by default (the converse). + // see https://www.npgsql.org/doc/failover-and-load-balancing.html + "MultiHostConnectionTargets": { + // all connections use the same target mode + "Default": "Any", + // per connection overrides { "name": "Primary|Standby|Any|PreferPrimary|PreferStandby|ReadWrite|ReadOnly" } + "ByConnectionName": { } + } + }, + + // + // Enable to invoke UseKestrelHttpsConfiguration. See https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.hosting.webhostbuilderkestrelextensions.usekestrelhttpsconfiguration?view=aspnetcore-8.0 + // + "Ssl": { + "Enabled": false, + // + // Adds middleware for redirecting HTTP Requests to HTTPS. See https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.builder.httpspolicybuilderextensions.usehttpsredirection?view=aspnetcore-8.0 + // + "UseHttpsRedirection": true, + // + // Adds middleware for using HSTS, which adds the Strict-Transport-Security header. See https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.builder.hstsbuilderextensions.usehsts?view=aspnetcore-2.1 + // + "UseHsts": true + }, + + // + // Data protection settings. Encryption/decryption settings for Auth Cookies, Antiforgery tokens and custom data protection needs. + // + "DataProtection": { + "Enabled": false, + // + // Set to null to use the current "ApplicationName" value. + // This value determines encryption type or class. Meaning, different application names will not be able to decrypt each other's data. + // + "CustomApplicationName": null, + // + // Sets the default lifetime in days of keys created by the data protection system. + // Represents a number of days how long before keys are rotated. + // + "DefaultKeyLifetimeDays": 90, + // + // Data protection location: "Default", "FileSystem" or "Database" + // + // Note: When running on Linux, using Default location means keys will not be persisted. + // When keys are lost on restart, encrypted tokens (auth) will also not work on restart. + // Linux users should use FileSystem or Database storage. + // + "Storage": "Default", + // + // FileSystem storage path. Set to a valid path when using FileSystem. + // Note: When running in Docker environment, the path must be a Docker volume path to persist the keys. + // + "FileSystemPath": "./data-protection-keys", + // + // GetAllElements database command. Expected to return rows with a single column of type text. + // + "GetAllElementsCommand": "select get_data_protection_keys()", + // + // StoreElement database command. Receives two parameters: name and data of type text. Doesn't return anything. + // + "StoreElementCommand": "call store_data_protection_keys($1,$2)", + // + // Configure encryption algorithms for data protection keys or null to use the default algorithm. + // Values: AES_128_CBC, AES_192_CBC, AES_256_CBC, AES_128_GCM, AES_192_GCM, AES_256_GCM + // + "EncryptionAlgorithm": null, + // + // Configure validation algorithms for data protection keys or null to use the default algorithm. + // Values: HMACSHA256, HMACSHA512 + // + "ValidationAlgorithm": null, + // + // Key encryption method: "None", "Certificate", or "Dpapi" (Windows only) + // None: Keys are not encrypted at rest (default) + // Certificate: Keys are encrypted using an X.509 certificate + // Dpapi: Keys are encrypted using Windows Data Protection API (Windows only) + // + "KeyEncryption": "None", + // + // Path to the X.509 certificate file (.pfx) when using Certificate key encryption. + // + "CertificatePath": null, + // + // Password for the certificate file. Can be null for certificates without password. + // For security, consider using environment variable reference: "${CERT_PASSWORD}" + // + "CertificatePassword": null, + // + // When using Dpapi key encryption, set to true to protect keys to the local machine. + // If false (default), keys are protected to the current user account. + // + "DpapiLocalMachine": false + }, + + // + // Uncomment to configure Kestrel web server and to add certificates + // See https://learn.microsoft.com/en-us/aspnet/core/fundamentals/servers/kestrel/endpoints?view=aspnetcore-9.0 + // + "Kestrel": { + // "Endpoints": { + // "Http": { + // "Url": "http://localhost:5000" + // }, + // "HttpsInlineCertFile": { + // "Url": "https://localhost:5001", + // "Certificate": { + // "Path": "", + // "Password": "$CREDENTIAL_PLACEHOLDER$" + // } + // }, + // "HttpsInlineCertAndKeyFile": { + // "Url": "https://localhost:5002", + // "Certificate": { + // "Path": "", + // "KeyPath": "", + // "Password": "$CREDENTIAL_PLACEHOLDER$" + // } + // }, + // "HttpsInlineCertStore": { + // "Url": "https://localhost:5003", + // "Certificate": { + // "Subject": "", + // "Store": "", + // "Location": "", + // "AllowInvalid": "" + // } + // }, + // "HttpsDefaultCert": { + // "Url": "https://localhost:5004" + // } + // }, + // "Certificates": { + // "Default": { + // "Path": "", + // "Password": "$CREDENTIAL_PLACEHOLDER$" + // } + // }, + // "Limits": { + // "MaxConcurrentConnections": 100, + // "MaxConcurrentUpgradedConnections": 100, + // "MaxRequestBodySize": 30000000, + // "MaxRequestBufferSize": 1048576, + // "MaxRequestHeaderCount": 100, + // "MaxRequestHeadersTotalSize": 32768, + // "MaxRequestLineSize": 8192, + // "MaxResponseBufferSize": 65536, + // "KeepAliveTimeout": "00:02:00", + // "RequestHeadersTimeout": "00:00:30", + // "Http2": { + // "MaxStreamsPerConnection": 100, + // "HeaderTableSize": 4096, + // "MaxFrameSize": 16384, + // "MaxRequestHeaderFieldSize": 8192, + // "InitialConnectionWindowSize": 65535, + // "InitialStreamWindowSize": 65535, + // "MaxReadFrameSize": 16384, + // "KeepAlivePingDelay": "00:00:30", + // "KeepAlivePingTimeout": "00:01:00", + // "KeepAlivePingPolicy": "WithActiveRequests" + // }, + // "Http3": { + // "MaxRequestHeaderFieldSize": 8192 + // } + // }, + // "DisableStringReuse": false, + // "AllowAlternateSchemes": false, + // "AllowSynchronousIO": false, + // "AllowResponseHeaderCompression": true, + // "AddServerHeader": true, + // "AllowHostHeaderOverride": false + }, + + // + // Thread pool configuration settings for optimizing application performance + // + "ThreadPool": { + // + // Minimum number of worker threads in the thread pool. Set to null to use system defaults. + // + "MinWorkerThreads": null, + // + // Minimum number of completion port threads. Set to null to use system defaults. + // + "MinCompletionPortThreads": null, + // + // Maximum number of worker threads in the thread pool. Set to null to use system defaults. + // + "MaxWorkerThreads": null, + // + // Maximum number of completion port threads. Set to null to use system defaults. + // + "MaxCompletionPortThreads": null + }, + + // + // Authentication and Authorization settings + // + "Auth": { + // + // Enable Cookie Auth + // + "CookieAuth": false, + // + // Authentication scheme name for cookie authentication. Set to null to use default. + // + "CookieAuthScheme": "Cookies", + // + // Cookie validity duration in Postgres interval syntax: e.g. "14 days", "12 hours", "30 minutes". + // Set to null to fall back to the framework default (14 days). + // + "CookieValid": "14 days", + // + // Custom name for the authentication cookie. Set to null to use default. + // + "CookieName": null, + // + // Path scope for the authentication cookie. Set to null to use default. + // + "CookiePath": null, + // + // Domain scope for the authentication cookie. Set to null to use default. + // + "CookieDomain": null, + // + // Allow multiple concurrent sessions for the same user. + // + "CookieMultiSessions": true, + // + // Make cookie accessible only via HTTP (not JavaScript). + // + "CookieHttpOnly": true, + // + // Controls the SameSite attribute on the authentication cookie. Accepted values: + // "Strict" — cookie sent only on same-site requests. Most restrictive; CSRF-safe. + // "Lax" — cookie sent on same-site requests and top-level cross-site GETs (default). + // "None" — cookie sent on all cross-site requests. REQUIRED for cross-origin SPAs / + // mobile clients calling this API from a different origin. Browsers drop + // "SameSite=None" cookies without the Secure attribute, so CookieSecure + // must be set to "Always". + // "Unspecified" — omit the SameSite attribute entirely (legacy browser behavior). + // Set to null to use ASP.NET Core's default (typically "Lax"). + // + "CookieSameSite": null, + // + // Controls when the cookie's Secure attribute is set. Accepted values: + // "SameAsRequest" — Secure is set only when the request itself is HTTPS (default). + // "Always" — Secure is always set; browsers only send the cookie over HTTPS. REQUIRED + // alongside CookieSameSite="None" for cross-origin auth. + // "None" — Secure is never set; cookies are sent over HTTP as well as HTTPS. + // Set to null to use ASP.NET Core's default ("SameAsRequest"). + // + "CookieSecure": null, + // + // Enable Microsoft Bearer Token Auth (proprietary format, not JWT) + // + "BearerTokenAuth": false, + // + // Authentication scheme name for bearer token authentication. Set to null to use default. + // + "BearerTokenAuthScheme": "BearerToken", + // + // Bearer token expiration in Postgres interval syntax: e.g. "1 hour", "30 minutes", "2 days". + // Set to null to fall back to the framework default (1 hour). + // + "BearerTokenExpire": "1 hour", + // POST { "refresh": "{{refreshToken}}" } + "BearerTokenRefreshPath": "/api/token/refresh", + // + // Enable standard JWT (JSON Web Token) Bearer Authentication + // + "JwtAuth": false, + // + // Authentication scheme name for JWT authentication. Set to null to fall back to the framework default. + // + "JwtAuthScheme": "Bearer", + // + // Secret key used to sign JWT tokens. Must be at least 32 characters for HS256. + // IMPORTANT: Use a strong, unique secret in production. Store securely (e.g., environment variable). + // + "JwtSecret": null, + // + // JWT issuer (iss claim). Identifies the principal that issued the JWT. + // + "JwtIssuer": null, + // + // JWT audience (aud claim). Identifies the recipients that the JWT is intended for. + // + "JwtAudience": null, + // + // JWT access token expiration in Postgres interval syntax: e.g. "60 minutes", "1 hour", "30 seconds". + // Set to null to fall back to the framework default (60 minutes). + // + "JwtExpire": "60 minutes", + // + // JWT refresh token expiration in Postgres interval syntax: e.g. "7 days", "168 hours", "1 week". + // Set to null to fall back to the framework default (7 days). + // + "JwtRefreshExpire": "7 days", + // + // Validate the issuer (iss) claim. Set to true if JwtIssuer is configured. + // + "JwtValidateIssuer": false, + // + // Validate the audience (aud) claim. Set to true if JwtAudience is configured. + // + "JwtValidateAudience": false, + // + // Validate the token lifetime (exp claim). Default is true. + // + "JwtValidateLifetime": true, + // + // Validate the signing key. Default is true. + // + "JwtValidateIssuerSigningKey": true, + // + // Clock skew to apply when validating token lifetime. Format: PostgreSQL interval. + // Default is 5 minutes to account for clock differences between servers. + // + "JwtClockSkew": "5 minutes", + // + // URL path for JWT token refresh endpoint. POST with { "refreshToken": "..." } + // Returns new access token and refresh token pair. + // + "JwtRefreshPath": "/api/jwt/refresh", + // + // Named additional authentication schemes. Each entry registers a fully-fledged ASP.NET Core + // authentication scheme alongside the main one. A login function returning a scheme name in its + // `scheme` column signs the user in under that scheme — useful for "short-lived sensitive + // session", "separate admin scope", or "different JWT signing key per scope" patterns alongside + // the normal long-lived primary scheme. + // + // Each scheme has a `Type`: `Cookies`, `BearerToken`, or `Jwt`. Schemes inherit any unset field + // from the root Auth section so blocks stay small. See the type-specific override fields below. + // + // Validation: scheme name must not collide with the main scheme names (CookieAuthScheme, + // BearerTokenAuthScheme, JwtAuthScheme). Explicit `CookieName` values must be unique across all + // schemes. Refresh paths (BearerTokenRefreshPath / JwtRefreshPath) must be unique across all + // schemes that define one. Disabled schemes (`Enabled: false`) are skipped at startup. + // + "Schemes": { + // Example: a short-lived single-session cookie for sensitive operations (admin area, payment flow). + // Login functions can return `'short_session'` in the scheme column to sign users in under this scheme. + "short_session": { + "Type": "Cookies", + "Enabled": false, + "CookieValid": "1 hour", + "CookieMultiSessions": false + }, + // Example: a separate Microsoft bearer-token scheme with a shorter expiration than the main one. + // Each scheme can declare its own refresh path; if set, it must be unique across schemes. + "api_token": { + "Type": "BearerToken", + "Enabled": false, + "BearerTokenExpire": "30 minutes", + "BearerTokenRefreshPath": "/api/api-token/refresh" + }, + // Example: a separate JWT scheme with its own signing secret (different blast radius from the + // main JWT) and a much shorter access-token expiration. Inherits any unset JWT field from the + // root Auth section. JwtSecret must be ≥32 characters for HS256. + "admin_jwt": { + "Type": "Jwt", + "Enabled": false, + "JwtSecret": null, + "JwtIssuer": null, + "JwtAudience": null, + "JwtExpire": "5 minutes", + "JwtRefreshExpire": "1 hour", + "JwtRefreshPath": "/api/admin-jwt/refresh" + } + }, + // + // Enable external auth providers + // + "External": { + "Enabled": false, + // + // sessionStorage key to store the status of the external auth process returned by the signin page. + // The value is HTTP status code (200 for success, 401 for unauthorized, 403 for forbidden, etc.) + // + "BrowserSessionStatusKey": "__external_status", + // + // sessionStorage key to store the message of the external auth process returned by the signin page. + // + "BrowserSessionMessageKey": "__external_message", + // + // Path to the signin page to handle the external auth process. Redirect to this page to start the external auth process. + // Format placeholder {0} is the provider name in lowercase (google, linkedin, github, etc.) + // + "SigninUrl": "/signin-{0}", + // + // Sign in page template. Format placeholders {0} is the provider name, {1} is the script to redirect to the external auth provider. + // + "SignInHtmlTemplate": "Talking To {0}Loading...{1}", + // + // URL to redirect after the external auth process is completed. Usually this is resolved from the request automatically. Except when it's not. + // + "RedirectUrl": null, + // + // Path to redirect after the external auth process is completed. + // + "ReturnToPath": "/", + // + // Query string key to store the path to redirect after the external auth process is completed. + // Use this to set dynamic return path. If this query string key is not found, the ReturnToPath value is used. + // + "ReturnToPathQueryStringKey": "return_to", + // + // Login command to execute after the external auth process is completed. There are five positional and optional parameters: + // $1 - external login provider (if parameter exists, type text). + // $2 - external login email (if parameter exists, type text). + // $3 - external login name (if parameter exists, type text). + // $4 - external login JSON data received (if parameter exists, type text, JSON or JSONB). + // $5 - client browser analytics JSON data (if parameter exists, type text, JSON or JSONB). + // + // The command uses the same rules as the login enabled routine. + // See: "NpgsqlRest.“LoginPath" + // + "LoginCommand": "select * from external_login($1,$2,$3,$4,$5)", + // + // Browser client analytics data that will be sent as JSON to external auth command as the 5th parameter if supplied. + // + "ClientAnalyticsData": "{timestamp:new Date().toISOString(),timezone:Intl.DateTimeFormat().resolvedOptions().timeZone,screen:{width:window.screen.width,height:window.screen.height,colorDepth:window.screen.colorDepth,pixelRatio:window.devicePixelRatio,orientation:screen.orientation.type},browser:{userAgent:navigator.userAgent,language:navigator.language,languages:navigator.languages,cookiesEnabled:navigator.cookieEnabled,doNotTrack:navigator.doNotTrack,onLine:navigator.onLine,platform:navigator.platform,vendor:navigator.vendor},memory:{deviceMemory:navigator.deviceMemory,hardwareConcurrency:navigator.hardwareConcurrency},window:{innerWidth:window.innerWidth,innerHeight:window.innerHeight,outerWidth:window.outerWidth,outerHeight:window.outerHeight},location:{href:window.location.href,hostname:window.location.hostname,pathname:window.location.pathname,protocol:window.location.protocol,referrer:document.referrer},performance:{navigation:{type:performance.navigation?.type,redirectCount:performance.navigation?.redirectCount},timing:performance.timing?{loadEventEnd:performance.timing.loadEventEnd,loadEventStart:performance.timing.loadEventStart,domComplete:performance.timing.domComplete,domInteractive:performance.timing.domInteractive,domContentLoadedEventEnd:performance.timing.domContentLoadedEventEnd}:null}}", + // + // Client IP address that will be added to the client analytics data under this JSON key. + // + "ClientAnalyticsIpKey": "ip", + // + // External providers + // + "Google": { + // + // visit https://console.cloud.google.com/apis/ to configure your Google app and get your client id and client secret + // + "Enabled": false, + "ClientId": "", + "ClientSecret": "", + "AuthUrl": "https://accounts.google.com/o/oauth2/v2/auth?response_type=code&client_id={0}&redirect_uri={1}&scope=openid profile email&state={2}", + "TokenUrl": "https://oauth2.googleapis.com/token", + "InfoUrl": "https://www.googleapis.com/oauth2/v3/userinfo", + "EmailUrl": null + }, + "LinkedIn": { + // + // visit https://www.linkedin.com/developers/apps/ to configure your LinkedIn app and get your client id and client secret + // + "Enabled": false, + "ClientId": "", + "ClientSecret": "", + "AuthUrl": "https://www.linkedin.com/oauth/v2/authorization?response_type=code&client_id={0}&redirect_uri={1}&state={2}&scope=r_liteprofile%20r_emailaddress", + "TokenUrl": "https://www.linkedin.com/oauth/v2/accessToken", + "InfoUrl": "https://api.linkedin.com/v2/me", + "EmailUrl": "https://api.linkedin.com/v2/emailAddress?q=members&projection=(elements//(handle~))" + }, + "GitHub": { + // + // visit https://github.com/settings/developers/ to configure your GitHub app and get your client id and client secret + // + "Enabled": false, + "ClientId": "", + "ClientSecret": "", + "AuthUrl": "https://github.com/login/oauth/authorize?client_id={0}&redirect_uri={1}&state={2}&allow_signup=false", + "TokenUrl": "https://github.com/login/oauth/access_token", + "InfoUrl": "https://api.github.com/user", + "EmailUrl": null + }, + "Microsoft": { + // + // visit https://portal.azure.com/#blade/Microsoft_AAD_RegisteredApps/ApplicationsListBlade to configure your Microsoft app and get your client id and client secret + // Documentation: https://learn.microsoft.com/en-us/entra/identity-platform/ + // + "Enabled": false, + "ClientId": "", + "ClientSecret": "", + "AuthUrl": "https://login.microsoftonline.com/common/oauth2/v2.0/authorize?response_type=code&client_id={0}&redirect_uri={1}&scope=openid%20profile%20email&state={2}", + "TokenUrl": "https://login.microsoftonline.com/common/oauth2/v2.0/token", + "InfoUrl": "https://graph.microsoft.com/oidc/userinfo", + "EmailUrl": null + }, + "Facebook": { + // + // visit https://developers.facebook.com/apps/ to configure your Facebook app and get your client id and client secret + // Documentation: https://developers.facebook.com/docs/facebook-login/ + // + "Enabled": false, + "ClientId": "", + "ClientSecret": "", + "AuthUrl": "https://www.facebook.com/v20.0/dialog/oauth?response_type=code&client_id={0}&redirect_uri={1}&scope=public_profile%20email&state={2}", + "TokenUrl": "https://graph.facebook.com/v20.0/oauth/access_token", + "InfoUrl": "https://graph.facebook.com/me?fields=id,name,email", + "EmailUrl": null + } + }, + // + // WebAuthn/FIDO2 Passkey Authentication + // Provides phishing-resistant, passwordless authentication using device-native biometrics or PINs. + // + "PasskeyAuth": { + // + // Enable passkey authentication. + // + "Enabled": false, + // + // Enable registration endpoints. + // + "EnableRegister": false, + // + // Rate limiter policy name to apply to all passkey endpoints. + // It is recommended to enable rate limiting on passkey endpoints to protect against brute-force attacks. + // Set to the name of a configured rate limiter policy, or null to disable rate limiting. + // + "RateLimiterPolicy": null, + // + // Optional connection name for named DataSource or ConnectionString lookup. + // If null, uses the default DataSource or ConnectionString from NpgsqlRest options. + // + "ConnectionName": null, + // + // Command retry strategy name from CommandRetryOptions.Strategies. + // Set to null to disable command retry for passkey endpoints. + // + "CommandRetryStrategy": "default", + // + // Relying Party ID (domain name). Should match your application domain (e.g., "example.com"). + // If null, auto-detected from the request host. + // Note: IP addresses are not permitted - use "localhost" for local development. + // + "RelyingPartyId": null, + // + // Human-readable Relying Party name displayed to users during registration and authentication. + // If null, uses the ApplicationName from configuration. + // + "RelyingPartyName": null, + // + // Allowed origins for origin validation (scheme + domain + port). + // Example: ["https://example.com", "https://www.example.com"] + // If empty, auto-detected from the request. + // Note: IP addresses are not permitted - use "http://localhost:port" for local development. + // + "RelyingPartyOrigins": [], + // + // Post path for adding a passkey to an existing authenticated user (options). + // Post any additional data in the body as JSON (e.g., { "deviceName": "My Phone" }). + // Requires authentication. Set to null to disable this endpoint. + // + "AddPasskeyOptionsPath": "/api/passkey/add/options", + // + // Post path for adding a passkey to an existing authenticated user (completion). + // Post the WebAuthn response data in the body as JSON (challengeId, credentialId, attestationObject, clientDataJSON, transports). + // Additional JSON body fields are userContext passed through to the CompleteAddExistingUserCommand and optional analyticsData. + // Requires authentication. Set to null to disable this endpoint. + // + "AddPasskeyPath": "/api/passkey/add", + // + // Post path for registration options (new user with passkey). + // Post the user registration data the body as JSON (e.g., { "user_name": "...", "user_display_name": "...", "deviceName": "My Phone" }). + // No authentication required. Set to null to disable registration. + // + "RegistrationOptionsPath": "/api/passkey/register/options", + // + // Post path for registration completion (new user with passkey). + // Post the WebAuthn response data in the body as JSON (challengeId, credentialId, attestationObject, clientDataJSON, transports). + // Additional JSON body fields are userContext passed through to the CompleteAddExistingUserCommand and optional analyticsData. + // No authentication required. Set to null to disable registration. + // + "RegistrationPath": "/api/passkey/register", + // + // Post path for the login options endpoint. + // Post the user login data in the body as JSON (e.g., { "user_name": "..." } ). + // Posting the user_name is optional when using discoverable credentials. When discoverable credentials ate not enabled on the authenticator, user_name is required. + // + "LoginOptionsPath": "/api/passkey/login/options", + // + // Post path for the login completion endpoint. + // Post the WebAuthn response data in the body as JSON (challengeId, credentialId, authenticatorData, clientDataJSON, signature, userHandle) and optional analyticsData. + // + "LoginPath": "/api/passkey/login", + // + // Challenge timeout in minutes. Challenges not used within this time will expire. + // + "ChallengeTimeoutMinutes": 5, + // + // User verification requirement: + // - "preferred": Request UV if available, but allow authentication without it + // - "required": Require UV, fail if not available + // - "discouraged": Don't request UV (not recommended for most use cases) + // + // Practical implications: + // - "required": User MUST authenticate with biometric (fingerprint, face) or device PIN. + // High security - proves the person is present, not just possession of the device. + // - "preferred": Browser will request biometric/PIN if available, but allows passkey + // authentication even if UV isn't supported (e.g., older security keys). + // - "discouraged": Just proves device possession, no biometric/PIN prompt. Lower security. + // + // For most apps, use "preferred". For banking/sensitive apps, use "required". + // + "UserVerificationRequirement": "required", + // + // Resident key (discoverable credential) requirement: + // - "preferred": Request discoverable credentials if supported + // - "required": Require discoverable credentials, fail if not supported + // - "discouraged": Request non-discoverable credentials + // + // Practical implications: + // - "required": True passwordless. Browser shows passkey picker with all accounts at login. + // User picks account and authenticates with biometric/PIN. No username input needed. + // - "preferred"/"discouraged": User enters username first, then authenticates with passkey. + // + // For passwordless flows (no username field), set to "required". + // + "ResidentKeyRequirement": "required", + // + // Attestation conveyance preference - controls whether the server requests the authenticator + // to provide cryptographic proof of its identity (make/model) and security properties during registration. + // + // Options: + // - "none": Don't request attestation. Accept any valid authenticator without verifying its identity. + // Best for most apps - simpler, better user privacy, wider device compatibility. (Recommended) + // - "indirect": Request attestation but allow the browser/platform to anonymize it. Rarely useful. + // - "direct": Request full attestation certificate chain from the authenticator. + // Use when you need to verify the authenticator vendor/model meets security requirements. + // - "enterprise": Request enterprise-specific attestation for managed corporate devices + // where IT needs to verify only organization-approved hardware authenticators are used. + // + // When to use non-"none" values: + // - Banking/financial apps requiring hardware security keys only + // - Enterprise environments restricting to specific authenticator models + // - Compliance requirements mandating certain security certifications (FIDO2 L1/L2) + // + // For most consumer applications, "none" is the correct choice - you just want the user + // to authenticate securely, not audit their hardware. + // + "AttestationConveyance": "none", + // + // Whether to validate and update the signature counter (sign count). + // When true, validates that the new sign count is greater than stored, and updates it after authentication. + // When false, skips sign count validation and update entirely. + // Set to false if authenticators don't support it or you want to simplify your database schema. + // + "ValidateSignCount": true, + // + // SQL command to create a challenge when adding a passkey to an existing authenticated user. + // Parameters: + // - $1 = claims (json): JSON object with user claims from the authenticated session + // - $2 = body (json): JSON object from request body (e.g., { "deviceName": "My Phone" }) + // Expected return columns (by name): + // - status (int): HTTP status code. Return 200 to proceed, any other status aborts. + // - message (text): Error message when status != 200. + // - challenge (text): Base64-encoded random challenge bytes (typically 32 bytes). + // - challenge_id: Server-side identifier (uuid, int, bigint, or text). + // - user_handle (text): Base64-encoded random bytes (typically 32 bytes) for WebAuthn user.id. + // - user_name (text): Username displayed in the authenticator UI. + // - user_display_name (text): Display name shown in the authenticator UI. + // - exclude_credentials (text): JSON array of existing credentials. + // - user_context (json): Opaque JSON passed through to CompleteAddExistingUserCommand. + // Called by AddPasskeyOptionsPath endpoint + // + "ChallengeAddExistingUserCommand": "select * from passkey_challenge_add_existing($1,$2)", + // + // SQL command to create a challenge for standalone registration (new user). + // Parameter: $1 = JSON object from request body (e.g., { "user_name": "...", "display_name": "..." }) + // Expected return columns (by name): Same as ChallengeAddExistingUserCommand + // - user_context should NOT contain "id" field (distinguishes from add-existing-user flow) + // Called by StandaloneRegistrationOptionsPath endpoint + // + "ChallengeRegistrationCommand": "select * from passkey_challenge_registration($1)", + // + // SQL command to create a challenge for authentication. + // Parameters: + // - $1 = user_name (text, optional - null for discoverable credential flow) + // - $2 = body (json): JSON object from request body (e.g., { "deviceInfo": "..." }) + // Expected return columns (by name): status, message, challenge, challenge_id, allow_credentials + // Called by AuthenticationOptionsPath endpoint + // + "ChallengeAuthenticationCommand": "select * from passkey_challenge_authentication($1,$2)", + // + // Used by: Flow 1, Flow 2, Flow 3 (ALL flows) + // SQL command to verify and consume a challenge. + // Parameters: $1 = challenge_id (uuid, int, bigint, or text), $2 = operation (text: "registration" or "authentication") + // Returns: challenge (bytea) - the original challenge bytes, or NULL if not found/expired + // Called by all endpoints + // + "VerifyChallengeCommand": "select * from passkey_verify_challenge($1,$2)", + // + // SQL command to get credential data for authentication. + // Parameter: $1 = credential_id (bytea) + // Expected return columns (by name): status, message, public_key, public_key_algorithm, sign_count, user_context + // Note: user_context is passed through to CompleteAuthenticateCommand (typically contains user_id) + // Called by AuthenticatePath endpoint + // + "AuthenticateDataCommand": "select * from passkey_authenticate_data($1)", + // + // SQL command to complete adding a passkey to an existing user account. + // Parameters: + // - $1 = credential_id (bytea): Unique credential identifier from authenticator. + // - $2 = user_handle (bytea): WebAuthn user.id from registration options. + // - $3 = public_key (bytea): Public key in COSE format. + // - $4 = algorithm (int): COSE algorithm identifier (-7 for ES256, -257 for RS256). + // - $5 = transports (text[]): Transport hints (e.g., ["internal", "hybrid"]). + // - $6 = backup_eligible (boolean): Whether credential can be backed up/synced. + // - $7 = user_context (json): Opaque JSON from ChallengeAddExistingUserCommand (contains user ID). + // - $8 = analytics_data (json, optional): Client analytics with server-added IP. + // Expected return columns (by name): status, message + // Called by RegisterPath endpoint + // + "CompleteAddExistingUserCommand": "select * from passkey_complete_add_existing($1,$2,$3,$4,$5,$6,$7,$8)", + // + // SQL command to complete standalone passkey registration (creates new user). + // Parameters: Same as CompleteAddExistingUserCommand + // - user_context should NOT contain "id" field (creates new user instead of linking to existing) + // Expected return columns (by name): status, message + // Called by RegisterPath endpoint + // + "CompleteRegistrationCommand": "select * from passkey_complete_registration($1,$2,$3,$4,$5,$6,$7,$8)", + // + // Flow 3: Login -> AuthenticatePath endpoint (after signature validation) + // SQL command to update sign count and return user claims. + // Parameters: + // - $1 = credential_id (bytea) + // - $2 = new_sign_count (bigint) + // - $3 = user_context (json): Opaque JSON from AuthenticateDataCommand + // - $4 = analytics_data (json, optional): Client analytics with server-added IP + // Expected return columns (by name): status, user_id, user_name, user_roles (plus any custom claims) + // Called by AuthenticatePath endpoint + // + "CompleteAuthenticateCommand": "select * from passkey_complete_authenticate($1,$2,$3,$4)", + // + // The JSON key name used to add the client's IP address to the analytics data server-side. + // Set to null or empty string to disable IP address collection. + // + "ClientAnalyticsIpKey": "ip", + // + // Column name configuration for database responses + // + "StatusColumnName": "status", + "MessageColumnName": "message", + "ChallengeColumnName": "challenge", + "ChallengeIdColumnName": "challenge_id", + "UserNameColumnName": "user_name", + "UserDisplayNameColumnName": "user_display_name", + "UserHandleColumnName": "user_handle", + "ExcludeCredentialsColumnName": "exclude_credentials", + "AllowCredentialsColumnName": "allow_credentials", + "PublicKeyColumnName": "public_key", + "PublicKeyAlgorithmColumnName": "public_key_algorithm", + "SignCountColumnName": "sign_count" + } + }, + + // + // Serilog settings + // + "Log": { + // + // See https://github.com/serilog/serilog/wiki/Configuration-Basics#minimum-level + // Verbose, Debug, Information, Warning, Error, Fatal. + // Note: NpgsqlRest logger applies to main application logger, which will, by default have the name defined in the ApplicationName setting. + // + "MinimalLevels": { + "NpgsqlRest": "Information", + "NpgsqlRestClient": "Information", + "System": "Warning", + "Microsoft": "Warning" + }, + // + // Enable logging to console output. + // + "ToConsole": true, + // + // Minimum log level for console output: Verbose, Debug, Information, Warning, Error, Fatal. + // + "ConsoleMinimumLevel": "Verbose", + // + // Enable logging to file system. + // + "ToFile": false, + // + // File path for log files. + // + "FilePath": "logs/log.txt", + // + // Maximum size limit for log files in bytes before rolling to a new file. + // + "FileSizeLimitBytes": 30000000, + // + // Minimum log level for file output: Verbose, Debug, Information, Warning, Error, Fatal. + // + "FileMinimumLevel": "Verbose", + // + // Maximum number of log files to retain. + // + "RetainedFileCountLimit": 30, + // + // Create a new log file when size limit is reached. + // + "RollOnFileSizeLimit": true, + // + // Enable logging to PostgreSQL database. + // + "ToPostgres": false, + // $1 - log level text, $2 - message text, $3 - timestamp with tz in utc, $4 - exception text or null, $5 - source context + // + // PostgreSQL command to execute for database logging. Parameters: $1=level, $2=message, $3=timestamp, $4=exception, $5=source. + // + "PostgresCommand": "call log($1,$2,$3,$4,$5)", + // + // Minimum log level for PostgreSQL output: Verbose, Debug, Information, Warning, Error, Fatal. + // + "PostgresMinimumLevel": "Verbose", + // + // Enable OpenTelemetry protocol (OTLP) logging output. Requires an OTLP collector endpoint. + // + "ToOpenTelemetry": false, + "OTLPEndpoint": "http://localhost:4317", + "OTLPProtocol": "Grpc", // "Grpc" or "HttpProtobuf" + "OTLResourceAttributes": { + "service.name": "{application}", // application name from the ApplicationName setting + "service.version": "1.0", // application version, set to a static value or use a build process to update it + "service.environment": "{environment}" // environment name from the EnvironmentName setting + }, + "OTLPHeaders": {}, + "OTLPMinimumLevel": "Verbose", + + // + // See https://github.com/serilog/serilog/wiki/Formatting-Output + // + "OutputTemplate": "[{Timestamp:HH:mm:ss.fff} {Level:u3}] {Message:lj} [{SourceContext}]{NewLine}{Exception}" + }, + + // + // Response compression settings + // + "ResponseCompression": { + // + // Enable response compression for HTTP responses. + // + "Enabled": false, + // + // Enable response compression for HTTPS responses. + // + "EnableForHttps": false, + // + // Use Brotli compression algorithm when supported by client. + // + "UseBrotli": true, + // + // Use Gzip compression as fallback when Brotli is not supported. + // + "UseGzipFallback": true, + // + // Compression level: Optimal, Fastest, NoCompression, SmallestSize. + // + "CompressionLevel": "Optimal", + // + // MIME types to include for compression. + // + "IncludeMimeTypes": [ + "text/plain", + "text/css", + "application/javascript", + "text/javascript", + "text/html", + "application/xml", + "text/xml", + "application/json", + "text/json", + "image/svg+xml", + "font/woff", + "font/woff2", + "application/font-woff", + "application/font-woff2" + ], + // + // MIME types to exclude from compression. + // + "ExcludeMimeTypes": [] + }, + + // + // Antiforgery Token Configuration: Protects against Cross-Site Request Forgery (CSRF/XSRF) attacks. + // CSRF attacks occur when a malicious site tricks a user's browser into making unwanted requests to your application + // using the user's authenticated session (cookies). + // + // How it works: + // 1. Server generates a unique token for each session/request + // 2. Token is embedded in forms (hidden field) or sent via header (for AJAX) + // 3. On state-changing requests (POST, PUT, DELETE), server validates the token + // 4. Requests without valid tokens are rejected (400 Bad Request) + // + // Usage in HTML forms: + //
+ // + // ... + //
+ // + // Usage in AJAX/JavaScript: + // fetch('/api/endpoint', { + // method: 'POST', + // headers: { 'RequestVerificationToken': tokenValue }, + // body: JSON.stringify(data) + // }); + // + // Note: Antiforgery automatically sets the X-Frame-Options: SAMEORIGIN header to help prevent clickjacking. + // If you're using the SecurityHeaders middleware with X-Frame-Options, the Antiforgery header takes precedence + // (SecurityHeaders will skip X-Frame-Options when Antiforgery is enabled). + // + // Reference: https://learn.microsoft.com/en-us/aspnet/core/security/anti-request-forgery + // + "Antiforgery": { + // + // Enable antiforgery token validation for state-changing requests. + // + "Enabled": false, + // + // Name of the cookie that stores the antiforgery token. + // Set to null to use the ASP.NET Core default (unique per application, starts with ".AspNetCore.Antiforgery."). + // Custom names are useful when running multiple applications on the same domain. + // + "CookieName": null, + // + // Name of the hidden form field that contains the request verification token. + // This must match the name used in your HTML forms. + // + "FormFieldName": "__RequestVerificationToken", + // + // Name of the HTTP header that can contain the antiforgery token. + // Useful for AJAX requests where adding a form field is not possible. + // JavaScript can read the token from a cookie or meta tag and send it in this header. + // + "HeaderName": "RequestVerificationToken", + // + // When true, the server will NOT look for the token in the form body. + // Forces header-only validation - useful for pure API scenarios where all requests use headers. + // When false (default), server checks both form field and header. + // + "SuppressReadingTokenFromFormBody": false, + // + // When true, prevents the automatic X-Frame-Options: SAMEORIGIN header from being set. + // X-Frame-Options helps prevent clickjacking attacks by blocking the page from being embedded in iframes. + // Only set to true if: + // - You need your pages to be embedded in iframes from other origins, OR + // - You're setting X-Frame-Options elsewhere (e.g., in SecurityHeaders or at the proxy level) + // Default: false (header is set for security) + // + "SuppressXFrameOptionsHeader": false + }, + + // + // Static files settings + // + "StaticFiles": { + "Enabled": false, + "RootPath": "wwwroot", + // + // List of static file patterns that will require authorization. + // File paths are relative to the RootPath property and pattern matching is case-insensitive. + // Pattern can include wildcards (* matches any chars, ** matches recursively including /, ? matches single char). + // For example: *.html, /user/*, /admin/**/*.html + // + "AuthorizePaths": [], + "UnauthorizedRedirectPath": "/", + "UnauthorizedReturnToQueryParameter": "return_to", + "ParseContentOptions": { + // + // Enable or disable the parsing of the static files. + // When enabled, the static files will be parsed and the tags will be replaced with the values from the claims collection. + // The tags are in the format: {claimType} where claimType is the name of the claim that will be replaced with the value from the claims collection. + // + "Enabled": false, + // + // List of claims types used. These will be parsed to NULL if not found in the claims collection or user is not authenticated. + // Accepts an array of claim names ["name","email"] or an object of name->default {"name":"guest"} where the default is used when the claim is absent. + // + "AvailableClaims": [], + // + // List of environment variable names whose values are templated into static content (the same {NAME} tag syntax as claims). + // Resolved once at startup. Accepts an array ["BUILD_LABEL"] (missing -> empty string) or an object {"DEMO_FLAG":"false"} with per-name defaults. + // SECURITY: every listed value is served to any client - never list a secret (DB password, API key, signing token). + // + "AvailableEnvVars": [], + // + // Set to true to cache the parsed files in memory. This will improve the performance of the static files. It only applies to parsed content. + // Note: caching will occur before parsing, it applies only to templates, not parsed content. + // + "CacheParsedFile": true, + // + // Headers to be added to the response for static files. Set to null or empty array to ignore. + // + "Headers": [ "Cache-Control: no-store, no-cache, must-revalidate", "Pragma: no-cache", "Expires: 0" ], + // + // List of static file patterns that will parse the content and replace the tags with the values from the claims collection. + // File paths are relative to the RootPath property and pattern matching is case-insensitive. + // Pattern can include wildcards (* matches any chars, ** matches recursively including /, ? matches single char). + // For example: *.html, *.htm, *.txt, /pages/**/*.html + // + "FilePaths": [ "*.html" ], + // + // Name of the configured Antiforgery form field name to be used in the static files (see Antiforgery FormFieldName setting). + // + "AntiforgeryFieldName": "antiForgeryFieldName", + // + // Value of the Antiforgery token if Antiforgery is enabled. + // + "AntiforgeryToken": "antiForgeryToken" + } + }, + + // + // Cross-origin resource sharing + // + "Cors": { + // + // Enable Cross-Origin Resource Sharing (CORS) support. + // + "Enabled": false, + // + // List of allowed origins for CORS requests. Empty array allows no origins. + // + "AllowedOrigins": [], + // + // List of allowed HTTP methods for CORS requests. + // + "AllowedMethods": [ + "*" + ], + // + // List of allowed headers for CORS requests. + // + "AllowedHeaders": [ + "*" + ], + // + // Allow credentials (cookies, authorization headers) in CORS requests. + // Disabled by default: credentials must be enabled deliberately and only together with + // an explicit AllowedOrigins list (never with wildcard origins). + // + "AllowCredentials": false, + // + // Maximum age in seconds for preflight request caching (10 minutes). + // + "PreflightMaxAgeSeconds": 600 + }, + + // + // Security Headers: Adds HTTP security headers to all responses to protect against common web vulnerabilities. + // These headers instruct browsers how to handle your content securely. + // Note: X-Frame-Options is automatically handled by the Antiforgery middleware when enabled (see Antiforgery.SuppressXFrameOptionsHeader). + // Reference: https://owasp.org/www-project-secure-headers/ + // + "SecurityHeaders": { + // + // Enable security headers middleware. When enabled, configured headers are added to all HTTP responses. + // + "Enabled": false, + // + // X-Content-Type-Options: Prevents browsers from MIME-sniffing a response away from the declared content-type. + // Recommended value: "nosniff" + // Set to null to not include this header. + // + "XContentTypeOptions": "nosniff", + // + // X-Frame-Options: Controls whether the browser should allow the page to be rendered in a ,