Skip to content

Size Granian to the pool and return 503 on pool timeouts - #100

Merged
lesnik512 merged 1 commit into
mainfrom
feat/server-and-pool-config
Oct 17, 2026
Merged

lesnik512 merged 1 commit into
mainfrom
feat/server-and-pool-config

Conversation

@lesnik512

Copy link
Copy Markdown
Member

Sizes Granian to the database pool and turns pool exhaustion into a 503. Before this, Granian accepted up to 1024 requests per worker against a pool of 5. Extra requests waited 30 s on pool checkout and then failed with a 500.

Changes

  • APP_WORKERS (default 1) is passed to Granian's workers. Each worker has its own pool.
  • Backpressure. app/__main__.py sets Granian's backpressure to DB_POOL_SIZE + DB_MAX_OVERFLOW. Granian's default is backlog // workers (granian/server/common.py), and its CLI help says the value is per worker. Requests beyond the pool now wait in the socket backlog instead of holding an event-loop task while they queue on the pool.
  • Pool settings are bounded. DB_POOL_SIZE must be positive and DB_MAX_OVERFLOW non-negative. SQLAlchemy reads max_overflow=-1 as unlimited, which would give a backpressure below the pool size. pool_size=0 would give a backpressure of 0, and Granian treats 0 as unset and falls back to 1024.
  • DB_POOL_TIMEOUT (default 30 s, as before) is passed to the engine. A checkout timeout now returns 503 through exceptions.database_unavailable_handler. advanced-alchemy wraps the timeout in RepositoryError when it happens inside a repository call, and the autocommit transaction raises it raw. So the handler is registered for both sqlalchemy.exc.TimeoutError and RepositoryError, and it checks __cause__. In FastAPI the handler re-raises anything else, so Starlette's ServerErrorMiddleware still logs it and returns 500, as before. The more specific DuplicateKeyError, NotFoundError and ForeignKeyError handlers still win. postgres_retry doesn't retry a pool timeout, because it isn't a DBAPI error.
  • DB_POOL_RECYCLE is exposed with SQLAlchemy's default of -1. The template can't know the network, so AGENTS.md says to set it below any proxy or load balancer idle timeout. That avoids the 1-2 s retry backoff on the first request after an idle period.
  • AGENTS.md describes the sizing and lists the new handlers and env vars.

The Dockerfile (missing CMD, dev dependencies in the image) is left for a separate change. The image is shared with just test, so dropping dev dependencies needs its own build target.

Tests

  • test_main checks the Granian kwargs: workers from APP_WORKERS, and backpressure equal to pool size plus overflow.
  • tests/test_pool.py holds the only connection of a one-connection pool with a 0.1 s timeout. It checks that GET deck (the repository path) and POST deck (the autocommit path) both return 503. A second test pins that an unrelated repository error, a NUL byte that Postgres rejects, keeps its previous behaviour.
  • test_settings rejects DB_POOL_SIZE=0 and DB_MAX_OVERFLOW=-1.
  • just test passes 85 tests at 100% coverage, just test-migrations passes, and just lint is clean.

Add APP_WORKERS, DB_POOL_TIMEOUT and DB_POOL_RECYCLE. Set Granian backpressure to pool size plus overflow so excess requests wait in the socket backlog instead of on the pool, and require a positive pool size and non-negative overflow so that sum stays meaningful. A pool checkout timeout, raw or wrapped by advanced-alchemy in RepositoryError, now returns 503 instead of 500.
@lesnik512
lesnik512 merged commit 8017334 into main Oct 17, 2026
3 checks passed
@lesnik512
lesnik512 deleted the feat/server-and-pool-config branch October 17, 2026 12:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant