firestore: Client fails with 'Invalid database id %28default%29' from every GCP compute context, but works externally and via plain REST

Summary

google.cloud.firestore.Client reliably fails with InvalidArgument: 400 Invalid database id %28default%29 on every single call, for one specific GCP project, from every Google Cloud compute context — while the identical code with identical credentials succeeds 100% of the time from outside GCP, and the plain Firestore REST API succeeds 100% of the time from inside the same "broken" GCP contexts.

Environment

google-cloud-firestore 2.28.0 (also reproduced on 2.29.0)
google-api-core 2.35.0, grpcio 1.83.0, grpcio-status 1.83.0, protobuf 6.33.5/6.33.6, googleapis-common-protos 1.75.0/1.75.1, google-auth 2.56.2/2.57.0
Python 3.12 (Cloud Run) and 3.14 (local repro of the working case)
Firestore: Native mode, (default) database, us-central1, confirmed healthy via gcloud firestore databases list

Minimal repro (fails every time)

A bare python:3.12-slim Cloud Build step — no application code, no firebase-admin, nothing but the client library itself:

steps:
  - name: "python:3.12-slim"
    entrypoint: bash
    args:
      - "-c"
      - |
        pip install --quiet google-cloud-firestore==2.28.0 google-auth==2.56.2
        python -c "
        from google.cloud import firestore
        db = firestore.Client(project='<redacted-project-id>')
        doc = db.collection('users').document('<a real doc id>').get()
        print('SUCCESS_EXISTS:', doc.exists)
        "

Fails every time with:

google.api_core.exceptions.InvalidArgument: 400 Invalid database id %28default%29
	debug_error_string = "INVALID_ARGUMENT:Invalid database id %28default%29"

The same failure reproduces identically in:

Cloud Run Jobs (plain synchronous main(), no async/threading involved)
Cloud Run Services, both in a normal synchronous request handler and in a Starlette/FastAPI BackgroundTask
The above Cloud Build step (a third, unrelated compute product)

What does NOT reproduce it

The identical code, identical package versions (explicitly pinned to match), identical project, identical service-account-derived Application Default Credentials, run on a machine outside Google Cloud (a developer laptop) — succeeds 100% of the time.
The plain Firestore REST API (GET https://firestore.googleapis.com/v1/projects/{project}/databases/(default)/documents/{path}), called with the same credentials from inside every one of the "broken" GCP contexts above — succeeds 100% of the time, returning correct document data.

What we ruled out before filing

Project ID resolution — confirmed correct at every layer (google.auth.default()'s returned project, firebase_admin.App.project_id, firestore.Client.project all report the correct project id).
Credentials/scopes — explicitly constructed with scopes=["https://www.googleapis.com/auth/cloud-platform"] and force-refreshed; valid, well-formed token confirmed.
Cloud Run Jobs vs Services — both affected identically.
Execution environment — gen1 and gen2 both affected identically.
Explicit endpoint override — client_options=ClientOptions(api_endpoint="firestore.googleapis.com") made no difference.
Package version skew — pinned every relevant package to the exact versions confirmed working on the external machine; still fails inside GCP.
GRPC_DNS_RESOLVER=native (a known gVisor/c-ares workaround) — no effect.
Retries — 5 immediate retries in a loop all fail identically (this is not a transient/timing race).
database='(default)' passed explicitly to the constructor instead of relying on the default — no effect.

Impact

This makes the gRPC-based Firestore client entirely unusable for our Cloud Run-hosted backend on this project. We've worked around it by routing all Firestore access through the plain REST API directly instead of this library, but that's a real loss of the client library's ergonomics (typed values, transactions, batched writes, etc. all had to be hand-rolled against the REST wire format) for what looks like it should be a solvable bug.

Happy to provide more diagnostic detail (full stack traces, project id, exact request/response captures) via a private channel if that's more appropriate than a public issue.

Glad to see this note. Took me (and AI) awhile to figure out why the app stop retrieving data from Firestore after deployed to Render server.
Temporary workaround:
You can resolve this immediately by pinning your dependencies to bypass the broken google-api-core version. Use either of the options below in your requirements.txt:

Option 1: Pin google-api-core below the broken version (Recommended)
Force the environment to avoid 2.35.0 by capping it to 2.34.0 or lower:textfirebase-admin==7.5.0
google-cloud-firestore==2.29.0
google-api-core<=2.34.0
grpcio==1.83.0
Use code with caution.

Option 2: Match your exact local working environment
Align Render with your local machine setup to guarantee identical environment states:
firebase-admin==7.4.0
google-cloud-firestore==2.27.0
google-api-core==2.30.3
grpcio==1.80.0

To help deploy this patch seamlessly, let me know:Which package manager or deployment config you use on Render (e.g., standard requirements.txt, Poetry, Pipenv?)

@bongkf

I had a similar issue and setting "google-api-core<=2.34.0" resolved the problem. Thanks @bongkf

This is the same regression tracked in #18207 — google-api-core 2.35.0 changed path_template.expand() to pass every substituted value through urllib.parse.quote(), which corrupts Firestore's (default) database id to %28default%29 inside firestore_v1/base_client.py::_database_string. Details, wheel diff, and repro in that thread.

It also explains the environment split you observed, which isn't actually about GCP vs. external:

Fails on compute/deploy contexts: fresh builds resolve unpinned deps and pick up google-api-core==2.35.0 (released 2026-08-24).
Works externally / locally: those environments still have google-api-core<=2.34.0 installed. pip show google-api-core in each environment should confirm.
Works via plain REST: hand-rolled REST calls never go through path_template.expand(), so the id is never encoded.

The google-api-core<=2.34.0 pin already posted here is the right workaround. Probably worth closing this as a duplicate of #18207 so the confirmations land in one place.

@anantlumina-ui

Hi @anantlumina-ui, @bongkf, @alex-fer, @hemanthg4,

I'm sorry for the regression in google-api-core==2.35.0 due to #17753. I've yanked this release so unpinned builds will fall back to the previous version 2.34.0 as per https://pypi.org/project/google-api-core/#history. Thanks for preparing the detailed bug reports. I've sent #18215 for review so valid resource IDs like (default) used in google-cloud-firestore remain unencoded.

One additional manifestation worth documenting: against the Firestore emulator, the bad database id does not fail loudly the way it does in production — it fails silently.

Production rejects %28default%29 with 400 Invalid database id. The emulator instead treats projects/<project>/databases/%28default%29 as a different, empty database namespace: reads issued by the Python client (document gets, collection streams) return not-found / empty result sets with no error at all, while clients that build the resource path correctly (e.g. the JS web SDK) continue to address the real (default) data.

Downstream effect: any logic that branches on "document exists" (permission/role lookups, config reads, etc.) silently takes the not-found path instead of raising, so in emulator-backed test suites this release manifests as ordinary assertion failures while every component reports success — considerably harder to trace back to a dependency release than the production 400. In unpinned environments it also presents as apparent flakiness, since whether a given venv resolved 2.34.0 or 2.35.0 depends on when it was built.

Repro (emulator): seed a document under (default) via the JS SDK or an emulator import, then read it back with google-cloud-firestore + google-api-core==2.35.0 and FIRESTORE_EMULATOR_HOST set — the read returns not-found instead of raising.

Nothing needed beyond the fix already in flight — just flagging the silent emulator failure mode for anyone bisecting test failures from this release. Thanks for the fast yank!

fix(api-core): move http encoding into transcode method (#18218)

Any plans when a new release of google-api-core will be pushed so that downstreams can upgrade google-auth again to fix #18029?

Navigation Menu

Repository navigation

firestore: Client fails with 'Invalid database id %28default%29' from every GCP compute context, but works externally and via plain REST #18213

Summary

Environment

Minimal repro (fails every time)

What does NOT reproduce it

What we ruled out before filing

Impact

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

firestore: Client fails with 'Invalid database id %28default%29' from every GCP compute context, but works externally and via plain REST #18213

Description

Summary

Environment

Minimal repro (fails every time)

What does NOT reproduce it

What we ruled out before filing

Impact

Activity

bongkf commented on Aug 25, 2026

alex-fer commented on Aug 25, 2026

hemanthg4 commented on Aug 25, 2026

parthea commented on Aug 25, 2026

stevennoyce commented on Aug 25, 2026

glaubitz commented on Aug 29, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions