Add config-in-database live-tests workflow and config editing e2e tests#7231
Add config-in-database live-tests workflow and config editing e2e tests#7231
Conversation
66901b7 to
b0f17d7
Compare
6689ce6 to
abd7197
Compare
b0f17d7 to
a85514b
Compare
85beeb8 to
fdc9aa1
Compare
abd7197 to
5a11ce6
Compare
fdc9aa1 to
9888492
Compare
5a11ce6 to
b3ff029
Compare
9888492 to
95ea677
Compare
b3ff029 to
66c4f7e
Compare
95ea677 to
60adc3e
Compare
533b3be to
70ff73b
Compare
66c4f7e to
ac49210
Compare
70ff73b to
864e421
Compare
53c381d to
51d4005
Compare
864e421 to
50bd715
Compare
df0aba8 to
9ad935b
Compare
516678e to
9d03833
Compare
15f76cb to
5dc6d5b
Compare
b2057b4 to
9ef1dec
Compare
5882069 to
04e60b5
Compare
22252e8 to
c236373
Compare
04e60b5 to
fc53808
Compare
|
Majority of the diff is 2 duplicated function config files because they included a prompt that needed template_filesystem_access. This one patches them out. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c2363739d3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
c236373 to
a262b5e
Compare
Introduce a new CI workflow that exercises the config-in-database read/write path end-to-end. Unlike `live-tests.yml`, this workflow has no batch_writes matrix: it boots Postgres, stores the Postgres live-tests config glob via `gateway --store-config`, then runs the full live suite against a gateway booted from the database with `TENSORZERO_INTERNAL_FLAG_ENABLE_CONFIG_IN_DATABASE=true`. Unfortunately these tests need to run serially because concurrent edits conflict; I don't think we can get around this. Add e2e tests for config-editing PR #7185 review fixes Covers the three behavioral fixes from the PR review: - `test_config_editing_apply_persists_transitive_include_templates`: regression test for the Codex P1 — after applying a config whose template transitively includes a filesystem-backed template via MiniJinja `{% include %}`, the discovered template must be persisted in `stored_files` with the `ui-config-editor` creation source. Without the `write_stored_config_in_tx` merge fix, the row is never written and a reload from stored-config tables on a fresh gateway would break. Also sanity-checks the rendered system prompt on a downstream inference. - `test_config_editing_validate_accepts_transitive_include`: exercises the validate endpoint's happy path for filesystem discovery (the client-side precheck before `Apply` is clicked). - `test_config_editing_cas_rejects_stale_base_signature_for_new_edit`: after the CAS check simplification (removal of the `canonical_signature` short-circuit), a new edit built on top of a stale `base_signature` must still 409. Also updates the no-op apply test's doc comment now that the special-case short-circuit has been removed, and fixes stale `config::` imports to `config_toml::`.
- Add live-tests-config-in-database to ALLOWED_SKIP in check-all-general-jobs-passed.sh so merge-queue runs don't fail when the job is legitimately skipped - Delete both transitive-include e2e tests: the config-in-database gateway explicitly bans template_filesystem_access, so these tests can never pass in the config-editing workflow
2f0a711 to
091eabf
Compare
|
@BugBot review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 091eabf. Configure here.
Introduce a new CI workflow that exercises the config-in-database read/write path end-to-end. Unlike
live-tests.yml, this workflow has no batch_writes matrix: it boots Postgres, stores the Postgres live-tests config glob viagateway --store-config, then runs the full live suite against a gateway booted from the database withTENSORZERO_INTERNAL_FLAG_ENABLE_CONFIG_IN_DATABASE=true.Unfortunately these tests need to run serially because concurrent edits conflict; I don't think we can get around this.
Note
Medium Risk
Adds a new required CI job and a serial config-editing e2e suite that can affect pipeline duration/flakiness and gate merges if unstable.
Overview
Adds a new
live-tests-config-in-databaseCI workflow that boots the gateway in config-in-database mode (Postgres-only) via a compose override, runs the standard live test suite, then runs a dedicated serialconfig-editingnextest profile.Introduces new e2e tests under
config_editing::that exerciseGET /internal/config_toml,POST /internal/config_toml/validate, andPOST /internal/config_toml/apply(including CAS conflict/no-op, validation failure, and prompt/template update effects), plus adds theconfig-editingnextest profile and excludes these tests from the defaulte2eprofile.Adds config-in-db test fixtures/compose wiring (including a migration service that loads TOML into stored config tables) and wires the new job into
general.ymlandci/check-all-general-jobs-passed.shso it participates in required checks.Reviewed by Cursor Bugbot for commit 091eabf. Bugbot is set up for automated code reviews on this repo. Configure here.