feat: Full Stack v12 — safe traffic ramp-up, variation preview & experiment exclusion - #56
Conversation
… fixture
Byte-for-byte copy from javascript-sdk feat/anchored-bucketing-layout
(packages/bucketing/tests/cross-sdk-bucketing-vectors.json).
sha256 9993d0e6c5719cb17a4dcbcc73af9c0cb14ee0258e7fd29cc88f44b61083dd1b
59 vectors, versions {11,12}. The fixture IS the cross-SDK contract.
Beads: ai-driven-product-dev-padw
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TDD Phase 1: pytest AC1-AC9 + 59-vector parameterized golden runner. RED confirmed: 29 failed / 51 passed / 1 collection error — ImportError/ AttributeError on not-yet-existing anchored selectors + v12 value mismatches; v11 packed vectors pass (AC6 lock intact). Beads: ai-driven-product-dev-k1vf Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…EEN)
TDD Phase 2: anchored layout gated on experience.version > 11.
- bucketing.py: build_bucket_ranges + select_bucket_anchored (anchor scale
reuses DEFAULT_MAX_TRAFFIC; half-open anchor<=v<anchor+width; inactive arms
keep weight, zero width).
- experiences.py: _is_anchored_layout (float() coercion, '12' routes anchored
like JS Number('12')>11) + _build_variation_allocations (full ordered list
incl. inactive arms); gate wired into select_experience. Packed path
(_build_buckets + select_bucket) byte-for-byte untouched (AC6).
- Fixed one RED test bug: (11.9, False) -> (11.9, True), grounded on the JS
oracle's literal Number(version)>11 (no flooring).
Gates: full suite 949 passed; golden 59/59; mypy strict clean; ruff clean.
Beads: ai-driven-product-dev-k1vf
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…unt + conftest type Documentation-only (code review R1, conf 88%): rewrote the TDD RED-phase module docstrings in the 3 new test files to describe current GREEN state; corrected the pair-count figure 18 -> 8 (verified against the fixture); fixed conftest _load return-type annotation to Union[dict, list] for the bare-list anchored fixture. No source/fixture/assertion changes. Full suite 949 passed; ruff clean. Beads: ai-driven-product-dev-k77j Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
tests/parity/README.md + TESTING.md: list the anchored golden-vector fixture (anchored_bucketing_vectors.json, bare list, imported verbatim from the cross-SDK source) and its parity test alongside the MurmurHash3 parity proof; note it is not written by the fixture generator. Part of PR #52. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
BREAKING CHANGE: Experiences on bucketing contract v12 (experience version above 11) bucket visitors with the anchored layout, where each variation holds a fixed position derived from its cumulative traffic allocation. Changing an experience's total allocation now grows or shrinks each variation at its own boundary instead of shifting every variation range, so visitor-to-variation assignments under contract v12 differ from the earlier packed layout for the same configuration.
feat: anchored bucketing layout (contract v12) — Python SDK pass
Beads: ai-driven-product-dev-cope 19 new tests fail (missing debug_token field); 34 pre-existing pass. ruff clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Beads: ai-driven-product-dev-cope SDKConfig.debug_token: Optional[str] = field(repr=False) with strict validation. Config route carries debug_token + forced _conv_low_cache=1; never on track route. Full suite 968 passed, coverage 96.81% (eval 98%), mypy --strict + ruff clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Beads: ai-driven-product-dev-rq0l
Pure helper convert_sdk.parse_preview_param; '123.456'->('123','456'), malformed->None
(strict numeric-segment check, JS/Ruby sibling parity, decision I6). 18 tests, full
suite 986 passed, cov 96.84% (preview.py 100%), mypy+ruff clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Beads: ai-driven-product-dev-lgao 11 tests fail (get_preview_decision absent). Locks full-bypass matrix incl draft/paused/ non-running/zero-traffic variation + unknown-id->None. ruff clean. NOTE: experience-status & environment bypass cases are forward-compat locks — Python's normal path does not gate on those today (decision I9; flagged for story owner). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…REEN) Beads: ai-driven-product-dev-lgao get_preview_decision(experience, variation_id) in evaluation/experiences.py: reuses _find_variation (no status/traffic filter) -> full bypass; normal ExperienceResult shape; unknown variation->None; pure/no-side-effects. 11 tests, full suite 997 passed, eval cov 98%. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Beads: ai-driven-product-dev-5801 8 failed + 11 errored (missing fetch_config_by_experience + module memo globals). Locks: exp=<id>&_conv_low_cache=1 (+debug_token) unconditional; 60s wall-clock TTL; thread-safe single fetch under 6-thread barrier; cross-tenant isolation; never-DataStore; failed fetch not memoized (JS parity). ruff clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Beads: ai-driven-product-dev-5801 HttpxTransport.fetch_config_by_experience + _build_experience_route/query: exp=<id>& _conv_low_cache=1 (+debug_token), no environment (spec Resolution clause, I16). Module-level thread-safe 60s wall-clock memo keyed sdk_key:experience_id, never DataStore, failed fetch not memoized. Test-only isolation fixture added (I18). Full suite 1016 passed, cov 96.90%/eval 98%. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Beads: ai-driven-product-dev-1b47 15 fail (Context.set_preview absent) + 1 diagnose regression-lock passes. Locks AC3 (?exp=-only delivery), AC4 (full-bypass matrix incl different stored decision), AC6 (isolation: concurrent non-preview context normal), AC7 (inert: unknown exp/var, fetch 503 -> warning + normal). Transport kept a public port (I22 -> capability protocol in GREEN). ruff clean; 1016 pre-existing pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Beads: ai-driven-product-dev-1b47 Context.set_preview(experience_id, variation_id): eager resolution (local snapshot -> ?exp= fetch via SupportsPreviewFetch capability protocol -> get_preview_decision), per-context state, precedence over stored/bucketed decision for target only, other experiences normal, inert+warn on bad input. run_experiences synthesizes fetch-only target (I27 test locks it). Transport stays a non-breaking public port. 18 targeted, full suite 1034 passed, cov 96.81%/eval 98%, mypy+ruff clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Beads: ai-driven-product-dev-oq4q 5/6 fail (spies see track POSTs + store writes that must be zero under preview); 1 AC10 regression lock passes (normal path unaffected). Full-lifecycle proof across core.flush() + atexit hook; AC6 isolation via shared Core; all persist/track sites swept (I34). transport + _SpyDataStore spies. ruff clean; PY-5 suite 18/18 untouched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Beads: ai-driven-product-dev-oq4q Preview context leaves zero trace: track_bucketing skipped in run_experience/run_experiences (overrides enable_tracking=True), track_conversion unconditional no-op (returns DEDUPLICATED, I35), _persist_visitor_state no-ops -> zero DataStore writes. Gated at Context call sites (P2), per-context only (AC6 isolation). Full lifecycle incl flush()+atexit. Full suite 1040 passed, cov 96.82%/eval 98%, mypy+ruff clean, generated-pyi OK. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…on write Beads: ai-driven-product-dev-cszc Sweep-on-write inside _CONFIG_BY_EXPERIENCE_LOCK: closes unbounded-growth path keyed by link-param experience_ids; completes TTL-60s semantics. AC8/never-DataStore/failed-fetch- not-memoized/thread-safety preserved. ruby-sdk#41 parity (I36). 13 memo tests, full suite 1042 passed, cov 96.83%/eval 98%, mypy+ruff clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An experience dict with a missing or null "id" stringified to "None" and would falsely match a lookup for the literal experience_id "None" (str(None) == "None"). Reject a None id before the string comparison so malformed/id-less experiences never match — honoring the helper's documented "never mismatch on malformed bodies" contract. Adds 5 parametrized regression cases. Addresses Gemini code-review finding on #53. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Beads: ai-driven-product-dev-mm0v 18 new tests fail (field/method/envelope absent); 56 pre-existing pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Beads: ai-driven-product-dev-mm0v ContextState.bucketing field + with_bucketing (key-merge, carried through with_attributes/with_segments); 3-key envelope persist; hydrate w/ legacy 2-key + plain-attributes backward compat (empty bucketing). No eval/hash change. Full suite 1065 passed, 96.85% cov; ruff + mypy --strict clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Beads: ai-driven-product-dev-v0d1 5 new tests fail (sticky_bucketing param absent); 10 pre-existing pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Beads: ai-driven-product-dev-v0d1 select_experience gains pure sticky_bucketing param; short-circuits between qualifies() and get_bucket_value_for_visitor when stored vid resolves via _find_variation. Fresh+sticky returns share _build_experience_result helper (value-neutral). None-arg byte-identical. AC2/AC4/AC8/AC9. Full suite 1070 green, 96.87% cov; eval floor 98%; parity 214 green (AC9); ruff+mypy strict clean. Parity fixtures untouched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Beads: ai-driven-product-dev-av5a 6 integration tests (AC1/2/3/5/6/8) over exp-a/exp-b 8-row fixture fail (enable_storage/read/write wiring absent). AC4 row4 added next (GREEN dispatch). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Beads: ai-driven-product-dev-av5a enable_storage=True kwarg on run_experience/run_experiences (threaded via _evaluate_and_record); sticky read wires self._state.bucketing into select_experience (non-preview branch); gated write-after-hash (result and enable_storage and preview is None and bucketing.get(exp_id)!=vid) -> with_bucketing + _persist_visitor_state. AC1/2/3/4/5/6/8. Full suite 1077 green 96.87% cov; eval floor 98%; parity 214 (AC9); ruff+mypy strict clean. AC6 asserts filtered to state envelope (dedup marker is orthogonal tracking artifact). set_attributes/set_segments untouched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…oseout Beads: ai-driven-product-dev-rudj AC7: shared DataStore across two Cores -> sticky via hydration (zero hash on Core B); negative control: unshared per-instance stores do NOT share. No src change (PY-1..PY-3 mechanism already complete). AC9: parity 214 green. Full suite 1079 green 96.87% cov; eval floor 98%; mypy strict (core + serving_config_contract) + pyi-header gate clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Beads: ai-driven-product-dev-g4pc 6 new tests (AC11 conversion attribution x2, AC12 feature x2, AC13 diagnostics x2) fail w/ substantive assertion mismatches (sibling paths re-hash fresh); 9 pre-existing pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Beads: ai-driven-product-dev-g4pc Thread self._state.bucketing (pure read-only) through every variation-resolution path: run_feature/run_features (features.py), conversion attribution (conversions.py _compute_bucketing_assignments + create_conversion, tracker.py track), diagnose_experience/diagnose_feature (context.py). Read-only consumers mint no decision; write stays experience-run-only. Fixed stale conversions.py docstring. AC11 (attribution sticky both paths)/AC12 (feature sticky)/AC13 (diagnostics sticky). Full suite 1085 green 96.87% cov; eval floor 98%; parity 214 (AC9); mypy strict + serving contract + pyi-header clean. Hash contract + fixtures untouched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tr-coerce rehydration Beads: ai-driven-product-dev-93nf Gemini code-assist (PR #54): bucketing param + _hydrate_visitor_state return + stored_bucketing local -> Optional[Mapping[str,str]]; rehydration coerces keys+ values to str (defensive vs a DataStore persisting non-string IDs). +1 test. Type-only + one coercion; no write-path/behavior change. mypy strict clean, full suite 1086 green, parity 214. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Beads: ai-driven-product-dev-rloj Agent: sdk-python Additive/inert: RULE_TYPE_BUCKETED_INTO_EXPERIENCE_KEY + log_mutual_exclusion_target_not_found; no behavior change (AC7). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…data only) Beads: ai-driven-product-dev-xt69 Agent: sdk-python NamedTuple 8-row table (verbatim spec values) + build_mutual_exclusion_config(); 5 genuine sanity invariants. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Beads: ai-driven-product-dev-nz93 Agent: sdk-python 20 failed / 1 passed (default-ANY regression guard). Genuine RED via get_experience_by_key spy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…udiences + bucketed_into_experience_key Beads: ai-driven-product-dev-nz93 Agent: sdk-python Audience-level exclusion resolver (read-only, presence-only, negation-last, AC8 warn) + matching_options ALL/ANY (default ANY). is_rule_matched + generic walk byte-identical (AC7 proven). 21 new pass, full 1114, cov 96.71%, eval floor 97%, parity 214. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Beads: ai-driven-product-dev-b3sb Agent: sdk-python 15 tests: 3 generic match_types (equals/contains/less) bit-identical; matching_options ALL/ANY boundary (default-ANY == pre-seam); mixed-tree fails-closed guard. src untouched. Full 1129, cov 96.76%, eval 97%. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
feat(preview): experiment preview — debug_token + set_preview (qs-02)
feat: sticky-bucketing persistence (qs-03) — per-visitor decision map + full read-back parity
feat: mutual-exclusion audience rule bucketed_into_experience_key (qs-04)
There was a problem hiding this comment.
Code Review
This pull request implements several key features for the Convert Python SDK, including the anchored bucketing layout (contract v12), experiment preview capabilities with zero-trace suppression and process-wide memoization, sticky-bucketing persistence via a 3-key envelope, and mutual-exclusion audience rules (bucketed_into_experience_key). It also includes comprehensive integration and parity tests. Feedback on the changes highlights a robustness issue in the _sole_rule_item helper, which lacks defensive type-checking for nested configuration structures and could crash on malformed inputs; implementing the suggested type-validation block will ensure the SDK fails closed gracefully.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Wire qs-02 preview into demo/run_demo.py via a new CONVERT_DEMO_PREVIEW
env var ("{experienceId}.{variationId}"): parses with parse_preview_param,
calls context.set_preview, and prints the forced decision. Adds a dedicated
paused (non-running) preview-target experience to config_fixture.json with
numeric ids (900210001 / 900210101 / 900210102) so a real convert_preview
link parses and the demo proves the two headline guarantees — a paused
experience's variation is force-rendered (bypassing status + bucketing),
and the preview context is zero-trace (track_conversion with
force_multiple=True still yields tracked=False, distinguishing real
suppression from ordinary dedup). A different experience on the same
context is unaffected; malformed/unresolvable ids degrade to the AC7
inert-with-warning path.
demo/README.md gains a "Testing preview links" section and .env.example
documents the new vars. 5 TestPreviewFixture tests added. demo/ + fixture
+ docs + tests only — no SDK source change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Final release review — sign-off ✅ (with one non-blocking follow-up)Reviewed the consolidated branch first-hand in an isolated worktree at Verdict: approve for merge. Quality gates (CI-pinned deps)
Capability behavior (empirically probed, not assumed)
One finding — non-blocking, recommend follow-up
Suggested follow-up (post-merge): add a test for 🤖 Generated with Claude Code |
Full Stack v12 — next major SDK release
This branch delivers three major capabilities, shipped consistently across all Full Stack SDKs. All three are inert until activated on the backend/UI side, so this deploys safely in any order.
1. Anchored (ramping) bucketing layout
Makes changing an experiment's total traffic allocation safe on a running experiment.
versionproperty:> 11→ anchored pass (contract v12);≤ 11/ missing → packed pass, bit-identical to today. No schema change, zero migration.2. Experiment preview
Lets users preview any variation — including drafts and paused ones — in their own environment, with zero trace.
debugTokenconfig option (24h QA token) unlocks the full config, including draft/paused experiences.convert_preview={expId}.{varId}force a single variation with no side effects: no tracking events, no visitor-state writes, no impact on reporting.3. Mutual exclusion / inclusion audience rule
Lets teams control which experiments a visitor can be in at the same time, through the normal audience-targeting rules.
bucketed_into_experience_key, resolved against SDK-stored bucketing state.negated: true→ "NOT in experiment X" (mutual exclusion); default → "in experiment X" (inclusion).Notes
version12 stamp, plus the related convert-ui work (preview link/QR button, rule picker) and shared docs, are tracked and coordinated separately.