feat(website): record a website env's traffic as PCAP, served through traffic/v1 (FD-4683) - #469

Stacked on #448.

A website env can record every packet its websites' server and frontend containers send and receive, as PCAP, and serve the files through its card. This is opt-in.

• Turning it on: deploy_env takes capture_traffic, and agent-env env deploy takes --capture-traffic. It works for a website env, or a multi env with a website, on a VM. Any other env, and container mode, refuse it.

• Capture: each website container gets a tcpdump service beside it.

  • tcpdump comes from the gateway image and runs in that container's network namespace (network_mode: "service:<container>").
  • It's the only service holding NET_RAW/NET_ADMIN.
  • It writes rotating classic pcap files, 10 of 20 MB each with the oldest overwritten, to a volume no website container mounts. The gateway mounts it read-only.
  • The deploy is refused if a capture isn't running once the gateway is up.

• Protocol: the new agentenv_protocol.traffic defines urn:agentenv:traffic/v1 with two ops:

  • list returns a TrafficCaptureListing, oldest file first;
  • get streams one file as application/vnd.tcpdump.pcap.

  Client helpers: list_traffic_captures, fetch_traffic_capture, serves_traffic. The gateway's constants.py mirrors the names, with a test keeping them equal.

• Access: both ops require the per-deploy control token (DeployedEnv.control_token). The gateway removes Authorization from a proxied request only when it carries that token (compared with hmac.compare_digest), so no site can read the token, and a site's own bearer auth still reaches it.

• Collection: the new collect_traffic_capture step stores each capture as a pcap FileArtifact in one FileArtifactUniverse, recorded under context.metadata["traffic_captures"][env_id]. It's best effort unless fail_task_on_error is set. Teardown also collects any capturing env the run didn't collect, before the env closes, so a capture isn't lost with its VM.

Before merging

• Not yet run on a deployed VM. The compose wiring, gateway routes, token stripping, step and teardown are covered by unit tests.

Test

Unit suite on this branch: 13,358 passed, lint clean. One GitHub-clone test from main is deselected because it hangs on the local machine on main too.

🤖 Generated with Claude Code

Confidence Score: 4/5

Preserve partial capture results before merging so teardown cannot discard links to files it already uploaded.

Findings

1.  Partial captures lose their links ▶

### Issue 1
src/agent_env/task_step/task_steps/traffic_capture.py:166-169
`_store` saves the universe and its run reference only after every capture finishes. If teardown times out during a later download or upload, files already uploaded remain without a universe or run reference. The `finally` in `collect_uncollected_captures` saves only completed environment entries, so it cannot preserve these files. Teardown then removes the VM, preventing another attempt.

Save each completed file's reference as collection progresses, and preserve partial results when collection stops.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Adds opt-in website traffic capture through tcpdump services and the traffic/v1 protocol.

• Website containers can record bounded packet captures on VM deployments.
• Gateway cards list and stream captures through a token-protected traffic API.
• Task runs can save captures as artifacts and collect them before teardown.

The previous-review ancestry check failed, and the two revisions have different merge bases with the target. This review therefore used the full PR diff. All supplied previous threads were unnumbered and were not reposted.

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    W[Website container] --> T[tcpdump in shared network namespace]
    T --> V[Rotating PCAP volume]
    V --> G[Gateway read-only mount]
    G -->|traffic/v1 with control token| C[Capture collection]
    C --> F[File artifacts]
    F --> U[File artifact universe]
    U --> R[Stored run metadata]

_{Reviews (6) · Last reviewed commit: "fix(website): teardown saves the capture..." · Reviewed by Greptile}