Skip to content

About

CLI-first browser automation for AI agents

Resources

Stars

3 stars

Watchers

0 watching

Forks

Repository files navigation

browser-cli

CLI-first browser automation for AI agents

Repo • Features • Installation • Quick Start • Task And Automation Model • Testing


browser-cli is a browser automation tool for AI agents and developers who need reliable browser control from the command line.

Architecture

┌──────────────────────────────────────────────────────────────────────┐
│  Task/Automation Layer  (task.py + task.meta.json + automation.toml) │
├──────────────────────────────────────────────────────────────────────┤
│  Browser Daemon  ──►  60+ commands  ──►  Semantic Ref System         │
│  ├─ read: one-shot page capture                                      │
│  ├─ open/snapshot/click/fill: interactive control                    │
│  ├─ console/network/trace: observation & debugging                   │
│  ├─ verify-*: assertions                                             │
│  └─ ... 60+ commands total                                           │
├──────────────────────────────────────────────────────────────────────┤
│  Dual Backend: Managed Profile (Playwright) ◄──► Chrome Extension    │
└──────────────────────────────────────────────────────────────────────┘
Component Purpose
Browser Daemon Long-lived browser instance with daemon-backed CLI commands
Semantic Refs Stable element identifiers using bridgic-style reconstruction
Task Runtime Reusable task.py execution through browser_cli.task_runtime
Automation Service Persistent local service for published automation snapshots

Features

  • Dual Backend Architecture: managed profile mode by default, extension mode when real Chrome is connected and healthy
  • Semantic Ref System: stable refs that survive many DOM re-renders
  • Agent Isolation: X_AGENT_ID isolates visible tabs while sharing browser storage
  • JSON-First API: daemon-backed commands return structured JSON
  • Task Runtime: package browser logic as task.py + task.meta.json
  • Automation Publish Layer: publish immutable task snapshots and operate them through a local Web UI

Installation

Requirements:

  • Python 3.10+
  • uv
  • Stable Google Chrome

Install as a tool:

uv tool install browser-control-and-automation-cli
browser-cli doctor
browser-cli paths
browser-cli read https://example.com

The published package name is browser-control-and-automation-cli. The installed command remains browser-cli.

Run without installing:

uvx --from browser-control-and-automation-cli browser-cli read https://example.com

Install from Git:

uv tool install git+https://github.com/hongv/browser-cli.git
browser-cli --help

Installed users should start with docs/installed-with-uv.md. For removal and local cleanup guidance, see docs/uninstall.md.

Install Browser CLI Skills

Browser CLI ships with three packaged skills that can be installed into an agent skills directory:

  • browser-cli-converge
  • browser-cli-delivery
  • browser-cli-explore

Install them into the default skills root:

browser-cli install-skills

By default, Browser CLI writes the packaged skills into ~/.agents/skills. Use --dry-run to preview the install and --target to override the destination:

browser-cli install-skills --dry-run
browser-cli install-skills --target ~/.codex/skills

You can rerun the command safely. Existing packaged Browser CLI skills at the target path are replaced with the packaged versions from the installed wheel. After you upgrade Browser CLI, rerun browser-cli install-skills to refresh the copied skill files. If you use a custom skills root, rerun the command with the same --target path. For a longer installed-user walkthrough, see docs/install-skills.md.

Development

Clone the repository and sync the managed development environment:

git clone https://github.com/hongv/browser-cli.git
cd browser-cli
uv sync --dev

The CLI targets stable Google Chrome. Playwright Chromium is mainly useful for local integration testing and is installed through the repo environment.

On macOS, browser integration tests use Playwright's dedicated Chrome Headless Shell so test runs do not add desktop Chrome icons to the Dock. Install the revision required by the locked environment before running browser tests:

uv run playwright install --only-shell chromium

Collection and unit tests do not launch browsers. Integration setup checks host permissions and the exact test runtime once; a missing shell fails with setup instructions instead of falling back to desktop Chrome. Test daemons use temporary profiles without changing your home directory or Dock settings. Linux integration tests continue to cover stable Chrome, and node/smoke-extension.mjs checks an already connected real Chrome extension without launching another browser.

Optional: Extension Mode

For real-Chrome execution:

  1. Open chrome://extensions
  2. Enable Developer mode
  3. Click Load unpacked
  4. Select browser-cli-extension/

Once connected, browser-cli status reports extension capability state and the daemon can prefer the extension backend at safe idle points.

Extension mode opens agent pages in a green Browser CLI tab group in an existing normal Chrome window. It creates a window only if none is available. Workspace close and rebuild affect only Browser CLI-created tabs, even if you move tabs between groups or windows. Your other tabs stay open. After an extension reload or workspace rebuild, new pages reuse the existing green Browser CLI group. Duplicate Browser CLI groups in that window are merged without closing their pages or treating those pages as agent-owned. Temporary pages leave their group before closing so Chrome does not accumulate saved groups after repeated reads. Closed groups already saved by Chrome are outside the extension API and can be removed through Chrome's tab-group menu.

New extension tabs, one-shot reads and workspace placeholders open in the background. Adding pages preserves a collapsed tab group, and a fallback window opens without requesting focus. switch-tab selects the page for subsequent agent commands; it does not select the tab you see in Chrome. Click a tab in the Browser CLI group when you want to view it. Debugging notices, site-generated dialogs and the browser's own UI can still affect the window.

One-shot read and workspace placeholder pages do not attach the debugger. Interactive pages retain full CDP capabilities and network observation from their first navigation, so Chrome's debugging notice can remain visible while those pages are managed. Reload the unpacked extension after updating its files; this version adds the tabGroups permission.

Quick Start

If you installed Browser CLI with uv, use the dedicated installed-user guide at docs/installed-with-uv.md. The short version is:

browser-cli doctor
browser-cli paths
browser-cli read https://example.com

One-Shot Read

browser-cli read https://example.com
browser-cli read https://example.com --snapshot
browser-cli read https://example.com --scroll-bottom

Interactive Control

browser-cli open https://example.com
browser-cli snapshot
browser-cli click @8d4b03a9
browser-cli fill @input_ref "value"
browser-cli html
browser-cli status
browser-cli reload

Multi-Agent Tabs

X_AGENT_ID=agent-a browser-cli open https://example.com
X_AGENT_ID=agent-a browser-cli tabs

X_AGENT_ID=agent-b browser-cli open https://example.org
X_AGENT_ID=agent-b browser-cli tabs

One-off extraction for agents

Start with browser-cli status --json and reuse a healthy runtime. Keep one stable X_AGENT_ID across the task's commands; it isolates tabs while browser cookies and storage remain shared. A one-off task can finish with its output files; task.py, task.meta.json, and publishing are optional reuse steps.

For a complete local JSON export, put the page JavaScript in a file:

browser-cli eval --file extract.js --output items.json

extract.js contains an arrow function such as () => [...document.querySelectorAll('h2')].map(el => el.textContent). --file - reads JavaScript from stdin; eval-on REF supports the same sources. The file contains the complete data.result JSON value for eval/eval-on; other browser actions export their complete data object. Stdout returns an artifact with its absolute path, byte count, SHA-256, result type, and source. Files are written by the client, atomically, with owner-only permissions. A save failure preserves an existing file. Export does not increase transport limits.

Run dependent actions on the same active tab sequentially, or use a short browser-cli batch --file commands.json. Batches exclude other browser actions while running and do not roll back side effects. Prefer wait --text, network-wait, and verify-* to fixed sleeps.

All CLI errors, including argument errors, use ok/data/meta, error_code, and error_message JSON on stdout and a nonzero exit code. Browser actions already return JSON by default and also accept --json. --help remains text. Busy errors report execution_state: "not_started" and the blocking request; a timeout or disconnect leaves execution uncertain and must not trigger replay.

To interrupt an operation after caller cancellation, use close-tab PAGE_ID (or close for the active page) with the same X_AGENT_ID. These cleanup controls can run during a batch and close only that identity's owned page. They confirm browser closure before canceling the daemon wait. Failed closure retains the busy claim; successful closure reports the interrupted request separately and does not roll back earlier effects. Client disconnect alone leaves execution uncertain. Late results and artifact chunks from an interrupted request are discarded.

For batch, top-level ok describes request handling; data.success describes step success. Failed steps produce CLI exit code 75, including with --continue-on-error. batch --output saves partial results and returns success and counts alongside the artifact descriptor. Use --result-mode all when every successful step result must be retained.

Python BrowserCliTaskClient.batch() and Node BrowserCliClient.batch() raise BATCH_FAILED on step failure; the exception's payload retains partial results. Low-level Python invoke('batch', ...) and Node request('batch', ...) leave that check to the caller. Standalone scripts can save full values with Python browser_cli.task_runtime.write_json_artifact(path, value) or Node writeJsonArtifact(path, value) from the packaged client, then print only the returned descriptor. See the Chinese usage guide for the one-off workflow and recovery rules.

The CLI does not require a model turn for each action: one host Python or Node script can call the existing client, branch, loop, and print just its final fields. Use a short batch for a known sequence and stop at an observation or authorization boundary. See multi-step agent workflows.

snapshot --compact keeps the semantic tree and public ref fields while omitting repeated locator metadata. Optional --scope REF, repeated --role ROLE, and --max-lines N select regions, element types, and an explicit line limit. Only these opt-in views add data.view, including omission counts and truncated; default snapshot output is unchanged. A missing scope fails instead of returning the whole page. Python/Flow snapshot() and Node snapshot() accept the same options (maxLines in Node). Ref actions after a URL change require a fresh snapshot. Existing error codes stay stable; meta.failure_kind distinguishes page changes, stale locators, and deadlines.

With scope/role plus --interactive, the scope is resolved before interactive filtering and the line limit is applied last; ancestor context is retained. Clients verify the returned filters and truncation counts. A completed browser action followed by cleanup failure reports execution_state=completed; a partially filled form followed by rejection remains unknown. Neither outcome permits automatic replay.

Task And Automation Model

Browser CLI separates local authoring from durable publication:

  • task is local editable source
  • automation is a published immutable snapshot

Typical task layout:

tasks/
  my_task/
    task.py
    task.meta.json
    automation.toml

Discover the shipped examples or scaffold a new task bundle:

browser-cli task examples
browser-cli task template --output tasks/my_task

Validate and run a task directly:

browser-cli task validate tasks/my_task
browser-cli task run tasks/my_task --set url=https://example.com

Publish the current task directory into the automation service:

browser-cli automation publish tasks/my_task
browser-cli automation status
browser-cli automation ui

Publication semantics:

  • automation publish snapshots task.py, task.meta.json, and automation.toml together under ~/.browser-cli/automations/<automation-id>/versions/<version>/
  • if source automation.toml exists, Browser CLI uses it as the publish-time configuration truth
  • if source automation.toml is absent, Browser CLI publishes generated defaults and reports that explicitly via manifest_source

Export a persisted automation back to automation.toml:

browser-cli automation export my_task --output /tmp/my_task.automation.toml

Included examples:

Real-site publish example:

browser-cli task validate tasks/douyin_video_download
browser-cli automation publish tasks/douyin_video_download
browser-cli automation inspect douyin_video_download
browser-cli automation status

Inspect semantics:

  • browser-cli automation inspect <automation-id> shows the current live automation-service configuration
  • browser-cli automation inspect <automation-id> --version <n> shows snapshot_config for the immutable published version and live_config for the current service state
  • latest_run remains a separate operational view

Output Contracts

read

  • stdout: final rendered result only
  • stderr: diagnostics only

Exit codes:

  • 0: success
  • 1: unexpected internal error
  • 2: usage error
  • 66: empty content
  • 69: browser unavailable
  • 73: profile unavailable
  • 75: temporary read failure

Daemon-backed Commands

  • success stdout: JSON only
  • failure stderr: short error summary
  • stable machine-readable error codes include:
    • NO_ACTIVE_TAB
    • AGENT_ACTIVE_TAB_BUSY
    • TAB_NOT_FOUND
    • NO_SNAPSHOT_CONTEXT
    • REF_NOT_FOUND
    • STALE_SNAPSHOT
    • AMBIGUOUS_REF

Runtime Notes

  • Managed profile mode is the default backend.
  • Extension mode is the preferred real-Chrome backend when connected and healthy.
  • Driver rebinding happens only at safe idle points and is reported as state_reset.
  • runtime.timeout_seconds is the total wall-clock timeout for one automation run in the automation service.

Documentation

Testing

Run lint:

./scripts/lint.sh

Run tests:

./scripts/test.sh

Run guards:

./scripts/guard.sh

Run the full local validation flow:

./scripts/check.sh

If local version metadata looks stale after code changes, refresh the editable install before the validation flow:

uv sync --dev --reinstall-package browser-control-and-automation-cli

Fast Python 3.10 compatibility check:

uv run python scripts/guards/python_compatibility.py

When the runtime behaves unexpectedly, use:

browser-cli status
browser-cli reload
browser-cli status

The integration coverage is fixture-driven and local-first. It exercises:

  • navigation, tabs, and history
  • snapshot and rendered HTML capture
  • semantic ref reconstruction after DOM re-render
  • stale and ambiguous ref failures
  • iframe refs
  • ref-driven element actions
  • console, network, dialogs, trace, video, screenshot, and PDF
  • cookies, storage save/load, and X_AGENT_ID isolation
  • task runtime and automation publishing/service flows

Controlled captures and Node.js automation

Use browser-cli screenshot output.png --clip 0,120,800,600 --dpr 2 for a page rectangle, or add --viewport-width 1280 --viewport-height 720 for a temporary viewport that is restored after capture. Responses include actual image dimensions and capture geometry. Full-page captures preserve the current page viewport.

browser-cli batch --file commands.json --context-file context.json runs ordered actions in a disposable context with fixed viewport, DPR, CSS zoom, locale, timezone, and fonts. The context closes after success or failure. --result-mode last and --result-mode summary keep results compact.

A dependency-free Node.js client ships with the Python package; discover its ESM entrypoint and the actual daemon socket with browser-cli paths --json. status --json and doctor --json report service, extension, configuration, managed-profile, and output-directory health separately. Runtime status remains independent of filesystem permissions in the calling process.

browser-cli doctor --json also checks live runtime health through the daemon. The extension check executes a read-only Chrome tab query, even when no workspace exists. A connected extension with unusable browser APIs reports a failed check and recovery guidance. Workspace binding loss appears as a separate runtime warning. The extension checks itself while idle and can reload an unusable context once per five minutes. Automatic recovery waits for active operations and recordings; it never repeats a failed browser action. If recovery cannot finish, the popup's Reload Extension action reloads the extension itself.

Workspace bindings are saved locally and restored automatically when the extension or its background worker restarts. Existing owned tabs keep their page IDs; no manual workspace rebuild is needed while the original workspace remains available. Recovery verifies each tab's Chrome debugger target identity and window membership, excludes unrelated tabs, and respects an explicitly closed workspace. If the owned window has actually disappeared, the next browser command uses the normal workspace rebuild and reports the resulting state reset.

See the capture and automation API guide for complete CLI and Node.js examples, JSON contracts, context defaults, and driver transitions.

Acknowledgements

This project is deeply inspired by bridgic-browser. Browser CLI keeps the semantic ref and daemon-backed strengths while pushing the product toward a CLI-first, agent-first surface with reusable task and automation layers.

About

CLI-first browser automation for AI agents

Resources

Stars

3 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages