Video Composer Agent

pip install uv

uv add -r requirements.txt

Create a .env file with the following variables:

ANTHROPIC_API_KEY=sk-ant-...
HUGGINGFACE_TOKEN=hf_...
# Make sure the browser supports common audio/video codecs
PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH="/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"
# Alternatively, you can use a remote browser (connect over cdp)
# PLAYWRIGHT_WEB_SOCKET_URL=ws://localhost:3000

• Add auto-embedding of new docs
  • Add a tool to fetch the latest docs from the operator-ui repo
  • Checks hash of the file to see if it has changed
  • If it has changed, parse --- as chunks feed into embed pipeline
  • If it has not changed, skip embedding and just use the existing vector db
• Add BM25 to enable hybrid search
• Add MCP integration

  MCP is an open protocol that standardizes how applications provide context to LLMs. Think of MCP like a USB-C port for AI applications. Just as USB-C provides a standardized way to connect your devices to various peripherals and accessories, MCP provides a standardized way to connect AI models to different data sources and tools. Also see: Simple Chatbot

• Add DAPI support for speech recognition and synthesis workloads
• Add VideoLLaMA support
• Add State management for VideoEditor Tool
• Reuse browser session for forward steps and render the composition when editing instructions are fulfilled
• Feed browser session recording back to video undertanding model and enable agent to call pause/play/seek

To run the main script:

uv run main.py

Feel free to modify the main.py script to add new tools and modify the agent's behavior.

The docs_embedder.py script provides tools for embedding, searching, and evaluating documentation:

# Auto-embed from URL (default: operator-ui.vercel.app/llms.txt)
uv run docs_embedder.py auto-embed

# Debug mode (first 5 chunks only)
uv run docs_embedder.py auto-embed --debug

# Force update embeddings (ignore hash check)
uv run docs_embedder.py auto-embed --force

# Custom URL and hash file
uv run docs_embedder.py auto-embed --url "https://your-url.com/docs.txt" --hash-file "path/to/hash.txt"

# Basic search
uv run docs_embedder.py search "how to add text overlay"

# With reranking (more accurate)
uv run docs_embedder.py search --rerank "how to add text overlay"

# Limit results
uv run docs_embedder.py search --limit 10 "how to add text overlay"

# Show full content
uv run docs_embedder.py search --full-content "how to add text overlay"

# Filter by file
uv run docs_embedder.py search --filepath "video-clip.md" "how to trim video"

# Run evaluation on QnA pairs
uv run docs_embedder.py eval

The eval command:

• Tests search against known question/answer pairs
• Compares vector search vs reranking
• Shows metrics like accuracy, rank@1, and semantic matches
• Displays detailed results for each query