Skip to content

Latest commit

 

History

History
 
 

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

README.md

title VisionClaw Documentation
description Complete documentation for VisionClaw — the governed agentic mesh for real-time 3D knowledge graph exploration with GPU-accelerated physics, OWL 2 ontology reasoning, and multi-agent AI orchestration
updated-date 2026-04-18

VisionClaw Documentation

Real-time 3D knowledge graph exploration powered by Rust, CUDA GPU physics, OWL 2 ontology reasoning, and a multi-agent AI mesh.

← Back to Project | Quick Start | API Reference | Architecture


Quick Start

git clone https://github.com/DreamLab-AI/VisionClaw.git
cd VisionClaw && cp .env.example .env
docker-compose --profile dev up -d

Open http://localhost:3001 for the 3D graph interface, http://localhost:4000/api for the REST API, and http://localhost:7474 for the Neo4j browser.

Full setup details: Deployment Guide

Known Issues: Before debugging unexpected behaviour, check KNOWN_ISSUES.md — it tracks active P1/P2 bugs including the Ontology Edge Gap (ONT-001) and V4 delta instability (WS-001).


Documentation Map

graph LR
    subgraph "Tutorials"
        T1[Installation]
        T2[First Graph]
        T3[Neo4j Basics]
    end
    subgraph "How-To"
        H1[Deployment Guide]
        H2[Development Guide]
        H3[Agent Orchestration]
        H4[Features]
        H5[Operations]
    end
    subgraph "Explanation"
        E1[System Overview]
        E2[Backend CQRS]
        E3[Actor Hierarchy]
        E4[Client Architecture]
        E5[Ontology Pipeline]
        E6[Physics/GPU Engine]
        E7[XR Architecture]
        E8[Enterprise Contexts BC11-17]
    end
    subgraph "Reference"
        R1[REST API]
        R2[WebSocket Binary V5]
        R3[Neo4j Schema]
        R4[Agents Catalog]
        R5[Config & Env]
    end
    subgraph "Enterprise"
        ADR40[ADR-040 Identity]
        ADR41[ADR-041 Broker]
        ADR46[ADR-046 UI Arch]
        PRD2[PRD-002 Enterprise]
    end
Loading

Tutorials

Step-by-step lessons that teach VisionClaw by doing.

Tutorial Description
What is VisionClaw? Platform overview and key concepts
Installation Docker and native setup from zero
Creating Your First Graph Build and explore your first knowledge graph
Neo4j Basics Query and navigate the graph database

How-To Guides

Practical task-oriented instructions. See how-to/README.md for the full index.

Deployment & Infrastructure

Guide Description
Deployment Guide Docker Compose production deployment with NVIDIA GPU
Performance Profiling GPU physics, WebSocket, render, and Neo4j bottleneck detection
Quest 3 VR Setup Connect a Meta Quest 3 to VisionClaw's immersive XR mode
Infrastructure Overview Goalie integration and infra architecture
Port Configuration Service port mapping and networking
Infrastructure Tools Container management and diagnostic tools
Infrastructure Troubleshooting Container and networking issues

Development

Guide Description
Development Guide Rust/React setup, project structure, testing workflow
REST API Integration Guide NIP-98 auth, common API workflows, WebSocket combination patterns

Agent Orchestration

Guide Description
Agent Orchestration Deploy, configure, and coordinate the multi-agent AI system

Features

Guide Description
Navigation Guide 3D interface controls and spatial navigation
Filtering Nodes Graph node and edge filtering
Intelligent Pathfinding Semantic shortest-path traversal
Natural Language Queries Plain-English graph search
Semantic Forces Stress-majorisation layout algorithm
Voice Routing 4-plane voice architecture with LiveKit
Voice Integration STT/TTS pipeline setup
Nostr Auth NIP-07/NIP-98 authentication
Auth & User Settings User settings and session management
Ontology Parser OWL 2 parsing from Logseq Markdown
Hierarchy Integration Class hierarchy visualisation
Local File Sync GitHub-to-local file synchronisation
ComfyUI Setup ComfyUI SAM3D integration setup
System Health Monitoring System health monitoring — HealthDashboard, physics status, MCP relay controls
Welcome Tour Welcome tour configuration — steps, flows, skip behaviour, localStorage persistence
Workspace Management Workspace management — save/restore graph configurations, favourites, real-time sync
Command Palette Command palette — Ctrl+K, fuzzy search, custom command registration

Operations & Integration

Guide Description
Configuration Environment variables and runtime settings
Troubleshooting Common issues and diagnostic steps
Security Authentication, secrets management, and hardening
Telemetry & Logging Observability and log configuration
Pipeline Admin API Admin endpoints for pipeline management
Operator Runbook Production operations playbook
Maintenance Routine maintenance and upkeep tasks
Neo4j Integration Neo4j database connection and migration
Solid Integration Solid Pod integration overview
Solid Pod Creation Creating and managing user Solid Pods
ComfyUI Service ComfyUI Docker service integration

Explanation

Conceptual deep-dives that build understanding of how and why VisionClaw works.

Document What it explains
System Overview End-to-end architectural blueprint — all layers and their interactions
Backend CQRS Pattern Hexagonal architecture with 9 ports, 12 adapters, 114 command/query handlers
Actor Hierarchy 21-actor Actix supervision tree — roles, message protocols, failure strategies
Client Architecture React + Three.js component hierarchy, WebGL rendering pipeline, WASM integration
DDD Bounded Contexts Domain-Driven Design context map and aggregate boundaries
DDD Identity Contexts DID/Nostr + PodKey + Passkey identity bounded contexts
DDD Semantic Pipeline Semantic pipeline domain model and context boundaries
Ontology Pipeline GitHub Markdown → OWL 2 → Whelk reasoning → Neo4j → GPU constraints
Physics & GPU Engine CUDA force-directed physics, semantic forces, 55× GPU speedup
XR Architecture WebXR / Babylon.js immersive mode, Vircadia multi-user integration
Security Model Nostr DID auth, Solid Pod sovereignty, CQRS authorization, audit trail
Solid Sidecar Architecture JSON Solid Server sidecar for user Pod storage
User-Agent Pod Design Per-user Solid Pod isolation for agent memory
Technology Choices Rationale for Rust, CUDA, Neo4j, OWL 2, and Three.js selections
RuVector Integration RuVector PostgreSQL as AI agent memory substrate
Blender MCP Architecture Blender remote-control via WebSocket RPC + MCP tools
Deployment Topology Multi-container service map, network architecture, dependency chain, scaling
Agent-Physics Bridge How AI agent lifecycle states synchronise to the 3D physics simulation
DDD Enterprise Contexts (BC11–BC17) Judgment Broker, Workflow Lifecycle, Insight Discovery, Enterprise Identity, KPI Observability, Connector Ingestion, Policy Engine bounded contexts
DDD Insight Migration Context Insight Migration DDD context — MigrationCandidate aggregate in BC13, MigrationCase subtype in BC11, promotion lifecycle

Reference

Technical specifications for APIs, schemas, protocols, and configuration.

Full reference index: reference/INDEX.md

Reference Contents
REST API All HTTP endpoints — graph, settings, ontology, auth, pathfinding, Solid
WebSocket Binary Protocol Binary V2/V3/V4 message formats, connection lifecycle, client implementation
Neo4j Schema Graph node/edge types, ontology nodes, Solid Pod records, indexes
Agents Catalog Complete catalog of specialist agent skills by domain
Error Codes AP-E, DB-E, GR-E, GP-E, WS-E error code hierarchy with solutions
Glossary Definitions for domain terms used throughout the documentation
Physics Parameters UI slider → settings key → CUDA kernel mapping, effective ranges, FastSettle vs Continuous
Performance Benchmarks GPU physics, WebSocket, and API performance metrics
Environment Variables All .env variables with types, defaults, and descriptions
Docker Compose Options Service profiles, volumes, and compose file structure
MCP Protocol Model Context Protocol specification for agent orchestration
Protocol Matrix Transport protocol comparison — WebSocket, REST, MCP
Cargo Commands Rust build, test, and lint commands
Docker Commands Docker and docker-compose operational commands

Architecture Decision Records

Design decisions recorded as ADRs in docs/adr/.

ADR-015 through ADR-026 are not in this repository — those numbers were assigned to decisions that predated the current ADR process and were not backfilled.

Core Platform (ADR-011 to ADR-014)

ADR Title
ADR-011 Authentication Enforcement
ADR-012 WebSocket Store Decomposition
ADR-013 Render Performance Strategy
ADR-014 Semantic Pipeline Unification

Solid / Pod Integration (ADR-027 to ADR-030)

ADR Title
ADR-027 Pod-Backed Graph Views
ADR-028 SPARQL PATCH for Ontology Mutations
ADR-029 Type Index Discovery
ADR-030 Agent Memory Pods

Platform Consolidation (ADR-031 to ADR-039)

ADR Title
ADR-031 Layout Mode System
ADR-032 RATK Integration for WebXR
ADR-033 Vircadia SDK Decoupling
ADR-034 NEEDLE Bead Provenance System
ADR-036 Node Type System Consolidation
ADR-037 Binary Protocol Consolidation
ADR-038 Position Data Flow Consolidation
ADR-039 Settings/Physics Object Consolidation

ADR-035 is absent — the content was renumbered to ADR-037 (ADR-037-binary-protocol-consolidation.md carries the ADR-035 internal heading, a known inconsistency).

Enterprise Governance (ADR-040 to ADR-049)

ADR Status Title
ADR-040 Proposed Enterprise Identity Strategy (OIDC + Nostr)
ADR-041 Proposed Judgment Broker Workbench Architecture
ADR-042 Proposed Workflow Proposal Object Model
ADR-043 Proposed KPI Lineage Model
ADR-044 Proposed Connector Governance and Privacy Boundaries
ADR-045 Proposed Policy Engine Approach
ADR-046 Accepted Enterprise UI Architecture
ADR-047 Proposed WASM Visualization Components
ADR-048 Proposed Dual-Tier Identity Model (KGNode + OntologyClass)
ADR-049 Proposed Insight Migration Broker Workflow

Six of eight enterprise ADRs are Proposed — the features are built and operational but the decisions are pending formal ratification.

RuVector Federation

Document Title
RVF Integration AFD RuVector Federation Architecture Feature Design
RVF Integration DDD RuVector Federation Domain-Driven Design
RVF Integration PRD RuVector Federation Product Requirements

Design Documents

Exploratory design documents in docs/design/.

  • Nostr Relay Integration — Architecture for VisionClaw ↔ Nostr relay bridging
  • Nostr Solid Browser Extension — Browser extension design for Nostr + Solid identity
  • Enterprise Drawer UI — Full specification for the enterprise slide-out drawer: geometry, WASM ambient effects, Zustand store, keyboard shortcut, ARIA, graph dimming, rollback plan
  • Insight Migration Loop — Research corpus for the tacit→explicit knowledge bridging workflow (prior art, bridge theory, physics mapping, acceptance tests, scoring)

Product Requirements Documents

Document Description
PRD-001: Pipeline Alignment Backend physics/settings pipeline alignment requirements
PRD-002: Enterprise UI Enterprise control plane UI requirements (broker, KPI, workflows, connectors, policy)
PRD: Agent Orchestration Improvements Multi-agent orchestration improvements
PRD: Bead Provenance Upgrade NEEDLE-pattern bead provenance lifecycle upgrade
PRD: XR Modernisation WebXR modernisation for Quest 3 and Babylon.js
PRD: Insight Migration Loop The KG→Ontology migration workflow requirements, scoring, state machine, KPIs, risks

Domain Models

Document Description
DDD Bead Provenance Context Bead lifecycle domain model — content-addressed provenance events
DDD XR Bounded Context XR presence domain model — avatar, session, spatial audio

Audits

QE audit reports in docs/audits/.

Report Description
QE Enterprise Audit Enterprise feature security, accessibility, and test coverage audit (2026-04)
2026-04-17 Master Report Full QE fleet audit — frontend graph loading, backend settings routing, failure patterns, regression risk
Frontend Graph Loading Graph worker, WebSocket, binary protocol analysis
Backend Settings Routing Settings PUT pipeline and actor routing audit
Similar Failure Patterns Cross-system failure pattern catalogue
Regression Risk Risk matrix for recent physics/WebSocket changes
Regression Tests Regression test specifications

Use Cases

Industry applications and positioning in docs/use-cases/.

Document Description
Use Case Overview Cross-industry application summary
Industry Applications Sector-specific use cases (creative, research, enterprise)
Competitive Analysis Positioning against alternative approaches
Quick Reference One-page capability and positioning card

Other

Document Description
Testing Guide Unit, integration, and E2E testing strategy
Security Security model, threat surface, and hardening guidance
Infrastructure Inventory Container services, ports, and environment inventory
Contributing Contribution workflow, branching conventions, code standards
Changelog Version history and release notes
Use Cases Industry use cases and case studies
Git Support Git workflow and branching strategy

Maintained by DreamLab AI — Issues | Discussions