Skip to content

Latest commit

 

History

History
47 lines (35 loc) · 1.74 KB

File metadata and controls

47 lines (35 loc) · 1.74 KB

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[Unreleased]

0.1.0 - 2025-01-15

Added

Core Features

  • Analysis Pipeline: Token counting, chunking, salience scoring, and redundancy detection
  • Compression Strategies: Deduplication, summarization, sliding window, and hybrid approaches
  • Budget Optimization: Greedy and ILP-based chunk selection under token limits
  • Tokenization: Support for GPT-4, Claude, and Llama models via tiktoken and sentencepiece
  • Embeddings: OpenAI embedding provider with fake provider for testing

Interfaces

  • Python SDK: Async API with analyze(), compress(), and optimize() functions
  • CLI: Commands for analyze, compress, and viz operations
  • REST API: FastAPI-based API with endpoints for analysis, compression, and visualization
  • Web UI: SvelteKit dashboard with interactive charts and visualizations

Infrastructure

  • Storage: SQLite + JSONL persistence for analysis runs
  • Testing: Comprehensive test suite with 80%+ coverage
  • Documentation: MkDocs site with tutorials and API reference
  • CI/CD: GitHub Actions for testing, linting, and releases
  • Docker: Containerized API and web applications

Visualization

  • Embedding scatter plots (UMAP projection)
  • Token timeline charts
  • Interactive run dashboard
  • CLI text-based visualizations

Documentation

  • Quick start guide
  • Three detailed tutorials
  • Complete API reference
  • Example notebooks and scripts