📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.🎉

A High-Performance LLM Inference Engine with vLLM-Style Continuous Batching

Implementation of PagedAttention from vLLM paper - a breakthrough attention algorithm that treats KV cache like virtual memory. Eliminates memory fragmentation, increases batch sizes, and dramatically improves LLM serving throughput.

(Experimental) A high-throughput and memory-efficient inference and serving engine for LLMs with a optimized GB10 kernel

High-performance On-Device MoA (Mixture of Agents) Engine in C++. Optimized for CPU inference with RadixCache & PagedAttention. (Tiny-MoA Native)

A from scratch LLM inference engine build in PyTorch with custom GPT2/LLaMA/ transformers, kv cache, paged kv cache, continuous batching and A100 benchmarks

AI Infra 手写算子实现Qwen2推理，支持Paged Attention

Discrete-event simulator for LLM inference serving — PagedAttention memory management and continuous batching

LangChain integration for Parallel Context-of-Experts Decoding (PCED)

🤖 Enhance task management with Tiny MoA, a GPU-free multi-agent system that plans, reasons, and collaborates efficiently in real time.

vLLM - High-throughput, memory-efficient LLM inference engine with PagedAttention, continuous batching, CUDA/HIP optimization, quantization (GPTQ/AWQ/INT4/INT8/FP8), tensor/pipeline parallelism, OpenAI-compatible API, multi-GPU/TPU/Neuron support, prefix caching, and multi-LoRA capabilities

High-Performance LLM Inference Engine with PagedAttention & Continuous Batching | 高性能LLM推理引擎 - 内存浪费<5%, 吞吐率+50%