inference-serving

Popular repositories Loading

1. energy-inference energy-inference Public

   Forked from grantwilkins/energy-inference

   Code for MPhil Thesis at University of Cambridge

   Python 1

2. infrastructure infrastructure Public

   Jupyter Notebook

3. vllm vllm Public

   Forked from vllm-project/vllm

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python

4. Sequence-Scheduling Sequence-Scheduling Public

   Forked from zhengzangw/Sequence-Scheduling

   PyTorch implementation of paper "Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline".

   Python

5. LLM-serving-with-proxy-models LLM-serving-with-proxy-models Public

   Forked from James-QiuHaoran/LLM-serving-with-proxy-models

   Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Prediction

   Jupyter Notebook

6. pytorch-transformer pytorch-transformer Public

   Forked from hkproj/pytorch-transformer

   Attention is all you need implementation

   Jupyter Notebook