speech-llm

A speech-to-text system you can train for $25. A frozen speech encoder and a Qwen LLM joined by a small trained projector, with word timestamps, speaker diarization, a batched HTTP server, a live demo, and a free course.

《SpeechGen: Unlocking the Generative Power of Speech Language Models with Prompts》

flow mirror models from JZX AI Labs

DuplexJev-32B and DuplexJev-4B: speech decision models for full-duplex voice agents. Typed single-token readout from a frozen speech LLM, served with vLLM. Try it: api.adventists.cn/duplexjev

End-to-end spoken-dialogue framework: deterministic semantic speech units, unified interleaved token streams with turn-control tokens, a duplex turn-taking state machine with barge-in and latency budgets, deterministic session replay, and turn-taking evaluation on planted synthetic dialogues.

Open research toward end-to-end spoken dialogue systems. Ships SoviaMate-Codec — a neural audio codec for LLM integration with ASR-constrained encoding, enhancement training, and zero-shot speaker adaptation.

Personal research notes and paper summaries in Speech AI — from ASR, TTS, and Enhancement to Speech-Language Models (SpeechLLM) and Audio-Multimodal AI.

Voxtral-based medical domain multimodal AI system. Supports automatic speech recognition (ASR), speech question answering (SQA), and text question answering (TQA).

MEng thesis: calibrated merging of LoRA adapters in a 3B Speech LLM recovers single-task performance on average across 7 speech tasks, using 16× less training data than joint multi-task training.