Repository navigation
Add support for Microsoft VibeVoice models #42758
Description
Model description
Model Details + Issue Description
VibeVoice-1.5B and VibeVoice-Realtime-0.5B, Microsoft TTS models. It seems that these models are not yet available in Transformers. When trying to load them with the following code:
from transformers import pipeline
pipe = pipeline("text-to-speech", model="microsoft/VibeVoice-1.5B")I get the error below:
ValueError: The checkpoint you are trying to load has model type `vibevoice` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
You can update Transformers with the command `pip install --upgrade transformers`. If this does not work, and the checkpoint is very new, then there may not be a release version that supports this model yet. In this case, you can get the most up-to-date code by installing Transformers from source with the command `pip install git+[https://github.com/huggingface/transformers.git`](https://github.com/huggingface/transformers.git%60)Am I doing something wrong or checkpoint for these models is not going to be available in Transformers.
Open source status
- The model implementation is available
- The model weights are available
Provide useful links for the implementation
https://huggingface.co/microsoft/VibeVoice-1.5B
https://huggingface.co/microsoft/VibeVoice-Realtime-0.5B
Activity
Rocketknight1 commentedon Dec 11, 2025
These models aren't yet supported in Transformers, and didn't supply remote modeling code. Their structure is a little unusual, but if someone wants to attempt a PR to add them we'd definitely be interested.
Aniketsy commentedon Jan 3, 2026
@Rocketknight1 I’m relatively new to adding new model architectures to Transformers. would you be able to suggest one or two existing TTS models in the codebase that would be good references for implementing
VibeVoice, in terms of structure or pipeline integration?That would help me follow established patterns and keep the PR aligned with Transformers conventions. Thanks!
ebezzam commentedon Jan 21, 2026
The VibeVoice TTS models are being added with these PRs:
JagjotB commentedon Sep 2, 2026
Hi! I noticed that VibeVoice-1.5B support recently landed in #40546, while VibeVoice-Realtime-0.5B still appears to need native Transformers integration.
I’d be interested in taking on the Realtime-0.5B portion of this issue. I’ve started looking through the existing VibeVoice integration and Microsoft’s streaming implementation to understand the architectural differences.
Would #40546 be the preferred reference for structuring the integration, and is anyone currently working on the Realtime model?
ebezzam commentedon Sep 2, 2026
hi @JagjotB, yes VibeVoice 1.5B/7B is in! Next would be the realtime model. I started it some time back with #42781, but it was on hold since it depends on the main VibeVoice model to best make use of our modular philosophy. The main architecture different iirc is that they use two LMs.
Going forward: that PR needs to be synced with main. Probably easier for me to do that. But if you urgently need the model, you can open a PR off #42781 and try doing the sync?
JagjotB commentedon Sep 2, 2026
@ebezzam Thanks for the context! I don’t urgently need the model, so I’m happy to let you handle the sync since you already have the implementation underway.
I’d still be interested in contributing to the Realtime integration once it’s synced. If there are any pieces that would be useful to split out, such as tests, pipeline/processor integration, generation behavior, docs, or follow-up fixes, I’d be happy to take one on.
fix(deps): #232 - revert transformers floor to >=5.16.1,!=5.17.0
fix(mcp): #226/#233 - VITS default for generate-speech template, mess…
Develop (#241)
ArthurZucker commentedon Oct 20, 2026
Good news: this is now supported in transformers as
vibevoice(added in #40546).