Skip to content

multi-speaker tacotron2 enhancements - #2261

Merged
mravanelli merged 2 commits into
speechbrain:developfrom
BenoitWang:mstts_pr
Dec 13, 2023
Merged

mravanelli merged 2 commits into
speechbrain:developfrom
BenoitWang:mstts_pr

Conversation

@BenoitWang

@BenoitWang BenoitWang commented Nov 22, 2023 •

Copy link
Copy Markdown
Collaborator

What does this PR do?

Hi @mravanelli @pradnya-git-dev, as discussed, this PR contains some enhancements of the multi-speaker tacotron2 (#2120) based on my experiments.

The modifications are:

  • Use characters instead of phonemes (which shows better alignments with our tacotron)
  • Do 16k generation instead of 22k with the 16k hifi-gan pretrained
  • Limit the maximum audio length to 10 seconds
  • Train with more speakers (add train-clean-360 into training set)
  • Make MS-Tacotron2 a child class of Tacotron2
  • Update the interface and the model
  • recipe tests

@mravanelli mravanelli added enhancement New feature or request work in progress Not ready for merge labels Nov 29, 2023
@BenoitWang

Copy link
Copy Markdown
Collaborator Author

Hi @mravanelli @pradnya-git-dev , for now I'm still trying to make the alignments work with more speakers by adding train-clean-360, the experiments may take long, I think we can merge these first modifications while waiting.

@mravanelli
mravanelli marked this pull request as ready for review December 13, 2023 18:02
@mravanelli

Copy link
Copy Markdown
Collaborator

Thank you @BenoitWang for the improvements. I tested and everything runs. As mentioned, we can merge this PR and follow up in the future with another one for further changes.

@mravanelli
mravanelli merged commit f285f19 into speechbrain:develop Dec 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request work in progress Not ready for merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants