A working proof-of-concept of Telegram voice calls using Pyrogram can be found here: https://github.com/bakatrouble/pylibtgvoip. Thanks to @bakatrouble.
Note
This page will be updated with more information once voice calls become eventually more usable and more integrated in Pyrogram itself.