Skip to content

grpc streaming example crashes and hangs #515

Closed
Closed
grpc streaming example crashes and hangs#515
Assignees
jerjoupuneith
Labels
ML

Description

@thecloudist
thecloudist
opened on Sep 12, 2016

Just cloned the latest repo for grpc -
running this on Raspbian (Raspberry Pi)

Cannot even break out of the program at all.
The sample rate of my usb mic is limited to 44100 and 48000. I tried changing those in the example but I get the same results.

I have pyaudio installed up to date.
Sound recording and playback both work with pyaudio.

The ~/.asoundrc script is here:

pcm.!default {
    type asym
    playback.pcm "hw:0,0"
    capture.pcm "hw:1,1"
}

# pcm.!default {
#        type hw
#        card 0
# }

ctl.!default {
        type hw
        card 0
}

#pcm.record {
#    type plug;
#    slave {
#        pcm "hw 1,1"
#    }
# }

Here is the output of the program.

(grpc) pi@raspberrypi:~/python-docs-samples/speech/grpc $ python transcribe_streaming.py
ALSA lib pcm_dmix.c:1022:(snd_pcm_dmix_open) unable to open slave
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pulse.c:243:(pulse_connect) PulseAudio: Unable to connect: Connection refused

ALSA lib pulse.c:243:(pulse_connect) PulseAudio: Unable to connect: Connection refused

ALSA lib pcm_dmix.c:1022:(snd_pcm_dmix_open) unable to open slave
Cannot connect to server socket err = No such file or directory
Cannot connect to server request channel
jack server is not running or cannot be started
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/pi/gopigo/Projects/Awareness/voice/python-docs-samples/speech/grpc/grpc/local/lib/python2.7/site-packages/grpc/_channel.py", line 184, in consume_request_iterator
    for request in request_iterator:
  File "transcribe_streaming.py", line 112, in request_stream
    data = audio_stream.read(chunk)
  File "/home/pi/gopigo/Projects/Awareness/voice/python-docs-samples/speech/grpc/grpc/local/lib/python2.7/site-packages/pyaudio.py", line 608, in read
    return pa.read_stream(self._stream, num_frames, exception_on_overflow)
IOError: [Errno -9981] Input overflowed

Activity

  1. theacodes
    added
    ML
    on Sep 12, 2016
  2. theacodes
    assigned
    jerjou
    and
    puneith
    on Sep 12, 2016

  3. jerjou commented on Sep 12, 2016

    @jerjou
    jerjou
    on Sep 12, 2016
    Contributor

    Would you mind posting a snippet of the code that works with pyaudio, and the output that you get?

  4. thecloudist commented on Sep 12, 2016

    @thecloudist
    thecloudist
    on Sep 12, 2016
    Author

    Here is what I have working:

    pyaudio_examples_steve.zip

  5. jerjou commented on Sep 12, 2016

    @jerjou
    jerjou
    on Sep 12, 2016
    Contributor

    Two things:

    • Can you post the output that you get when you run the code that works?
    • Can you post the output of git diff on the directory where you're running transcribe_streaming.py?

  6. thecloudist commented on Sep 12, 2016

    @thecloudist
    thecloudist
    on Sep 12, 2016
    Author

    will do. thanks

  7. thecloudist commented on Sep 12, 2016

    @thecloudist
    thecloudist
    on Sep 12, 2016
    Author

    The output when it is successful:

    pi@raspberrypi:~/gopigo/Projects/Awareness/voice/pyaudio_test $ python pyaudio_test.py
    ALSA lib pcm_dmix.c:1022:(snd_pcm_dmix_open) unable to open slave
    ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
    ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
    ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
    ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
    ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
    ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
    ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
    ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
    ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
    ALSA lib pulse.c:243:(pulse_connect) PulseAudio: Unable to connect: Connection refused

    ALSA lib pulse.c:243:(pulse_connect) PulseAudio: Unable to connect: Connection refused

    ALSA lib pcm_dmix.c:1022:(snd_pcm_dmix_open) unable to open slave
    Cannot connect to server socket err = No such file or directory
    Cannot connect to server request channel
    jack server is not running or cannot be started

    • recording
    • done recording
      pi@raspberrypi:~/gopigo/Projects/Awareness/voice/pyaudio_test $ aplay pyaudio-output.wav
      Playing WAVE 'pyaudio-output.wav' : Signed 16 bit Little Endian, Rate 48000 Hz, Mono

    The diff is in the attachment:

    grpc_diff.txt

  8. thecloudist commented on Sep 12, 2016

    @thecloudist
    thecloudist
    on Sep 12, 2016
    Author

    By the way:

    This works --- >

    (grpc) pi@raspberrypi:/python-docs-samples/speech/grpc $ python transcribe.py gs://voice_test/audio.flac
    alternatives {
    transcript: "how old is the Brooklyn Bridge"
    confidence: 0.982678949833
    }
     pi@raspberrypi:    /python-docs-samples/speech/grpc $

  9. jerjou commented on Sep 12, 2016

    @jerjou
    jerjou
    on Sep 12, 2016
    Contributor

    I'm not seeing the diff? I'm seeing a dump of your audio devices in the attachment..

  10. thecloudist commented on Sep 12, 2016

    @thecloudist
    thecloudist
    on Sep 12, 2016
    Author

    Sorry: 1 sec --
    grpc_diff.txt

  11. jerjou commented on Sep 12, 2016

    @jerjou
    jerjou
    on Sep 12, 2016
    Contributor

    Okay, I can't seem to reproduce the error on my box (I don't have a raspberry pi handy at the moment).

    Googling around, it looks like it happens because audio data is coming in faster than you're using it, and the portaudio buffer overflows. I'd guess that this is because, on the raspberry pi, the hardware is just a tad too slow / constrained to keep up with the sampling.

    I would try playing with your chunk size (increasing it, perhaps?) and using the lowest sample rate your audio device supports (44100, from what you said). There are also a number of workarounds posted in that stackoverflow thread that you might try.

    Notably, since you know what platform you're running it on (I used pyaudio in this sample mostly because it's cross-platform), one answer suggests using the alsaaudio library instead of pyaudio, which may get you the efficiency needed to keep up with the input stream.

    Alternatively, you could spin up a thread to consume and store the input stream, and then have the request_stream function grab whatever's been buffered and send that to the api. You'll have to deal with locking a shared object between threads (which is always a pain), but this is probably the Right Way to do it.

    Let us know what you try / what results you get!

  12. theacodes commented on Sep 12, 2016

    @theacodes
    theacodes
    on Sep 12, 2016
    Contributor

    @jerjou if you'll remind me next time you come to Seattle I can bring in my RPi2.

  13. thecloudist commented on Sep 12, 2016

    @thecloudist
    thecloudist
    on Sep 12, 2016
    Author

    @jerjou - I concur with your analysis. If the alsaaudio example works, that is it can keep up with the sampling, then it does suggest that this is an efficiency issue. I am also looking at what the enormously popular (but somewhat limited) Amazon Echo 'on yur pi' project does to guarantee audio can get to the service and responses come back pretty quickly on a Raspi.

    I am building a voice-controlled robot car, not a virtual assistant. Although, conceptually, I could with the GCP/Voice, NLP, etc, etc which I strive to do eventually.

    What I need to do first if I want to use the streaming grpc service is to get a test program working. Once there, I can slice and dice things, build a simple library which my mains can call and I know it will work each time. Right now, I cannot get past this problem.

    FYI - Googlers: Kas Sata and Glen Shires @io16 built a gopigo voice-controlled car with vision based on GCP that was the inspiration for my project. If you have access to them, they may know how to get this working on a Pi.

    Cheers!

  14. jerjou commented on Sep 12, 2016

    @jerjou
    jerjou
    on Sep 12, 2016
    Contributor

    I just pushed a quick-and-dirty version that fills an input buffer in a separate thread. See if this works for you:

    https://github.com/GoogleCloudPlatform/python-docs-samples/blob/threaded-speech/speech/grpc/transcribe_streaming.py

  15. 8 remaining items

  16. jerjou commented on Sep 19, 2016

    @jerjou
    jerjou
    on Sep 19, 2016
    Contributor

    FYI cleaned up the thread handling, and merged into master.

  17. engelke
    added a commit that references this issue on Nov 28, 2022

    chore(deps): update dependency pytest to v7.2.0 (#515)

    96eafbf
  18. dandhlee
    added a commit that references this issue on Nov 29, 2022

    chore(deps): update dependency pytest to v7.2.0 (#515)

    97dede1
  19. telpirion
    added a commit that references this issue on Jan 18, 2023

    chore(deps): update dependency flask to v2.1.2 (#515)

    0cfc1a5
  20. dandhlee
    added a commit that references this issue on Feb 6, 2023

    chore(deps): update dependency flask to v2.1.2 (#515)

    0a196ad
  21. telpirion
    added a commit that references this issue on Mar 13, 2023

    chore(deps): update dependency flask to v2.1.2 (#515)

    6969c51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

ML

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions