Pubsub API redesign broke support for PUBSUB_EMULATOR_HOST #3886

Activity

1. bpicolo

   changed the title [-]Pubsub API redesign doesn't support PUBSUB_EMULATOR_HOST[/-] [+]Pubsub API redesign broke support for PUBSUB_EMULATOR_HOST[/+] on Aug 27, 2017
2. lukesneeringer commented on Aug 28, 2017

   

   lukesneeringer

   on Aug 28, 2017

   Contributor

   More actions

   Hi @bpicolo,
   Thanks for letting me know.

   I admit unfamiliarity with the Pub/Sub emulator. I will look into what I need to do to make this work for you. If it is just an alternative hostname, that will work, but the incantation is slightly different.
3. dhermes commented on Aug 28, 2017

   

   dhermes

   on Aug 28, 2017

   Contributor

   More actions

   @lukesneeringer Let's chat about this. I had great support for Pub/Sub, Bigtable and Datastore, but it got nuked when we switched from tox to nox.
4. bpicolo commented on Aug 28, 2017

   

   bpicolo

   on Aug 28, 2017

   Last edited by bpicolo

   Author

   More actions

   @lukesneeringer Should just be the hostname I would think (maybe https cert checking and the like as well, unsure what the actual code path looks like)
   Seems worthwhile to add some sort of acceptance tests for this?
5. lukesneeringer commented on Aug 28, 2017

   

   lukesneeringer

   on Aug 28, 2017

   Contributor

   More actions

   @bpicolo Probably.:-)

   If it is just hostname, you can send the service_path keyword argument to the client (it gets passed to that object if added in **kwargs here.
6. dhermes commented on Aug 28, 2017

   

   dhermes

   on Aug 28, 2017

   Contributor

   More actions

   We previously did not have any CI testing for the emulator integration, but now that we control the Docker image for CircleCI, it would be very easy to add the gcloud SDK (where the emulators live) to the Docker image. Yay!
7. hannesholst commented on Aug 29, 2017

   

   hannesholst

   on Aug 29, 2017

   Last edited by hannesholst

   More actions

   @lukesneeringer & @dhermes Is there an ETA for a fix? The workaround with the **kwargs doesn't work since there's still some https stuff failing alongside it...
8. wscheep commented on Sep 4, 2017

   

   wscheep

   on Sep 4, 2017

   More actions

   Same problem here, hoping for a quick fix as we're using the emulator to test most of our services..

   👍React with 👍7Reacted by Thijs Terlouw, Auke Bakker, Julien Singler, Alexandre Da Costa, Eric Berlin, Jeff Payne and Uwe Jugel
9. cgthayer commented on Sep 18, 2017

   

   cgthayer

   on Sep 18, 2017

   More actions

   :-( uhhh, how were the API changes tested before the release if the emulator broke #test-in-prod

   😕React with 😕4Reacted by Charles Thayer, Auke Bakker, Alexandre Da Costa and Jeff Payne
10. robhaswell commented on Sep 21, 2017

    

    robhaswell

    on Sep 21, 2017

    More actions

    (This issue is a blocker for me deploying this wonderful new client 😢)

    @dhermes

    @lukesneeringer Let's chat about this. I had great support for Pub/Sub, Bigtable and Datastore, but it got nuked when we switched from tox to nox.

    Thank you for your work on emulator support, however for Pub/Sub there was a noticeable drawback, which was that the client library required any valid GCP service account credentials before it would proceed to use the emulator. Hopefully the replaced emulator support won't have this drawback and will use the emulator without credentials, as e.g. the Datastore client does.

    👍React with 👍2Reacted by Alexandre Da Costa and Jeff Payne
11. tseaver

    added

    api: pubsubIssues related to the Pub/Sub API.Issues related to the Pub/Sub API.

    on Sep 22, 2017
12. robhaswell commented on Oct 6, 2017

    

    robhaswell

    on Oct 6, 2017

    More actions

    Hi team, is this issue on the work schedule at all? This part of our system needs some refactoring and I'm trying to work out if we should delay so we can upgrade to the new client. Thanks.

    👍React with 👍3Reacted by Julien Singler, Jorrit Posthuma and Suhaas Prasad
13. suhaasprasad commented on Oct 25, 2017

    

    suhaasprasad

    on Oct 25, 2017

    More actions

    Any workaround here? Passing in the service_path and service_port to the client doesn't seem to work and prints the following error:

    E1025 00:04:54.085080000 140737010254784 ssl_transport_security.c:940] Handshake failed with fatal error SSL_ERROR_SSL: error:100000f7:SSL routines:OPENSSL_internal:WRONG_VERSION_NUMBER.
14. marcfargas commented on Oct 25, 2017

    

    marcfargas

    on Oct 25, 2017

    Last edited by marcfargas

    More actions

    Hi,

    According to Google documentation the developer (users of the library) should not have to do anything in code to use the emulator. Only set the proper environment variable. Also that is the way it worked before this ticket (apparently).

    There are two things missing IMHO: 1) a test case that checks that when the PUBSUB_EMULATOR_HOST is set all requests to PUBSUB are forwarded there, and 2) support inside both the subscriber and the publisher to set service_path and service_port accordingly at runtime without user intervention.

    Also as per @suhaasprasad comment, the client should allow for non-SSL connections when honouring the EMULATOR variable.

    Am I missing something?
15. marcfargas commented on Oct 25, 2017

    

    marcfargas

    on Oct 25, 2017

    More actions

    Temporary workaround (ugly) solves the SSL problem:

            if os.environ.get('PUBSUB_EMULATOR_HOST'):
                grpc_channel = grpc.insecure_channel(os.environ['PUBSUB_EMULATOR_HOST'])
                publisher = pubsub_v1.PublisherClient(channel=grpc_channel)
            else:
                publisher = pubsub_v1.PublisherClient()

    👍React with 👍3Reacted by Mark Whitfield, Rob Haswell and Daniele Esposti
16. suhaasprasad commented on Oct 25, 2017

    

    suhaasprasad

    on Oct 25, 2017

    More actions

    I'm still unclear as to the root issue, but I've reverted to using version 0.27.0 of the google-cloud-pubsub library for now.
17. lukesneeringer

    closed this as completedin #4269on Oct 30, 2017
18. lukesneeringer commented on Oct 30, 2017

    

    lukesneeringer

    on Oct 30, 2017

    Contributor

    More actions

    I just merged a fix for this; release should be soon (sometime this week, hopefully in the early half).

    😄React with 😄2Reacted by euri10 and Rob Haswell

    🎉React with 🎉1Reacted by Marc
19. robhaswell commented on Oct 30, 2017

    

    robhaswell

    on Oct 30, 2017

    More actions

    Amazing, thanks @lukesneeringer.
20. JustinBeckwith

    assigned

    lukesneeringer

    on Feb 1, 2021