Skip to content

chore(pubsub): Add inventory wait and stream shutdown nack - #36479

Open
aandreassa wants to merge 2 commits into
mainfrom
pubsub-shutdown-stream-inventory
Open

chore(pubsub): Add inventory wait and stream shutdown nack#36479
aandreassa wants to merge 2 commits into
mainfrom
pubsub-shutdown-stream-inventory

Conversation

@aandreassa

Copy link
Copy Markdown
Contributor

Adds internal primitives in Inventory and Stream in preparation for exposing subscriber shutdown options in a follow-up change:

  1. Added Inventory#wait_until_empty(timeout) to block until messages finish processing or time out.
  2. Updated Stream#stop and Stream#wait! to support immediate nacking and waiting for message processing to complete.
  3. Added unit tests in inventory_test.rb and stream_test.rb.

Adds internal primitives in `Inventory` and `Stream` in preparation for exposing subscriber shutdown options in a follow-up change:

1. Added `Inventory#wait_until_empty(timeout)` to block until messages finish processing or time out.
2. Updated `Stream#stop` and `Stream#wait!` to support immediate nacking and waiting for message processing to complete.
3. Added unit tests in `inventory_test.rb` and `stream_test.rb`.
@aandreassa
aandreassa marked this pull request as ready for review August 28, 2026 16:53
@aandreassa
aandreassa requested review from a team and yoshi-approver as code owners August 28, 2026 16:53
@aandreassa
aandreassa requested a review from torreypayne August 28, 2026 16:53

@torreypayne torreypayne left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also include some context from the design docs on what this is solving/how it rolls up to the larger initiative?

Comment thread google-cloud-pubsub/lib/google/cloud/pubsub/message_listener/inventory.rb Outdated
Comment thread google-cloud-pubsub/lib/google/cloud/pubsub/message_listener.rb
Comment thread google-cloud-pubsub/lib/google/cloud/pubsub/message_listener/stream.rb Outdated
- Allow sequencer to dispatch subsequent ordered messages during wait_for_processing
- Budget timeout across inventory and callback thread pool in wait!
- Interrupt inventory wait_until_empty when stopped
- Add unit tests for ordered message shutdown, timeout budgeting, and inventory stop
@aandreassa

Copy link
Copy Markdown
Contributor Author

@torreypayne thanks for the feedback. What kind of context were you hoping for?

Today, stopping a subscriber abandons buffered messages in memory without NACKing them, leaving them in limbo until server-side ack deadlines expire, while queued callbacks are abruptly dropped.

This initiative introduces two standard behaviors:

  • nack_immediately: Actively NACKs all held messages on shutdown so other workers can process them immediately without latency.
    -wait_for_processing: True graceful shutdown, stops pulling new messages but drains in-memory messages before exiting.

I am adding some docs in the last PR as well!

@aandreassa
aandreassa requested a review from torreypayne August 28, 2026 21:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants