feat: optional concurrent async automatic function calling - #2813
Open
a2105z wants to merge 2 commits into
Open
Conversation
Add AutomaticFunctionCallingConfig.run_concurrently so async AFC can execute multiple tool calls from one model response via asyncio.gather. Defaults remain sequential. Fixes googleapis#2281.
Author
|
@Venkaiahbabuneelam this PR adds optional AutomaticFunctionCallingConfig.run_concurrently for parallel async tool execution via asyncio.gather; whenever you have a moment, I would really appreciate your review — thank you for your time. |
Annotate func_response as StringDict so the error-path assignment type checks after concurrent AFC extraction.
Author
|
@Venkaiahbabuneelam sorry — mypy failed on this one too. Different issue than the MCP PR: after extracting the async helper, the error-path dict typing got messed up. Pushed a fix; local mypy is clean now. If you can re-approve / re-run checks when you get a chance thatd be awesome. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
AutomaticFunctionCallingConfig.run_concurrentlyso async automatic function calling can execute multiple tool calls from one model response concurrently viaasyncio.gather.generate_content/generate_content_streamand adds unit tests for order, concurrency, and default sequential execution.Test plan
pytest google/genai/tests/afc/test_get_function_response_parts.py(9 passed)run_concurrently=Trueasync gather path; default remains sequential; sync AFC unchangedgenerate_contentwith two slow tools +run_concurrently=Truefinishes near one-tool latency