Skip to content

Add hooks for the root repo#186578

Open
reidbaker wants to merge 2 commits into
flutter:masterfrom
reidbaker:r-create-intial-hooks
Open

Add hooks for the root repo#186578
reidbaker wants to merge 2 commits into
flutter:masterfrom
reidbaker:r-create-intial-hooks

Conversation

@reidbaker
Copy link
Copy Markdown
Contributor

Add hooks for analyze and format to the flutter root .agents directory.
Hooks are a deterministic way to tie into the agent lifecycle in the google internal version of antigravity.
Stop means when the agent is done run the scrips defined. The scripts included are dart code that limits the format and analyze scope to only files modified in git that are below the directory that contains .agents. This pr runs for the whole repo but if you were to add it to a sub directory and open the internal version of antigravity there it would not run on git changes that were outside of that scope.

If you open your editor to a sub folder as of today 2026-05-15 the hooks will not run.
This does not work with claude but could in the future flutter/skills#140.

Most importantly if you have been struggling with your agent stopping with code that has analyzer or formatting warnings then these hooks could help you be less frustrated.

Pre-launch Checklist

@flutter-dashboard flutter-dashboard Bot added the CICD Run CI/CD label May 15, 2026
@flutter-dashboard
Copy link
Copy Markdown

It looks like this pull request may not have tests. Please make sure to add tests or get an explicit test exemption before merging.

If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix?

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing. If you believe this PR qualifies for a test exemption, contact "@test-exemption-reviewer" in the #hackers channel in Discord (don't just cc them here, they won't see it!). The test exemption team is a small volunteer group, so all reviewers should feel empowered to ask for tests, without delegating that responsibility entirely to the test exemption group.

@reidbaker reidbaker requested review from jmagman and jtmcdole May 15, 2026 14:27
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces agent hooks for Dart formatting and analysis, adds the dart_hooks package as a dependency in dev/tools/pubspec.yaml, and updates the project's lock file. Feedback indicates that the hook commands require an explicit --package-config flag to resolve the dart_hooks package from the correct context. Additionally, the timeout for the analysis hook should be increased to avoid intermittent failures, and the pubspec.lock file needs correction as dart_hooks is currently misidentified as a transitive dependency.

Comment thread .agents/hooks.json Outdated
Comment thread .agents/hooks.json Outdated
Comment thread .agents/hooks.json Outdated
Comment thread pubspec.lock
@github-actions github-actions Bot removed the CICD Run CI/CD label May 15, 2026
Comment thread .agents/hooks.json
"Stop": [
{
"type": "command",
"command": "../bin/cache/dart-sdk/bin/dart --packages=../.dart_tool/package_config.json run dart_hooks:agent_dart_format --source hook --log",
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This whole pattern looks strange, it looks like a case for dart install and executables
https://dart.dev/tools/dart-install
https://dart.dev/tools/pub/pubspec#executables

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you like me to figure out dart install and executables before landing this change?

@reidbaker reidbaker added the CICD Run CI/CD label May 15, 2026
@reidbaker reidbaker requested a review from jmagman May 15, 2026 17:47
Copy link
Copy Markdown
Member

@gmackall gmackall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean that we will always run dart format? That seems like an odd pattern to me if so (though let me know if I am misunderstanding). What if the latest prompt is me asking a clarification question which involves no changes do code, or if my latest request is changes to kotlin/java/c++/BUILD.gn/pubspec.yaml/etc exclusively. Will it mean the agent will run dart format always in those cases?

@reidbaker
Copy link
Copy Markdown
Contributor Author

Does this mean that we will always run dart format? That seems like an odd pattern to me if so (though let me know if I am misunderstanding). What if the latest prompt is me asking a clarification question which involves no changes do code, or if my latest request is changes to kotlin/java/c++/BUILD.gn/pubspec.yaml/etc exclusively. Will it mean the agent will run dart format always in those cases?

This specific pr means if you open the internal version of antigravity to the root of flutter/flutter and when the agent thinks it is done working it will run dart format on the modified dart files in git then run dart analyze --fatal-infos on the modified dart files. If/when one of those fails it will tell the agent it is not done with the reason (the files that need formatting and the analyzer warnings) so it can fix those.

Q: "What if the latest prompt is me asking a clarification question which involves no changes do code" Assuming you have the internal version of antigravity open to the root directory. When the agent thinks it is done the scripts will run and see no modified dart files and exit cleanly. If you have modified dart code then format and analyze will run on the files you have modified.

Q: "If my latest request is changes to kotlin/java/c++/BUILD.gn/pubspec.yaml/etc exclusively [will it run analyze and format]?" the scripts will run and detect no changed dart files and move on. If your latest request touched kotlin files and you have modified non committed dart files they will be formatted and analyzed.

We are limited by the lifecycle events that the internal version of antigravity gives us. To my knowledge, on 2026-05-15, there are no ways to get the files the agent thinks it touched. We could run on file write but agents modify files with sed and scripts and I didnt want to run on every write only when the agent thinks it is "done". Frequently in my experience the agent will run dart analyze as part of its development process but not dart analyze --fatal-infos.

Finally lets say you are working on a test that explicitly needs checked in files that have analyzer warnings. The nice thing about the script as authored is you can check those files in and they will not be analyzed. This exposes a failure mode where the agent can check things in before analyze and formatting them but I wanted an easy to describe bypass path.

I hope that helps.

@reidbaker reidbaker requested a review from gmackall May 15, 2026 18:51
@gmackall
Copy link
Copy Markdown
Member

Q: "If my latest request is changes to kotlin/java/c++/BUILD.gn/pubspec.yaml/etc exclusively [will it run analyze and format]?" the scripts will run and detect no changed dart files and move on. If your latest request touched kotlin files and you have modified non committed dart files they will be formatted and analyzed.

This seems surprising to me. Is this not what additions to the context window such as
https://github.com/flutter/flutter/blob/master/.agents/rules/dart-editing.md
are attempting to solve? Are they not satisfactory?

In general I'm somewhat suspicious of making opinionated global configurations of everyone's agents. If we believe we should always run these commands, why do we choose this hook instead of a git pre submit hook (similar to the engine formatting hook that doesn't allow you to push unformatted code, even to your local branch)

@reidbaker
Copy link
Copy Markdown
Contributor Author

Q: "If my latest request is changes to kotlin/java/c++/BUILD.gn/pubspec.yaml/etc exclusively [will it run analyze and format]?" the scripts will run and detect no changed dart files and move on. If your latest request touched kotlin files and you have modified non committed dart files they will be formatted and analyzed.

This seems surprising to me. Is this not what additions to the context window such as https://github.com/flutter/flutter/blob/master/.agents/rules/dart-editing.md are attempting to solve? Are they not satisfactory?

In general I'm somewhat suspicious of making opinionated global configurations of everyone's agents. If we believe we should always run these commands, why do we choose this hook instead of a git pre submit hook (similar to the engine formatting hook that doesn't allow you to push unformatted code, even to your local branch)

It is what they are attempting to solve and failing to solve. Markdown no matter how aggressively or repeatedly insisted upon can be lost under large context windows and agent do just decide to not follow those instructions. If what you want is deterministic then hooks are one of the few ways to have determinism in an agentic development environment. We dont have the ability to run evals in an IDE right now but from personal experience this is one of the best things you can do to get your agent to author better dart before you look at it. Best of all it is mostly invisible.

"In general I'm somewhat suspicious of making opinionated global configurations of everyone's agents." Fair enough and that is what the review process is for. It is also why I went with what I considered to be the least controversial hooks to start with. If format and analyze fail (which happens a bunch on initial cl upload) then basically all our test get spun up and fail so it prevents you from even seeing what your code breaks.

Q: "why do we choose this hook instead of a git pre submit hook?" Pre submit hooks impact everyone's flow and these only impact agentic flows. The benefit is that the amount of time this adds to what humans are sitting and waiting on is rounded down to zero because you should not be watching your agent work you should be reviewing what happens when it is finished. Where as I might commit work that is in a partial state when I need to change branches. You can get around precommit hooks but you have introduced frustration to a human workflow.
Finally dart-editing.md is attempting to have global (repo wide) opinionated global configurations for everyones agents. The pattern isn't wrong and the subset of things that should be the same for everyone on the repos is worthy of skepticism. This however is not changing any repo wide rules it just makes agents better at following them with an easy bypass for people.

@reidbaker
Copy link
Copy Markdown
Contributor Author

@gmackall go/flutter-pr-186578-context

Copy link
Copy Markdown
Member

@gmackall gmackall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the implementation here would make this change pretty annoying in the bad path. An example:
Recently I made added a new platform view test dart file in dev/integration_tests/android_engine_test/lib/hcpp/ designed to test out some local changes.

I swapped back and forth the master branch and my branch with changes to test out the functionality.

Then I ask a clarifying question about the java changes of my implementation. The agent spends some time thinking and replies. Then the hooks fire and format and analyze my local, untracked test file.

This hijacks the window, replaces the reply with notes about how it fixed analyze and format failures, and notable hides the entire original reply deep in the "worked for N minutes" section (it's not revealed by scrolling up, you need to know to expand the work log and then find it amidst the tool calls and thinking). This is extremely annoying, and would be very confusing to someone who is unfamiliar with this configuration and is focusing only on non-dart changes at the moment (and asked a question unrelated to dart).

@reidbaker
Copy link
Copy Markdown
Contributor Author

reidbaker commented May 15, 2026

My recap of what @gmackall and I talked about out of band.
We have a fundamental disagreement about the theory of putting items in the context window. Including things like the continue json blob of a tool call that is unrelated to a query.

##Specific use case that in theory could be made worse.

  1. A user has modified dart code that is not checked in and not formatted or has analyzer warning.
    A second chat window is opened to ask an agent about any work that is not touching the dart code with the format or analyzer warning.
    the second chat windows context is polluted by fixing the dart analyzer issues.

  2. At scale the tool calls could add a minimal number of tokens due to the continue.
    I offered to do a blind test against an agent that had 1000 tool calls that were no ops to see if @gmackall could tell the difference but we both agreed it was unlikely that we would repeat the test on every new model and so @gmackall considered the test not convincing.

I consider this no different than the skills, mcp servers, rules and agents.md files we already include which are based on their ability to make the agents we have right now better.
Note: As I am typing I realized we could combine the calls into a single tool so that there was only one tool call so even the dimension of number of tool calls on stop would not scale for all tools under our teams control.

##Places we agree

  • The lifecycle events we have access to are not ideal. What we would want are the files that were modified by the agent during a response.
  • This change auto applies to anyone who opens flutter/flutter and there is not an easy the ability to opt out. This is not due to a lack effort. The organizational apis for isolation do not exist. We maybe could create a local file that opts the scripts out of execution but by design that would look the same as no modified files which does not get past @gmackall's theoritecial issue.

Possible avenues forward

  1. Do not land this change and insist that anyone that wants this install it manually.
  2. Do not land this change and wait until there are different lifecycle hooks or different inputs.
  3. Land this change despite the theoretical concerns with no modifications.
  4. Land this change after a head to head test with a large number (1000) of tool calls that are no opts
  5. Land the hooks but in a sub directory that does not have anyone who strongly objects as an active contributor. Possibly the framework team could see if they universally wanted this change.
  6. Something else.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CICD Run CI/CD

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants