Add an AI skill that helps create bug reports#33708
Add an AI skill that helps create bug reports#33708betatim wants to merge 4 commits intoscikit-learn:mainfrom
Conversation
This skill walks the user through the process of creating a good bug report. It asks questions and helps the user fill in each section of the issue template.
| - Keep lines to 79 characters where practical. | ||
| - The script should be copy-pasteable into a Python terminal and reproduce the | ||
| problem with no modifications. | ||
|
|
There was a problem hiding this comment.
Shall we cross-reference doc/developers/minimal_reproducer.rst or will this waste tokens for little extra benefits compared to this high level summary?
There was a problem hiding this comment.
I asked the agent (cursor) to evaluate what is in the skill (the list here), what is in the reproducer guide, compare the two and come up with reasons to include a reference and reasons not to include it. It thinks that the guide doesn't really cover much beyond what an agent already knows. I think that the reproducer guide mostly contains stuff that is "obvious" in terms of how to make a good reproducer. Obvious in the sense that it is good advice for any and all projects. So I think that LLMs already contain that knowledge.
It did suggest that it might be worth including it as a link when it presents the reproducer to the user. So that the human has a way to evaluate the reproducer. That seems like a reasonable idea, though I wonder how many humans will follow the link.
|
|
||
| ## Step 2: Triage | ||
|
|
||
| Before drafting anything, determine what kind of problem the user has. |
There was a problem hiding this comment.
Shall we refer to the agent operator as "the user"? I find it ambiguous as it could be confused with other users of the scikit-learn library. In #33780 I chose to refer to that person as "the agent operator" or "the human operator".
There was a problem hiding this comment.
However, maybe this is a common naming convention in skill files. If so maybe it's fine to keep this as is.
There was a problem hiding this comment.
I have no idea :-/ I feel like "the user" should be a good description, "agent operator" feels very clunky.
What could "the user" be misinterpreted as? Maybe I am thinking too much like a human but if I imagine myself as the agent reading the skill, then I'd understand "the user" as the person using me.
Sounds good to me. In addition, we could also mention the |
Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
We want the human to write the "interest in fixing" section, the agent shouldn't fill it out for them.
Reference Issues/PRs
Related to #33398
What does this implement/fix? Explain your changes.
This skill walks the user through the process of creating a good bug report. It asks questions and helps the user fill in each section of the issue template.
It is a bit tricky to test this skill because I don't know of a bug. I will try and find one from the existing open issues and try to reproduce it locally and then see if the skill activates and does the right thing.
Cursor and Claude look for skills in different locations, so I created a
skills/directory to store the actual skill and then symlinked it from the editor specific locations. For other editors we probably need more symlinks. This seems like the least bad option, though ideally all the editors would look in one common location :-/AI usage disclosure
I used AI assistance for:
Any other comments?
How to place this skill in a place that your agent discovers when you install scikit-learn as a package (not using the git repo) is something I don't yet know how to do. My best idea so far is that you'd have to run some kind of command after installing the package that detects which agent you are using and installs the skill for you. I think we can already merge this PR without figuring out how to solve the install question. I'm thinking that if we only have it in the repo we will get a few users using it (maybe the core maintainers) and see if the skill is useful. In parallel we can figure out the install question and then if we like it all make it so all users get this skill (and other skills).