Add discovery method to bug report template#32075
Conversation
|
Definitely a good idea, maybe free text might work better than drop down? |
| - Bug affects real work that uses Matplotlib | ||
| - Actively looking for bugs to report |
There was a problem hiding this comment.
| - Bug affects real work that uses Matplotlib | |
| - Actively looking for bugs to report | |
| - Through organic Matplotlib usage and impacts user work | |
| - Systematically searching for bugs (e.g. fuzzing) |
There was a problem hiding this comment.
That is my entry for the color of the bikeshed.
There was a problem hiding this comment.
bikeshed on the bike shed - impacts on/to user work
There was a problem hiding this comment.
I spent a couple minutes trying to think of a 3rd category and came up blank
There was a problem hiding this comment.
I have further mixed Tom's paint suggestion.
| id: discovery | ||
| attributes: | ||
| label: Bug discovery | ||
| description: How did you become aware of this bug? |
There was a problem hiding this comment.
Optional: Simpler language
| description: How did you become aware of this bug? | |
| description: How did find of this bug? |
| label: Bug discovery | ||
| description: How did you become aware of this bug? | ||
| options: | ||
| - Bug affects real work that uses Matplotlib |
There was a problem hiding this comment.
| - Bug affects real work that uses Matplotlib | |
| - While using Matplotlib |
There was a problem hiding this comment.
I wanted to avoid the implied assumption that the reporter is the user. They could be a support person or other colleague helping the user with debugging. They could be a downstream maintainer re-raising something reported to their repo, or raising something that broke in their CI.
There was a problem hiding this comment.
I read "while using" as "while someone was using Matplotlib", which does "During usage of Matplotlib" or a modification of Tom's suggestion "Through organic usage of Matplotlib" work?
There was a problem hiding this comment.
Which even shorter "Organic Matplotlib Use"
Let’s stick with a selection. This is simpler for users as they don’t need to come up with a description and simpler for us as we can quickly interpret the standard answers and don’t have to interpret user text. IMHO the added level of precision of a free text is not needed here. Possibly add “Other” as a third option so that users don’t have to select a seemingly improper option. |
|
I would be less concerned with how the bug/flaw was found and more concerned with whether it matters. I'd suggest a checkbox or amendment of the reproducer checkbox that makes it clear we will prioritize issues with practical end-use examples, not standalone theoretical issues. |
That works for me too. The free text explains too is probably covered by the reproducer. |
I think how it was found is a factor in how much it matters: if someone reports a bug that came up while doing their science then they clearly thought it was important enough to take time out of their day to report. Someone who is actively looking for bugs to report doesn’t have that same friction of whether bug-reporting is something they want to spend their time on. The response can also vary depending on how it was found. If there is a genuine user, then they will be happy if we can provide a workaround or explain a different way to do it (whether we ultimately fix the bug or not). If there is no user then trying to work out how to do the thing is wasted effort. |
| attributes: | ||
| label: Code for reproduction | ||
| description: >- | ||
| If possible, please provide a minimum self-contained example. If you |
There was a problem hiding this comment.
| If possible, please provide a minimal self-contained example, preferably demonstrating a visualization end-use case. If you |
There was a problem hiding this comment.
This makes perfect sense when addressing the bug auto-discoverer, but I'm not sure how it reads to the inexperienced user who is only just learning how to make a minimal reproducer. Is it more confusing?
There was a problem hiding this comment.
Sure, maybe not well worded. A few iterations with Claude (;-) yielded:
If possible, show a real plotting task that reproduces the issue. Keep it self-contained with all necessary code and data, but focus on actual usage, not synthetic edge cases.
|
@rcomer I understand and agree that it is the case that automated reports are less likely to show important issues. However, I think it would be helpful for the user to try to craft a reproducer that uses this in an actual use case - I've suggested a change above. If an automated tool did this, and it seemed a reasonable use case, I'd still think we would investigate it. I'm more likely to get excited about a logic flaw in Grouper if a flaw in end user behaviour can be demonstrated. |
|
@jklymak OK I think I follow you now and agree that the |
PR summary
Recently it seems people use AI to search for Matplotlib bugs in order to report them. Examples RippleTester and probably williamlus and smith2936.
This is not necessarily good or bad, but I think it would be useful to know up front whether the report comes from a genuine use case. If we know it did not, we do not waste time asking the reporter "what were you really trying to do?"
Bikeshedding on the exact wording welcome!
AI Disclosure
No AI used
PR checklist