CI: Implement conditional verbose logging [verbose]#32081
Draft
Aaratrika-Shelly wants to merge 1 commit into
Draft
CI: Implement conditional verbose logging [verbose]#32081Aaratrika-Shelly wants to merge 1 commit into
Aaratrika-Shelly wants to merge 1 commit into
Conversation
1 task
Member
|
Thanks for the contribution. IMHO this doesn't really help with the problem. Flakiness means that you cannot systematically test this; i.e. you cannot take a PR with a failing run, add the verbose label and rerun. There are only two approaches that can help:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is a follow up to the discussion in #31087 and the disscusion regarding my GSOC proposal on discoourse (https://discourse.matplotlib.org/t/gsoc-2026-intro-aaratrika-shelly-indirecttransforms-test-hardening/26173/10)
PR summary
This PR implements the verbose toggle for the Matplotlib CI pipelines (Github Actions, Azure Pipelines and AppVeyor).
Right now the logs dont provide much information while debugging test flakiness. This makes finding the issue really difficult. To get the logs, we need to manually edit the config files, which is time consuming.
This PR helps automate the process by introducting pytest -vv command. This command lets aanyone (maintainer or contributer) trigger the verbose logs without making permanent config changes. It helps to get logs for each run, which in turn helps diagnose the flaky tests I identified during my GSOC 2026 research.
I used PYTEST_ADDOPTS as an environment variable since it is a native functionality of pytest.
-I introduced two triggers to cater to 2 different users.
Commit Message- for contributors/
Github Label(ci:verbose) for maintainers: They can turn on verbosity for PR without creation of a new commit
Consistency across Platforms: By making it available via GitHub Actions (Linux/macOS), Azure Pipelines (Windows) and AppVeyor (Legacy Windows), I was able to provide consistency of debugging experience for the community members, whatever platform they use when encountering a problem.
I have opened this as a Draft to verify the trigger logic across all CI runners. I've included
[verbose]in this commit to verify the behavior in the logsAI Disclosure
Proofreading PR desciption.
PR checklist
cc: @rcomer, @story645, @timhoffm