Skip to content

Latest commit

 

History

History

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

README.md

.github/workflows/

This directory hosts every GitHub Actions workflow that runs in microsoft/testfx, including the AI-powered agentic workflows generated with the gh aw CLI.

Two file styles coexist:

  • Regular workflows — plain *.yml files authored by hand and consumed directly by GitHub Actions.
  • Agentic workflows*.md sources compiled to companion *.lock.yml files via gh aw compile. The .md file is the source of truth; the .lock.yml is generated and must be regenerated whenever the source changes.

Reusable building blocks for agentic workflows live under shared/ and are imported through the imports: frontmatter field.

Working on agentic workflows

Important

Never hand-edit *.lock.yml (or any generated dependency manifest such as package.json, requirements.txt, or go.mod that gh aw compile may emit under .github/workflows/). They are all regenerated by gh aw compile.

# Install the gh-aw CLI extension (once per machine)
gh extension install github/gh-aw

# Compile a single workflow after editing its .md source. Strict mode is the
# default — keep it that way. NEVER set `strict: false` in frontmatter.
gh aw compile <workflow-id>

# When in doubt, force strict-mode validation across all workflows
gh aw compile --strict

# Trigger a workflow on demand
gh aw run <workflow-id>             # interactive
gh aw run <workflow-id> --ref main  # against a specific branch

# Inspect or debug a recent run
gh aw logs <workflow-id>
gh aw audit <run-id>

For deeper guidance — creating, updating, debugging, upgrading, or wrapping MCP servers — see the dispatcher .github/agents/agentic-workflows.agent.md, which routes to the canonical gh-aw prompts.

Secrets & authentication

Agentic workflows authenticate through repository secrets:

Secret / permission Used for Notes
COPILOT_GITHUB_TOKEN GitHub Copilot CLI (model inference) Fine-grained PAT. Preferably replaced by the copilot-requests: write permission — see below.
GH_AW_GITHUB_TOKEN GitHub MCP reads / safe-output writes that need more than the default GITHUB_TOKEN Fine-grained PAT and the token that used to be forced by lockdown mode. Preferably replaced by a GitHub App — see below. Currently unset: the compiler's token chain (GH_AW_GITHUB_MCP_SERVER_TOKEN || GH_AW_GITHUB_TOKEN || GITHUB_TOKEN) falls back to the per-run GITHUB_TOKEN when this secret is absent, so leave it unset unless a workflow needs elevated access.

Important

Fine-grained PATs expire, and the Microsoft Open Source enterprise now hard-rejects any fine-grained PAT whose lifetime exceeds 8 days (the API returns 403 with a message pointing at the token's settings page). A PAT-based setup therefore breaks on a short cycle: when the token lapses — or simply outlives the 8-day window — every workflow that depends on it fails at once (e.g. the Checkout PR branch step 403s on the collaborator-permission and PR lookups) and files a burst of [aw] … failed issues. Prefer the two PAT-free options below — together they let this repo run agentic workflows with no long-lived PAT at all.

Fast unblock: delete the GH_AW_GITHUB_TOKEN secret (run gh secret delete GH_AW_GITHUB_TOKEN --repo microsoft/testfx). Because no source workflow forces a custom PAT anymore (lockdown was removed repo-wide and all declare min-integrity: none), every workflow then degrades gracefully to the built-in GITHUB_TOKEN. The only case that still needs elevated auth is a write-back on a fork PR (where GITHUB_TOKEN is read-only) — use the GitHub App below for those.

Preferred: eliminate the expiring PATs

1. Replace COPILOT_GITHUB_TOKEN with copilot-requests: write. When a workflow's permissions: block grants copilot-requests: write, gh-aw authenticates Copilot inference with the per-run GitHub Actions token and bills through the org's Copilot subscription — no PAT, no secret to rotate. When the permission is present any COPILOT_GITHUB_TOKEN value is ignored for inference.

permissions:
  contents: read
  copilot-requests: write

2. Replace GH_AW_GITHUB_TOKEN with an org-owned GitHub App. A GitHub App mints a short-lived token at the start of each run, scoped to the job's permissions:, and automatically revoked when the run ends (even on failure) — which satisfies the org's short-PAT policy without any manual rotation. A single App can serve every GitHub auth need in these workflows (MCP reads and safe-output writes); only COPILOT_GITHUB_TOKEN cannot use an App (covered by option 1 instead).

Set it up once (requires org admin to create/install the App):

  1. Create a GitHub App owned by the microsoft org (Settings → Developer settings → GitHub Apps). Grant the read/write repository permissions the workflows need (e.g. Contents, Issues, Pull requests), generate a private key (.pem), and install the App on microsoft/testfx.

  2. Store the App ID as a repository variable and the private key as a secret:

    gh variable set APP_ID   --repo microsoft/testfx --body "<app-id>"
    gh secret   set APP_PRIVATE_KEY --repo microsoft/testfx --body "$(cat path/to/private-key.pem)"
  3. Reference the App in the workflow frontmatter (source .md, then recompile):

    tools:
      github:
        toolsets: [repos, issues, pull_requests]
        github-app:
          client-id: ${{ vars.APP_ID }}
          private-key: ${{ secrets.APP_PRIVATE_KEY }}
    # and/or, for write-backs:
    safe-outputs:
      github-app:
        client-id: ${{ vars.APP_ID }}
        private-key: ${{ secrets.APP_PRIVATE_KEY }}

    Add ignore-if-missing: true under github-app: if a workflow must still run on fork PRs (where App secrets are unavailable); it then falls back to GH_AW_GITHUB_TOKEN || GITHUB_TOKEN.

See the upstream reference: https://github.com/github/gh-aw/blob/main/docs/src/content/docs/reference/auth.mdx#using-a-github-app-for-authentication.

Lockdown mode has been removed from this repo's workflows

Historically the four local workflows that read issues/PRs (add-tests.md, weekly-issue-activity.md, shared/address-review-shared.md, and shared/grade-tests-shared.md) set lockdown: true on the GitHub MCP tool. Lockdown mode rejected the default GITHUB_TOKEN and forced a custom PAT (GH_AW_GITHUB_MCP_SERVER_TOKEN || GH_AW_GITHUB_TOKEN || GITHUB_TOKEN), so a single missing/expired PAT failed all of them at activation.

lockdown: is deprecated upstream in favour of integrity filtering (min-integrity). These workflows already declared min-integrity: none (they intentionally examine any issue/PR), so lockdown: true only added the PAT requirement. It has been dropped; the content-filtering behaviour is unchanged, and the default GITHUB_TOKEN now suffices unless a workflow needs elevated access (then use the GitHub App above).

Note

Legacy fallback only: if you must keep a PAT (e.g. before the App is provisioned), store a fine-grained PAT — scoped to microsoft/testfx with read access to Contents, Metadata, Issues (and Pull requests for the PR workflows) — as GH_AW_GITHUB_TOKEN (gh aw secrets set GH_AW_GITHUB_TOKEN --value "…"), and rotate it before every expiry.

Note

Not every [aw] … failed issue is a token problem. The failure banner usually names the cause — AI credits budget exceeded, an engine/inference error, or transient container-image / AWF-binary download failures are all unrelated to authentication. Only the "Lockdown Check Failed … custom GitHub token" banner indicates a PAT issue.

Catalog

Agentic workflows

Code review

Workflow Trigger Description
review-on-open.agent.md PR opened (non-draft) Automatically runs the expert-reviewer agent when a non-draft PR is opened.
review.agent.md /review on a PR Runs the expert-reviewer agent on a pull request when a contributor comments /review.
review-after-autofix.agent.md PR push from Copilot or copilot-autofix label Re-runs the expert code review after new commits are pushed; closes the autofix loop after address-review.agent pushes fixes.
address-review.agent.md PR review with changes_requested (Copilot PRs) Automatically addresses code review feedback on Copilot-created PRs. Includes a circuit breaker (max 3 iterations).
autofix.agent.md /autofix on a PR Same behavior as address-review.agent but manually triggered.
pr-fix.md /pr-fix on a PR Diagnoses failing CI checks, applies fixes, runs tests/formatters, and pushes corrections to the PR branch.
msbuild-quality-review.md Weekly schedule + manual Reviews .props, .targets, Directory.Build.*, Directory.Packages.props, and NuGet build*/ extensions for authoring anti-patterns. Delegates to the msbuild-reviewer agent.

Build & test diagnostics

Workflow Trigger Description
build-failure-analysis.md Azure Pipelines microsoft.testfx check completed (failure) on a PR to main or rel/* Downloads the binary logs the failed Azure DevOps build already produced (all build legs — it does not rebuild), and the build-failure-analyst agent queries them via binlog-mcp, posts a summary comment, and attaches inline suggestion blocks. Advisory only — not a gating check.
build-failure-analysis-command.md /analyze-build-failure on a PR Re-runs the analysis on demand: inspects the PR's latest microsoft.testfx build and, only when it failed, downloads its binlogs and analyzes them (no rebuild).
add-tests.md /add-tests on a PR Generates unit tests for code introduced in a pull request.
grade-tests-on-pr.agent.md PR opened/reopened/synchronize/ready_for_review touching test/** Automatically grades new and modified test methods and posts a single PR scorecard comment via the grade-tests skill.
grade-tests.agent.md /grade-tests on a PR Re-runs the test-quality grading on demand.

Continuous quality improvers (scheduled)

Workflow Trigger Description
adhoc-qa.md Daily + manual Performs ad hoc, subjective quality assurance — verifies code builds, tests pass, docs are clear, structure is healthy. Opens discussions and may submit draft PRs.
code-simplifier.md Daily + manual + /code-simplify Analyzes recently modified code and opens PRs that simplify it while preserving behavior.
efficiency-improver.md Daily + manual + /efficiency-assist Green-software-focused assistant that identifies and implements energy/compute efficiency improvements.
perf-improver.md Daily + manual + /perf-assist Performance-focused assistant that identifies bottlenecks and lands measured improvements.
test-improver.md Daily + manual + /test-assist Testing-focused assistant that improves test quality and coverage.
repository-quality-improver.md Weekday schedule + manual Daily analysis of repository quality, rotating focus areas. Opens tracking issues like this one.
daily-file-diet.md Daily + manual Identifies oversized source files and opens actionable refactoring issues.
unskip-closed-tests.md Weekly + manual Finds tests skipped via [Ignore("…#issue")] whose tracking issue is now closed, verifies they pass, and opens a PR re-enabling them.
duplicate-code-detector.md Schedule + manual Identifies duplicate code patterns and suggests refactoring opportunities.
malicious-code-scan.md Schedule + manual Reviews code changes from the last 3 days for suspicious patterns indicating malicious or agentic threats.
markdown-linter.md Schedule + manual + issues Runs Markdown quality checks using Super Linter and opens issues for violations.
link-checker.md Daily Daily automated link checker that finds and fixes broken links in documentation files.
glossary-maintainer.md Schedule + manual Maintains and updates the documentation glossary based on codebase changes.

Issue & PR housekeeping

Workflow Trigger Description
sub-issue-closer.md Schedule + manual + issues Recursively closes parent issues when all sub-issues are 100% complete.
dependabot-issue-bundler.md Issues Finds all open Dependabot PRs and creates bundle issues for each runtime + manifest file.
dependabot-pr-bundler.md Daily + manual Bundles compatible Dependabot updates into single PRs, runs tests, and opens draft PRs.
weekly-issue-activity.md Weekly + manual + issues Weekly summary of issue activity including trends, charts, and insights.

Regular workflows

Workflow Trigger Description
agentic_commands.yml PR, issue comment, issues Dispatches /-prefixed slash commands typed in comments to the right agentic workflow.
agentics-maintenance.yml Schedule + manual + reusable + issues Maintains the agentic workflow ecosystem itself (re-compilation, dependency bumps, etc.).
backport.yml /backport comment + issues + schedule Backports merged PRs to release branches on demand.
backport-base.yml Reusable Shared logic invoked by backport.yml to perform the actual backport.
check-vendored-files.yml Schedule + manual + PR + issues Verifies that files vendored from external sources (such as eng/common) stay in sync.
copilot-setup-steps.yml PR + push + manual Bootstraps a Copilot Coding Agent environment with the right .NET SDK and tooling.
dedup-analysis.yml Schedule + manual + issues Code Duplication Analysis (jscpd-based).
enable-auto-merge.yml pull_request_target Enables auto-merge on eligible PRs.
fv-docs-validation.yml PR + push + manual Validates documentation referenced by the friend-validation (FV) program.
markdownlint.yml PR Runs markdownlint on changed Markdown files.

Shared components

Reusable agentic-workflow snippets imported via imports: in workflow frontmatter:

Component Used by
shared/address-review-shared.md address-review.agent.md, autofix.agent.md
shared/build-failure-analysis-shared.md build-failure-analysis.md, build-failure-analysis-command.md
shared/formatting.md Quality improver workflows (output formatting conventions)
shared/msbuild-review-shared.md msbuild-quality-review.md
shared/grade-tests-shared.md grade-tests-on-pr.agent.md, grade-tests.agent.md
shared/repo-build-setup.md Workflows that need to restore + build the repo before the agent runs
shared/reporting.md Quality improver workflows (issue/PR body templates)
shared/review-shared.md review.agent.md, review-on-open.agent.md, review-after-autofix.agent.md

Conventions

  • Strict mode is mandatory. Workflow frontmatter must not set strict: false. When in doubt, run gh aw compile --strict.
  • Source of truth. Edit the .md file (and any imported shared/*.md); never the .lock.yml.
  • One change, one compile. After editing an agentic workflow source, run gh aw compile <workflow-id> and commit the regenerated .lock.yml in the same change.
  • Same applies to Dependabot updates that touch generated manifests (e.g. package.json / requirements.txt / go.mod) if gh aw compile ever emits them under .github/workflows/: never merge those PRs directly; update the source .md files and rerun gh aw compile --dependabot to bundle the fixes.
  • Pinned actions only. Strict mode pins every uses: reference to a SHA; the compiler enforces this.
  • Minimal permissions. Workflows declare the least privilege they need; write capabilities flow through gh-aw safe-outputs: rather than direct permissions: write-all.