Skip to content

Gate external PRs on an assigned, linked issue - #3291

Draft
maxisbey wants to merge 1 commit into
mainfrom
pr-intake-gate
Draft

Gate external PRs on an assigned, linked issue#3291
maxisbey wants to merge 1 commit into
mainfrom
pr-intake-gate

Conversation

@maxisbey

Copy link
Copy Markdown
Contributor

Adds an intake gate for pull requests from outside the maintainer team: a PR stays open only if it links an open issue its author is assigned to (or one labeled help wanted). Everything else is closed by a bot with an explanation and reopens automatically once a maintainer assigns the author. CONTRIBUTING.md is rewritten around that policy.

Motivation and Context

Over the last six months this repo received ~640 pull requests from outside the maintainer team — 2.7× the previous six months — of which 24 were merged. 41% of newly opened issues now attract an external PR within 48 hours (median under 11 hours), the open-PR backlog is ~80% external, and CONTRIBUTING.md's existing "issue first, no drive-by agents" rules have no mechanical backing. Reviewing a PR properly costs the same as it always did; producing one no longer does. With the maintainer time we actually have, issues are the contribution we can use, and this makes the repo say so and behave accordingly.

The workflow is adapted from PrefectHQ/fastmcp's require-issue-link.yml (which came from langchain's); pydantic and pydantic-ai run similar gates. Within this org, inspector has already gone issues-only and typescript-sdk restricted PR creation for a month in June for the same reason.

Behaviour

  • External, non-draft PR must Fixes / Closes / Resolves an open issue in this repo where the author is an assignee, or the issue carries help wanted. Otherwise: missing-issue-link label, one comment, closed.
  • Exempt: anyone with triage or better on the repo (resolved from the collaborator-permission capability flags, so private org membership and custom roles don't matter), bot accounts, drafts (re-checked on ready-for-review).
  • Ways back in: a maintainer assigns the author on the linked issue → the PR reopens itself and its red check re-runs; the author fixes the description → reopens on the edited event; anyone triage+ reopens the PR or removes the label → sticky bypass-issue-check.
  • If GitHub refuses to reopen (branch force-pushed or deleted while closed, or a sibling PR from the same branch is already open) the gate works out which, keeps the control label so the PR stays findable, and replaces its comment with specific instructions rather than telling people to open another PR.
  • PRs numbered below 3200 predate the gate and are left alone unless a maintainer evaluates one via workflow_dispatch; from 3200 up a PR is evaluated on its next event. Once labeled, a PR is managed normally regardless of number.
  • Ships in dry-run. It logs PASS/FAIL and [dry-run] would … lines and mutates nothing until the repository variable PR_GATE_ENFORCE is set to true.

Differences from the fastmcp version (also listed in the file header): one job and one script for all three entry points (PR events, issue assignment, manual dispatch) so admission and reopening can't drift; PR state is always read live rather than from the event payload, which closes a race where a queued run could strip the label without reopening and orphan the PR; linked issues must be open and still in this repo; gated PRs are found with the list API rather than Search; reopen-before-unlabel with the refused-reopen diagnosis above; trust from capability flags rather than role-name strings; the waiver label is our existing help wanted.

Docs

  • CONTRIBUTING.md: new "Why issues, not pull requests", "How pull requests get in", "Who we actively want to hear from" sections; the assignment rules (a bare "please assign me" is noise; engaging with the issue is the conversation we assign on; reporters have first claim); label table corrected (ready for work means queued for a maintainer, not an invitation).
  • AGENTS.md: an agent-facing statement of the policy at the top, since that's the file coding agents load.
  • .github/pull_request_template.md: a short repo-level template that leads with the Fixes # line the gate looks for, replacing the inherited org template here.

How Has This Been Tested?

  • A mock-Octokit harness driving the script through 16 scenarios (no link; link to help wanted; maintainer reopening their own vs a gated PR; triage-role and bot label removal; Dependabot; hand-closed PRs; closed/transferred/cross-repo/PR-number references; both refused-reopen diagnoses; issue assignment; dispatch backfill; drafts; pass→fail comment un-minimizing; a planted marker comment), in enforce and dry-run modes.
  • actionlint and zizmor --pedantic clean (the pull_request_target trigger carries an inline justification; the workflow never checks out or executes PR code and interpolates nothing from the PR into the script).
  • The endpoint shapes it depends on (collaborators/{user}/permission capability flags for admins, outsiders and app logins; issues.listForRepo with creator+labels+state=closed; minimizeComment/unminimizeComment) were checked against the live API, and the same permission set is what fastmcp's copy has been running with since May.
  • Real traffic gets exercised in dry-run after merge before enforcement is switched on.

Breaking Changes

None for SDK users. For contributors: PRs opened without an assigned, linked issue will be closed automatically once enforcement is on; CONTRIBUTING.md describes the path.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Repository automation

Checklist

  • I have read the MCP Documentation
  • My code follows the repository's style guidelines
  • New and existing tests pass locally
  • I have added appropriate error handling
  • I have added or updated documentation as needed

Additional context

Rollout after merge:

  1. Leave it in dry-run for a few days and read the Actions → Require Linked Issue logs against real traffic.
  2. Label hygiene: drop help wanted from the closed issues that still carry it, and update the help wanted / ready for work label descriptions to match CONTRIBUTING.md.
  3. Set PR_GATE_ENFORCE=true (Settings → Secrets and variables → Actions → Variables).
  4. Backfill any already-open PRs we want evaluated: gh workflow run require-linked-issue.yml -f pr_number=<n>.
  5. Stand up a triage-permission collaborators team as the trusted-contributor group the docs refer to; until then a maintainer reopening a PR is the per-PR equivalent.

AI Disclaimer

Unsolicited pull requests now outnumber issues four to one and almost none
are reviewable in the time we have. This adds a workflow that closes an
external PR unless its description links an open issue the author is
assigned to (or one labeled "help wanted"), and reopens it automatically
once a maintainer assigns them. Maintainers, triage-role collaborators,
bots and drafts are exempt; reopening a PR or removing the control label
is a sticky maintainer override. PRs numbered below 3200 predate the gate
and are only evaluated on manual dispatch.

The workflow is adapted from PrefectHQ/fastmcp's require-issue-link.yml
(itself from langchain), restructured into a single script that always
reads PR state live, requires linked issues to be open and in this repo,
finds gated PRs via the list API rather than search, and diagnoses a
refused reopen instead of guessing. It ships in dry-run: set the
PR_GATE_ENFORCE repository variable to "true" to enforce.

CONTRIBUTING.md is rewritten around the policy (issues are the
contribution; how PRs get in; who we want to hear from), AGENTS.md gains
an agent-facing statement of it, and a short repo-level PR template
leads with the Fixes line the gate looks for.

No-Verification-Needed: workflow and docs only; exercised with a mock harness, actionlint and zizmor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant