Restructure Windows CLI integration into scenario matrix with missing shell/env/path/chaos coverage by Copilot · Pull Request #38526 · github/gh-aw

Copilot · 2026-06-11T06:35:36Z

windows-cli-integration.yml had strong baseline checks but missed several required dimensions (systematic scenario modeling, no-profile shells, PATH/PATHEXT quirks, path/Unicode edge cases, env toggles, and negative behavior). This change consolidates coverage into a matrix-driven harness and fills the missing cases with explicit timeout-guarded assertions.

Matrix-driven scenario model
- Replaced ad-hoc command blocks with a single pwsh runner that executes a structured scenario matrix across:
  - shell (pwsh, powershell, cmd)
  - launch mode (direct binary vs PATH lookup)
  - environment shape (NO_COLOR, TERM=dumb, TERM unset, CI=true, CI unset)
  - PATH style (workspace-first, toolcache-first, duplicate entries)
Shell + launch-mode coverage
- Added explicit default-profile and -NoProfile variants for both pwsh and powershell.
- Added cmd invocation coverage through the same timeout-controlled process harness.
PATH/PATHEXT + command resolution
- Added PATH-order/shadowing checks and validated resolved binary path (where gh-aw) behavior.
- Added PATHEXT assertion and extensionless gh-aw invocations from pwsh/cmd.
Filesystem/path edge cases
- Added execution from paths with spaces, mixed slash style, and parentheses.
- Added execution in a non-ASCII working directory.
Command surface + chaos case
- Added missing command variants: help, run --help (in addition to existing help/version coverage).
- Added explicit unknown-subcommand test asserting:
  - non-zero exit
  - useful error text
  - fast completion under bounded timeout
Hang protection consistency
- All command executions now use explicit per-command WaitForExit timeouts (including cmd-path scenarios), eliminating reliance on job-level timeout for hang detection.

- name: Windows CLI scenario matrix (shell × launch-mode × env-shape × path-style)
  shell: pwsh
  run: |
    # matrix loop executes combinations and validates exit/output/timeout
    Invoke-CliProcess -FilePath "pwsh" -ArgumentList @("-NoProfile", "-Command", "& gh-aw --help")
    Invoke-CliProcess -FilePath "cmd" -ArgumentList @("/d", "/s", "/c", "gh-aw --help")
    Invoke-CliProcess -FilePath $env:BINARY -ArgumentList @("totally-unknown-subcommand-xyz") `
      -TimeoutMs 10000 -ExpectFailure -ExpectedErrorPattern "unknown|not found|invalid|unrecognized"

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Expands the Windows CLI integration workflow to run a broader scenario matrix across shells, PATH/launch modes, and environment-variable shapes to catch Windows-specific regressions (timeouts, PATH resolution, ANSI output, unicode paths, etc.).

Changes:

Replaces individual Start-Process checks with a reusable Invoke-CliProcess helper (timeouts, env overrides, output capture).
Adds a scenario matrix spanning pwsh, powershell, and cmd with different PATH/env configurations.
Adds additional checks for PATHEXT resolution, tricky path shapes (spaces/parentheses/mixed slashes), unicode working directory, minimal environment, and unknown-subcommand failure behavior.

Show a summary per file

File	Description
.github/workflows/windows-cli-integration.yml	Introduces a PowerShell-based scenario matrix and process runner helper to broaden Windows CLI integration coverage.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 1/1 changed files
Comments generated: 5

+            $proc = [System.Diagnostics.Process]::Start($psi)
+            if (-not $proc.WaitForExit($TimeoutMs)) {
+              try { $proc.Kill($true) } catch {}
+              throw "TIMEOUT: '$FilePath $($ArgumentList -join ' ')' exceeded ${TimeoutMs}ms"
+            }
+
+            $stdout = $proc.StandardOutput.ReadToEnd()
+            $stderr = $proc.StandardError.ReadToEnd()
+            $output = "$stdout`n$stderr"


+          $toolcacheDir = Join-Path $env:RUNNER_TEMP "toolcache-like"
+          $spaceDir = Join-Path $env:RUNNER_TEMP "path with spaces"
+          $parenDir = Join-Path $env:RUNNER_TEMP "Program Files (x86)"
+          $unicodeDir = Join-Path $env:TEMP "tëst-dïr-αβγ"
+          foreach ($dir in @($toolcacheDir, $spaceDir, $parenDir, $unicodeDir)) {
+            New-Item -ItemType Directory -Force -Path $dir | Out-Null
          }
-          if ($proc.ExitCode -ne 0) { throw "Exit code $($proc.ExitCode)" }
-          Add-Content $env:GITHUB_STEP_SUMMARY "✅ [pwsh] version OK"
+          Copy-Item -Path $binary -Destination (Join-Path $spaceDir "gh-aw.exe") -Force
+          Copy-Item -Path $binary -Destination (Join-Path $parenDir "gh-aw.exe") -Force


+            @{ name = "powershell-default/direct/default/default"; shell = "powershell-default"; launch = "direct"; env = "default"; path = "default" },
+            @{ name = "powershell-noprofile/direct/default/default"; shell = "powershell-noprofile"; launch = "direct"; env = "default"; path = "default" },
+            @{ name = "pwsh-default/path/default/workspace-first"; shell = "pwsh-default"; launch = "path"; env = "default"; path = "workspace-first" },
+            @{ name = "cmd/path/default/toolcache-first"; shell = "cmd"; launch = "path"; env = "default"; path = "toolcache-first" },


+            if ($scenario.launch -eq "path") {
+              $whereResult = Invoke-CliProcess -FilePath "cmd" -ArgumentList @("/d", "/s", "/c", "where gh-aw") -EnvironmentOverrides $envOverrides -UnsetEnvironment $unsetVars
+              $firstResolved = ($whereResult.Output -split "`r?`n" | Where-Object { $_ -match "gh-aw\.exe$" } | Select-Object -First 1)
+              if (-not $firstResolved -or -not $firstResolved.StartsWith($workspace, [System.StringComparison]::OrdinalIgnoreCase)) {
+                throw "PATH resolution mismatch for '$($scenario.name)': $firstResolved"
+              }
+            }


+          foreach ($scenario in $scenarioMatrix) {
+            $env:PATH = $originalPath
+            switch ($scenario.path) {
+              "workspace-first" { $env:PATH = "$workspace;$originalPath" }
+              "toolcache-first" { $env:PATH = "$toolcacheDir;$workspace;$originalPath" }
+              "duplicate-workspace" { $env:PATH = "$workspace;$workspace;$originalPath" }
+              default {}
+            }


Initial plan

4e582c3

Copilot AI assigned Copilot and pelikhan Jun 11, 2026

Copilot started work on behalf of pelikhan June 11, 2026 06:35 View session

Copilot AI linked an issue Jun 11, 2026 that may be closed by this pull request

[windows-integration] Improve windows-cli-integration.yml: matrix coverage, chaos scenarios, and missing test dimensions #38522

Closed

6 tasks

test: expand windows CLI integration coverage matrix

d669085

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Improve windows-cli-integration.yml for matrix coverage and test dimensions~~ Restructure Windows CLI integration into scenario matrix with missing shell/env/path/chaos coverage Jun 11, 2026

Copilot finished work on behalf of pelikhan June 11, 2026 06:54

Copilot AI requested a review from pelikhan June 11, 2026 06:54

pelikhan approved these changes Jun 11, 2026

View reviewed changes

pelikhan marked this pull request as ready for review June 11, 2026 07:21

Copilot AI review requested due to automatic review settings June 11, 2026 07:21

pelikhan merged commit 99c0150 into main Jun 11, 2026

pelikhan deleted the copilot/windows-integration-improvement branch June 11, 2026 07:21

Copilot AI reviewed Jun 11, 2026

View reviewed changes

This was referenced Jun 11, 2026

Smoke Test: Claude - 27331325626 #38545

Closed

Smoke Test: Copilot - 27331373301 #38549

Closed

[docs] Update documentation for features from 2026-06-11 #38593

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Restructure Windows CLI integration into scenario matrix with missing shell/env/path/chaos coverage#38526

Restructure Windows CLI integration into scenario matrix with missing shell/env/path/chaos coverage#38526
pelikhan merged 2 commits into
mainfrom
copilot/windows-integration-improvement

Copilot AI commented Jun 11, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Copilot AI commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Jun 11, 2026 •

edited

Loading