Skip to content

[rb] streamline tests on github actions runners#17550

Merged
titusfortner merged 3 commits into
trunkfrom
ci-ruby
Jun 5, 2026
Merged

[rb] streamline tests on github actions runners#17550
titusfortner merged 3 commits into
trunkfrom
ci-ruby

Conversation

@titusfortner
Copy link
Copy Markdown
Member

@titusfortner titusfortner commented May 22, 2026

🔗 Related Issues

Ruby implementation of #17539
Supersedes #16813

💥 What does this PR do?

Splits execution into two pieces: manual/scheduled and PR/Push

  • PR/Push

    • Runs "Smoke tests": se-manager and os-sensitive and unit tests on stable browser versions
    • Runs only affected targets within smoke tests, so if nothing substantive is modified, no tests run
    • With [rb] Run unit tests as a single Bazel target instead of per-file #17616 it is much faster to run the unit tests with the browser tests than separate
    • Tests marked to run "pending" they will be "skipped"
  • Scheduled/Manual

    • Runs "Full tests": browser tests for each browser on beta browser versions if applicable
    • Includes bidi tests but not remote/grid tests
    • Alternate interpreter/version unit tests
    • Tests marked "pending" will be executed expecting to fail, we can update guards based on manual runs without failing user tests
  • Removes processing commit messages and PR titles for [rb] to decide what to run for a PR

  • Removes lint as a test so it doesn't run rbe and on github - runs via rubocop run target instead of test

Breakdown by target and integration test

Numbers are approximate based on averages

Scope Trigger Integration targets Integration examples Wall clock Runner time
ci-rbe (unchanged) PR/Push 231 4,035
ci-ruby (current) PR/Push/Scheduled 66 1,161 22m 114m
smoke (post-PR) PR/Push < 16 < 196 18m 42m
full (post-PR) Scheduled 141 2,429 53m 166m

🔧 Implementation Notes

  • Everything is run without pinned drivers (keeps the github cache < 10GB) but means downloading browsers/drivers with selenium manager during execution
  • windows/linux browser is set to "yes" because it just needs not to be empty
  • mac browser is set to "safari" even though other browser targets will be run because it needs safaridriver setup in bazel.yml
  • skip-rbe is included as a tag in smoke tests for consistency even though nothing in Ruby is tagged with that now
  • /bazel-test-if-targets.sh wrapper is needed in smoke tests case none of the affected targets passed in match the other tags to prevent bazel from erroring

🤖 AI assistance

  • No substantial AI assistance used
  • AI assisted (complete below)
    • Tool(s):
    • What was generated:
    • I reviewed all AI output and can explain the change

@selenium-ci selenium-ci added C-rb Ruby Bindings B-build Includes scripting, bazel and CI integrations labels May 22, 2026
@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented May 22, 2026

Review Summary by Qodo

(Agentic_describe updated until commit e883e27)

Streamline Ruby CI workflow with smoke and full test job separation

✨ Enhancement

Grey Divider

Walkthroughs

Description
• Restructure Ruby CI workflow to split tests into smoke and full test jobs
• Replace commit message/PR title parsing with explicit target passing
• Add wrapper script to handle bazel test exit code 4 gracefully
• Update browser tag filters to use stable versions for smoke tests
• Remove lint as a separate test job and convert to binary
• Adjust unit test matrix to cover truffleruby and latest MRI only
Diagram
flowchart LR
  A["CI Trigger"] --> B["Read Targets"]
  B --> C["Lint"]
  B --> D["Unit Tests"]
  B --> E["Smoke Tests"]
  B --> F["OS Tests Full"]
  E --> G["Test Results"]
  D --> G
  F --> G
  C --> G

Loading

Grey Divider

File Changes

1. .github/workflows/ci-ruby.yml ⚙️ Configuration changes +63/-36

Restructure workflow with smoke and full test jobs

• Split integration tests into smoke and os-tests-full jobs with separate conditions
• Add workflow inputs for targets and smoke flag to control test execution
• Remove build job and lint as separate test job
• Update unit test matrix to only run truffleruby and MRI 4.0.4 on schedule/manual
• Add bazel-test-if-targets.sh wrapper for smoke tests with tag filters
• Configure browser versions: stable for smoke tests, beta for full tests

.github/workflows/ci-ruby.yml


2. .github/workflows/ci.yml ⚙️ Configuration changes +13/-3

Extract and pass Ruby targets to workflow

• Rename rb output to rb_targets and add process_binding function
• Extract language-specific targets from bazel-targets.txt instead of boolean check
• Pass targets parameter to ci-ruby.yml workflow
• Remove commit message and PR title parsing for Ruby binding decision

.github/workflows/ci.yml


3. scripts/github-actions/bazel-test-if-targets.sh ✨ Enhancement +20/-0

Add bazel test wrapper for graceful no-targets handling

• New wrapper script for bazel test command
• Converts exit code 4 (no matching targets) to success with notice message
• Preserves other exit codes for proper error reporting

scripts/github-actions/bazel-test-if-targets.sh


View more (3)
4. rb/spec/tests.bzl ⚙️ Configuration changes +2/-2

Update browser tag filters to stable versions

• Update os-sensitive tag filter from beta browsers to stable browsers
• Update se-manager tag filter from beta browsers to stable browsers
• Changes affect smoke test execution to use stable Chrome, Firefox, Edge, Safari

rb/spec/tests.bzl


5. rb/spec/unit/selenium/webdriver/guard_spec.rb 🧪 Tests +1/-1

Add conditional skip for pending tests

• Modify pending disposition check to skip when SKIP_PENDING environment variable is set
• Allows conditional skipping of pending tests during CI execution

rb/spec/unit/selenium/webdriver/guard_spec.rb


6. rb/BUILD.bazel ⚙️ Configuration changes +0/-13

Remove lint test target and convert to binary

• Remove rb_test import from rules_ruby
• Remove lint rb_test target definition
• Convert lint to binary target for manual execution

rb/BUILD.bazel


Grey Divider

Qodo Logo

@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented May 22, 2026

Code Review by Qodo

🐞 Bugs (6) 📘 Rule violations (3) 📎 Requirement gaps (0) 🎨 UX issues (0)

Grey Divider


Action required

1. Safari setup skipped 🐞 Bug ☼ Reliability
Description
os-tests-full passes browser: safari-local into bazel.yml, but bazel.yml only runs the
Safari setup step when inputs.browser == 'safari'. This prevents safaridriver --enable from
running for the full-test macOS Safari job, causing Safari tests to run without required setup and
likely fail.
Code

.github/workflows/ci-ruby.yml[R99-104]

+          - os: macos
+            browser: safari-local
+    with:
+      name: ${{ matrix.os }}-tests-full (${{ matrix.browser }})
      browser: ${{ matrix.browser }}
      os: ${{ matrix.os }}
Evidence
The macOS full-test matrix uses safari-local and forwards it as the browser input, but the
Safari setup step is guarded by an exact equality check for safari, so it will not run for
safari-local.

.github/workflows/ci-ruby.yml[85-105]
.github/workflows/bazel.yml[191-193]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The Ruby `os-tests-full` job uses `browser: safari-local` (to tag-filter Safari local tests), but the reusable workflow `bazel.yml` only enables SafariDriver when `inputs.browser == 'safari'`. As a result, macOS full tests can execute Safari targets without SafariDriver being enabled.

## Issue Context
- `ci-ruby.yml` forwards `matrix.browser` as the `browser` input to `bazel.yml`.
- `bazel.yml` has a dedicated Safari enablement step guarded by an exact string match.

## Fix Focus Areas
- .github/workflows/bazel.yml[191-193]
- .github/workflows/ci-ruby.yml[85-105]

## Suggested fix
Update the Safari setup condition to cover Safari variants used as `browser` inputs (e.g., `safari-local`, and optionally `safari-preview` if used elsewhere). For example:
- Change `if: inputs.browser == 'safari'` to `if: startsWith(inputs.browser, 'safari')`

Alternative (less preferable): keep `matrix.browser: safari-local` for tag filtering, but pass a separate `browser: safari` (or add a dedicated input) purely to trigger setup steps.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Schedule-gated jobs never run 📘 Rule violation ☼ Reliability
Description
ci-ruby.yml is only invoked via workflow_call from ci.yml, but it branches on
github.event_name == 'schedule'; in a reusable workflow the event name is workflow_call, so the
scheduled/manual “full” jobs never execute. This can cause scheduled CI to run the smoke path
instead of the intended full Ruby coverage, reducing CI robustness.
Code

.github/workflows/ci-ruby.yml[32]

+    if: github.event_name == 'schedule' || (github.event_name == 'workflow_dispatch' && !inputs.smoke)
Evidence
The compliance requirement for CI/workflow robustness is violated because the workflow’s
precondition checks use the wrong event context: ci.yml is scheduled, but the called ci-ruby.yml
will not see github.event_name == 'schedule', preventing the intended full-test jobs from running.

.github/workflows/ci-ruby.yml[3-16]
.github/workflows/ci-ruby.yml[28-33]
.github/workflows/ci-ruby.yml[85-88]
.github/workflows/ci.yml[3-12]
.github/workflows/ci.yml[127-134]
Best Practice: Learned patterns

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`.github/workflows/ci-ruby.yml` is a reusable workflow (`on: workflow_call`) but uses `if: github.event_name == 'schedule'` to decide whether to run `unit-tests` and `os-tests-full`. When invoked from `.github/workflows/ci.yml` (which *is* scheduled), the called workflow still sees `github.event_name == 'workflow_call'`, so the schedule-gated jobs never run.

## Issue Context
Scheduled runs are defined in `.github/workflows/ci.yml`, which calls the Ruby workflow via `uses: ./.github/workflows/ci-ruby.yml`. The Ruby workflow should distinguish smoke vs full based on the *caller* context, not its own `github.event_name`.

## Fix Focus Areas
- .github/workflows/ci-ruby.yml[3-16]
- .github/workflows/ci-ruby.yml[28-33]
- .github/workflows/ci-ruby.yml[53-56]
- .github/workflows/ci-ruby.yml[85-88]
- .github/workflows/ci.yml[3-12]
- .github/workflows/ci.yml[127-134]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. Invalid matrix.tag-filters access 📘 Rule violation ☼ Reliability
Description
The workflow references matrix.tag-filters, but matrix keys containing - cannot be accessed with
dot-notation, causing expression evaluation failure or empty --test_tag_filters and potentially
running the wrong test set. This makes CI behavior non-deterministic and undermines safe CI gating.
Code

.github/workflows/ci-ruby.yml[76]

+        --test_tag_filters=${{ matrix.tag-filters }}
Evidence
PR Compliance ID 14 requires hardening CI/scripts to be robust and deterministic. The workflow uses
a hyphenated matrix key (tag-filters) but references it via matrix.tag-filters, which is not a
safe/deterministic GitHub Actions expression access pattern for such keys, risking misconfigured
Bazel arguments.

.github/workflows/ci-ruby.yml[53-77]
Best Practice: Learned patterns

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`.github/workflows/ci-ruby.yml` defines a matrix key named `tag-filters`, but later references it as `${{ matrix.tag-filters }}`. In GitHub Actions expressions, keys with hyphens must be accessed using bracket notation (e.g., `matrix['tag-filters']`) or renamed to an underscore key.

## Issue Context
If this expression fails (or resolves unexpectedly), the `--test_tag_filters=...` argument may be empty/incorrect, leading to incorrect test selection and unreliable CI outcomes.

## Fix Focus Areas
- .github/workflows/ci-ruby.yml[53-77]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


View more (7)
4. Unvalidated targets shell injection ✓ Resolved 📘 Rule violation ⛨ Security
Description
workflow_dispatch introduces a user-provided targets input that is later interpolated into a
shell command without validation/quoting, enabling command injection and unsafe argument splitting.
This violates the CI/script hardening requirement and can lead to unexpected command execution or
incorrect Bazel invocation.
Code

.github/workflows/ci-ruby.yml[R10-15]

  workflow_dispatch:
+    inputs:
+      targets:
+        description: Specify Bazel Targets
+        type: string
+        default: '//rb/...'
Evidence
PR Compliance ID 14 requires safe argument handling in CI scripts. The PR adds a manually-supplied
targets input and later expands ${{ inputs.targets }} directly into a shell command, which is
unsafe because the expanded text can be interpreted by the shell as additional commands/flags.

.github/workflows/ci-ruby.yml[10-15]
.github/workflows/ci-ruby.yml[73-80]
Best Practice: Learned patterns

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A `workflow_dispatch` string input (`inputs.targets`) is user-controlled and later inserted directly into a `bazel test` shell command, allowing shell metacharacters (e.g., `;`, `&&`) to be interpreted as commands.

## Issue Context
This workflow now accepts `targets` via manual dispatch. The value is used in the `os-tests` job’s `run` script, so it must be handled as data (arguments) rather than executable shell text.

## Fix Focus Areas
- .github/workflows/ci-ruby.yml[10-15]
- .github/workflows/ci-ruby.yml[73-80]

## Suggested approach
- Read `inputs.targets` into a variable inside quotes.
- Split into a bash array safely (e.g., `read -r -a targets_arr <<< "$targets"`).
- Invoke Bazel with `"${targets_arr[@]}"` so each target is a separate argument and shell metacharacters are not executed.
- Optionally validate that each entry matches expected Bazel target patterns (e.g., starts with `//`).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


5. IE remote tests included ✓ Resolved 🐞 Bug ≡ Correctness
Description
The Windows os-tests job uses --test_tag_filters=edge,edge-remote,unit,skip-rbe,-ie, but IE
remote targets are tagged ie-remote (not ie) and also include skip-rbe, so they can still be
selected and executed on Windows despite the -ie exclusion.
Code

.github/workflows/ci-ruby.yml[59]

+            tag-filters: edge,edge-remote,unit,skip-rbe,-ie
Evidence
The workflow’s Windows tag filter includes skip-rbe and only excludes ie. In the Ruby test rule
generator, IE is defined with the skip-rbe tag, and remote tests are tagged as {browser}-remote
while inheriting the browser tags. Also, some integration tests do not override the default
browsers=BROWSERS.keys(), so IE/IE-remote targets exist and are eligible for selection.

.github/workflows/ci-ruby.yml[54-62]
rb/spec/tests.bzl[130-141]
rb/spec/tests.bzl[170-171]
rb/spec/tests.bzl[195-215]
rb/spec/integration/selenium/webdriver/BUILD.bazel[30-37]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The Windows `os-tests` matrix entry tries to exclude IE by adding `-ie`, but IE remote targets use the tag `ie-remote` and inherit `skip-rbe`, so they still match the positive filters.

### Issue Context
`rb_integration_test` generates both local and `*-remote` test targets, and remote targets inherit the browser-specific tag list (including `skip-rbe` for IE).

### Fix Focus Areas
- .github/workflows/ci-ruby.yml[57-60]

### Suggested fix
Update the Windows `tag-filters` to also exclude the remote IE tag, e.g.:
- `edge,edge-remote,unit,skip-rbe,-ie,-ie-remote`

(Alternatively, remove `skip-rbe` from the Windows include list if it is not intended to pull in unrelated `skip-rbe` tests.)

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


6. grep exits read-targets ✓ Resolved 🐞 Bug ☼ Reliability
Description
In ci.yml, process_binding assigns lang_targets via a pipeline containing grep; when there are no
matching Ruby targets, grep exits non-zero and can fail the entire “Read targets” step instead of
simply producing no rb_targets output.
Code

.github/workflows/ci.yml[R89-94]

+          process_binding() {
+            local pattern=$1 tag=$2
+            local lang_targets
+            lang_targets=$(echo "$targets" | tr ' ' '\n' | grep -E "^${pattern}[:/]" | tr '\n' ' ' | sed 's/ *$//')
+            [ -n "$lang_targets" ] && echo "${tag}_targets=$lang_targets" >> "$GITHUB_OUTPUT"
+          }
Evidence
The process_binding() function uses a grep -E pipeline to compute lang_targets (line 92). If
no targets match ^//rb[:/], grep returns non-zero; without || true this can fail the step
rather than just leaving rb_targets unset.

.github/workflows/ci.yml[81-100]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`process_binding()` uses `grep` inside a command substitution without guarding for the “no matches” exit code. When no Ruby targets exist, this can terminate the step early and fail CI.

### Issue Context
The workflow should continue and simply omit `rb_targets` when there are no matching targets.

### Fix Focus Areas
- .github/workflows/ci.yml[81-100]

### Suggested fix
- Make the pipeline tolerant of no matches, e.g. append `|| true` to the *whole* pipeline, or use `grep ... || :`.
 - Example:
   - `lang_targets=$(echo "$targets" | tr ' ' '\n' | grep -E "^${pattern}[:/]" | tr '\n' ' ' | sed 's/ *$//' || true)`
- Consider switching `echo` to `printf '%s\n' "$targets"` to avoid echo edge cases.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


7. Dispatch missing targets input ✓ Resolved 🐞 Bug ≡ Correctness
Description
ci-ruby.yml uses ${{ inputs.targets }} in the os-tests Bazel command, but targets is only
defined under workflow_call; a manual workflow_dispatch run can expand to an empty target list
and cause bazel test to fail.
Code

.github/workflows/ci-ruby.yml[R74-76]

+        --test_tag_filters=${{ matrix.tag-filters }}
+        ${{ inputs.targets }}
+        || { code=$?; [ "$code" -eq 4 ] && exit 0; exit "$code"; }
Evidence
The workflow defines targets only for workflow_call (lines 4-9) but the os-tests run command
always interpolates ${{ inputs.targets }} (line 75).

.github/workflows/ci-ruby.yml[3-11]
.github/workflows/ci-ruby.yml[47-76]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`os-tests` references `inputs.targets`, but `targets` is only declared for `workflow_call`. For `workflow_dispatch` runs, this input is not defined, so the Bazel command may receive no explicit targets.

### Issue Context
The workflow declares `workflow_dispatch:` with no inputs, but still relies on `inputs.targets`.

### Fix Focus Areas
- .github/workflows/ci-ruby.yml[3-11]
- .github/workflows/ci-ruby.yml[47-76]

### Suggested fix
Add `workflow_dispatch.inputs.targets` mirroring the `workflow_call` input (same default `//rb/...`), or change the command to fall back to `//rb/...` when `inputs.targets` is empty.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


8. Bazel exit code 4 ignored ✓ Resolved 📘 Rule violation ☼ Reliability
Description
The os-tests job treats bazel test exit code 4 as success, which can mask misconfiguration
(e.g., filters producing “no tests found”) and allow CI to go green without running any Ruby tests.
This violates the requirement to avoid error-swallowing constructs in CI/scripts that hide failures.
Code

.github/workflows/ci-ruby.yml[R74-76]

+        --test_tag_filters=${{ matrix.tag-filters }}
+        ${{ inputs.targets }}
+        || { code=$?; [ "$code" -eq 4 ] && exit 0; exit "$code"; }
Evidence
PR Compliance ID 15 forbids error-swallowing in CI/scripts that can hide failures. The added shell
handler explicitly exits 0 when bazel test returns code 4, masking “no tests found”
conditions.

.github/workflows/ci-ruby.yml[74-76]
Best Practice: Learned patterns

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The `os-tests` Bazel invocation converts exit code `4` into success (`exit 0`). This can hide “no tests found” situations caused by incorrect target selection or tag filters, resulting in false-green CI.

## Issue Context
`bazel test` exit code `4` typically indicates that no tests were executed. Treating this as success in PR CI can allow Ruby changes to merge without test coverage.

## Fix Focus Areas
- .github/workflows/ci-ruby.yml[74-76]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


9. macOS runs safari-preview tests 🐞 Bug ☼ Reliability
Description
The macOS os-tests job includes skip-rbe as a positive --test_tag_filters value, which matches
safari-preview tests because they carry the skip-rbe tag; the workflow only performs Safari setup
when inputs.browser == 'safari', so safari-preview targets may run without intended setup.
Code

.github/workflows/ci-ruby.yml[R60-62]

+          - os: macos
+            browser: safari
+            tag-filters: safari,safari-remote,unit,skip-rbe,-lint
Evidence
macOS tag filters include skip-rbe. In Ruby test definitions, safari-preview is tagged with
skip-rbe and is macOS-compatible, and safari integration BUILD files explicitly request
safari-preview targets. Meanwhile, the runner setup only enables Safari when the workflow input is
exactly safari.

.github/workflows/ci-ruby.yml[47-76]
rb/spec/tests.bzl[142-167]
rb/spec/integration/selenium/webdriver/safari/BUILD.bazel[3-13]
.github/workflows/bazel.yml[265-267]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`skip-rbe` is included as a positive tag filter on macOS. Ruby safari-preview targets are tagged `skip-rbe`, so they can be selected even though the job is intended to run Safari.

### Issue Context
- `safari-preview` browser definition includes `skip-rbe` and is macOS-compatible.
- Some integration specs explicitly generate both `safari` and `safari-preview` targets.
- GitHub Actions setup only enables safaridriver when `inputs.browser == 'safari'`.

### Fix Focus Areas
- .github/workflows/ci-ruby.yml[52-63]

### Suggested fix
- Remove `skip-rbe` from macOS `tag-filters`, or
- Add an explicit exclusion: `-safari-preview,-safari-preview-remote` (and `-safari-preview-bidi` if applicable).
- Optionally, if you *do* want safari-preview coverage, add explicit setup steps and set `browser: safari-preview` so bazel.yml can configure it.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


10. Boolean passed as browser ✓ Resolved 🐞 Bug ≡ Correctness
Description
ci-ruby.yml sets matrix.browser: true (boolean) and passes it into bazel.yml where browser is
declared as a string input; this can break reusable-workflow input validation or lead to unexpected
conditional behavior (inputs.browser != '').
Code

.github/workflows/ci-ruby.yml[R54-59]

+          - os: ubuntu
+            browser: true
+            tag-filters: unit,skip-rbe,-lint
+          - os: windows
+            browser: true
+            tag-filters: edge,edge-remote,unit,skip-rbe,-ie,-lint
Evidence
ci-ruby sets browser: true in the matrix (boolean) and forwards it as `browser: ${{ matrix.browser
}}. bazel.yml declares browser as a string input and uses it in conditions like inputs.browser
!= ''`, so passing a boolean is unsafe.

.github/workflows/ci-ruby.yml[52-66]
.github/workflows/bazel.yml[19-28]
.github/workflows/bazel.yml[252-267]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`matrix.browser` is set to YAML boolean `true` for ubuntu/windows, then forwarded to reusable workflow input `browser` which is typed as `string`. This risks workflow-call validation errors and also changes behavior of steps gated on `inputs.browser != ''`.

### Issue Context
The called workflow declares `browser` as `type: string`.

### Fix Focus Areas
- .github/workflows/ci-ruby.yml[52-66]
- .github/workflows/bazel.yml[19-28]

### Suggested fix
- Change `browser: true` to a string value:
 - If you want “no specific browser but enable UI setup”, use `browser: 'enabled'` (string) and update bazel.yml docs accordingly.
 - If you actually intend Edge on Windows, set `browser: 'edge'`.
 - If Ubuntu is unit-only, set `browser: ''` and adjust tag filters accordingly.
- Keep macOS as `browser: safari` for safaridriver enablement.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

11. IE default browser expansion 🐞 Bug ☼ Reliability
Description
rb_integration_test now defaults to browsers=BROWSERS.keys(), which includes "ie", so any
BUILD.bazel that omits an explicit browsers list will start generating *-ie and *-ie-remote test
targets. This likely widens execution/analysis scope unintentionally (DEFAULT_BROWSERS excluding IE
is still defined but no longer used).
Code

rb/spec/tests.bzl[R171-172]

+def rb_integration_test(name, srcs, deps = [], data = [], browsers = BROWSERS.keys(), tags = []):
+    # Generate a library target that is used by //rb/spec:spec to expose all tests to //rb:rubocop.
Evidence
The default browser set now includes IE, and multiple BUILD files omit an explicit browsers list so
they will inherit that new default; the prior IE-excluding default list still exists but is unused,
suggesting an unintended behavior change.

rb/spec/tests.bzl[169-193]
rb/spec/integration/selenium/webdriver/remote/BUILD.bazel[3-9]
rb/spec/integration/selenium/webdriver/bidi/BUILD.bazel[3-16]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`rb_integration_test` changed its default `browsers` parameter from `DEFAULT_BROWSERS` (which excludes IE) to `BROWSERS.keys()` (which includes IE). This causes IE test variants to be generated across packages that don’t explicitly pass `browsers`, expanding the test matrix and leaving `DEFAULT_BROWSERS` unused.

### Issue Context
Several integration BUILD files call `rb_integration_test(...)` without specifying `browsers`, so they inherit the new default.

### Fix Focus Areas
- rb/spec/tests.bzl[169-193]
- rb/spec/integration/selenium/webdriver/remote/BUILD.bazel[3-9]
- rb/spec/integration/selenium/webdriver/bidi/BUILD.bazel[3-16]

### Suggested fix
1. Change the default parameter back to `DEFAULT_BROWSERS` (or explicitly define a new default list that excludes IE if that’s the intent).
2. If any specific suites truly need IE coverage, pass `browsers=["ie", ...]` explicitly in those BUILD files instead of enabling it globally.
3. Remove or repurpose `DEFAULT_BROWSERS` if you intend to keep the new behavior, so the code doesn’t carry misleading dead constants.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


12. Re-enabled skipped Ruby BiDi 🐞 Bug ☼ Reliability
Description
The Ruby target //rb/spec/integration/selenium/webdriver/bidi:browsing_context-firefox-beta-bidi
was removed from .skipped-tests, so it will now run in the RBE CI invocation that consumes this
file. This is a behavior change for CI stability/coverage and should be done only if the test is now
reliably passing.
Code

.skipped-tests[26]

--//rb/spec/integration/selenium/webdriver/bidi:browsing_context-firefox-beta-bidi
Evidence
ci-build.sh passes .skipped-tests as negative target patterns to Bazel; removing an entry means
Bazel will no longer exclude it. The Ruby BiDi package still defines BiDi-tagged integration tests
that produce the referenced *-bidi variants.

scripts/github-actions/ci-build.sh[18-25]
.skipped-tests[1-25]
rb/spec/integration/selenium/webdriver/bidi/BUILD.bazel[3-16]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
A Ruby BiDi target was removed from `.skipped-tests`, which is used to exclude known-problematic tests from the RBE CI run (`scripts/github-actions/ci-build.sh`). This change will cause the target to start running again in CI.

### Issue Context
If the skip was removed because the underlying issue is fixed, consider adding a reference (issue/PR) or ensuring the fix is part of this PR/series.

### Fix Focus Areas
- scripts/github-actions/ci-build.sh[18-25]
- .skipped-tests[1-25]

### Suggested fix
- If the test is still flaky/failing on RBE, restore the `-//rb/spec/integration/selenium/webdriver/bidi:browsing_context-firefox-beta-bidi` entry.
- If it is intentionally re-enabled, add a brief note in the PR description (or adjacent docs) explaining why it’s safe to remove and what changed to make it stable.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


13. Broken Ruby lint instructions 🐞 Bug ⚙ Maintainability
Description
The Bazel test target //rb:lint was removed, but the README still instructs running `bazel test
//rb:lint and claims bazel test //rb/...` runs lint. This will now fail (missing target) and
misleads contributors about how to run Ruby linting.
Code

rb/BUILD.bazel[R213-215]

rb_binary(
    name = "rubocop",
    testonly = True,
Evidence
The Ruby BUILD file no longer defines a lint test target, while the README still documents `bazel
test //rb:lint` as a supported command.

rb/BUILD.bazel[192-221]
README.md[435-445]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`//rb:lint` no longer exists (the lint `rb_test` wrapper was removed), but the README Ruby command table still references `bazel test //rb:lint` and implies `bazel test //rb/...` runs lint.

### Issue Context
Lint is now driven via Bazel `run` on `//rb:rubocop` (and the CI job uses `./go rb:lint`). Documentation should reflect the supported invocation.

### Fix Focus Areas
- rb/BUILD.bazel[192-221]
- README.md[435-445]

### Suggested fix
- Update README Ruby section:
 - Replace `bazel test //rb:lint` with `bazel run //rb:rubocop` and/or `./go rb:lint`.
 - Update `bazel test //rb/...` description to remove “and lint” (or reintroduce a `rb_test(name="lint")` wrapper if you want `bazel test` to run it).
- Ensure any other docs referring to `//rb:lint` are updated similarly.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


View more (2)
14. Ungated presses pointer twice 📘 Rule violation ☼ Reliability
Description
The ActionBuilder integration tests it 'presses pointer twice' and it 'sends keys to element'
were changed to run unconditionally, no longer excluding Safari/Safari Preview, which can cause
cross-browser CI failures when Actions/double-click or Actions+sendKeys behavior is not consistently
supported by a given browser/driver. These tests should be conditionally gated/skipped based on
runtime/browser feature support where unsupported.
Code

rb/spec/integration/selenium/webdriver/action_builder_spec.rb[162]

+        it 'presses pointer twice' do
Evidence
PR Compliance ID 13 requires guarding tests by runtime/browser feature support to avoid
cross-environment failures. In rb/spec/integration/selenium/webdriver/action_builder_spec.rb, the
modified tests at lines 41 and 162 are now unconditional (run for all browsers), while nearby
ActionBuilder/double-click-related tests in the same file still explicitly exclude Safari/Safari
Preview via except: {browser: %i[safari safari_preview]}, indicating there is known or expected
variance in support that these two tests should also account for.

rb/spec/integration/selenium/webdriver/action_builder_spec.rb[162-168]
rb/spec/integration/selenium/webdriver/action_builder_spec.rb[170-179]
rb/spec/integration/selenium/webdriver/action_builder_spec.rb[41-50]
rb/spec/integration/selenium/webdriver/action_builder_spec.rb[28-29]
Best Practice: Learned patterns

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Two browser-sensitive ActionBuilder integration tests (`it 'presses pointer twice'` and `it 'sends keys to element'`) were changed to run unconditionally. If Safari/Safari Preview (or another CI browser/driver) does not reliably support the specific Actions double-click / pointer behavior or Actions+send_keys behavior under test, this can introduce cross-browser CI failures.

## Issue Context
PR Compliance ID 13 requires gating tests by runtime/browser feature support to prevent cross-environment failures. Other ActionBuilder tests in the same spec file remain gated with `except: {browser: %i[safari safari_preview]}`, and an adjacent double-click equivalence test still excludes Safari/Safari Preview, implying known incompatibility or flakiness that should be handled consistently.

## Fix Focus Areas
- rb/spec/integration/selenium/webdriver/action_builder_spec.rb[41-50]
- rb/spec/integration/selenium/webdriver/action_builder_spec.rb[162-179]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


15. Affected browser tests can skip 🐞 Bug ≡ Correctness
Description
os-tests tag filters do not include chrome or firefox, but Ruby integration tests are tagged
with their browser name (e.g., chrome, firefox); if the affected targets include those tests,
--test_tag_filters can filter them out so they never run in ci-ruby despite being affected.
Code

.github/workflows/ci-ruby.yml[R53-62]

+        include:
+          - os: ubuntu
+            browser: true
+            tag-filters: unit,skip-rbe,-lint
+          - os: windows
+            browser: true
+            tag-filters: edge,edge-remote,unit,skip-rbe,-ie,-lint
+          - os: macos
+            browser: safari
+            tag-filters: safari,safari-remote,unit,skip-rbe,-lint
Evidence
Ruby integration tests are tagged with their browser name (local) and ${browser}-remote (remote).
ci-ruby os-tests only includes edge/safari tags, so chrome/firefox-tagged integration tests can be
filtered out even when passed via inputs.targets.

.github/workflows/ci-ruby.yml[47-76]
rb/spec/tests.bzl[170-215]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`os-tests` uses `--test_tag_filters` that select only edge/safari/unit/skip-rbe (plus exclusions). Ruby integration tests are tagged with `[browser]` and `[browser-remote]`, so affected chrome/firefox tests may be excluded and not exercised on GitHub-hosted OS runners.

### Issue Context
Ruby integration test generation appends the browser tag (e.g., `chrome`) to each test target.

### Fix Focus Areas
- .github/workflows/ci-ruby.yml[47-76]

### Suggested fix
- If you intend to run affected integration tests for chrome/firefox on OS runners, add `chrome,chrome-remote,firefox,firefox-remote` (and beta variants if needed) to the appropriate matrix entries.
- If you intentionally rely on RBE for chrome/firefox coverage, consider documenting that explicitly in the workflow and ensure required CI checks still cover those targets.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Qodo Logo

Comment thread .github/workflows/ci-ruby.yml Outdated
Comment thread .github/workflows/ci.yml
Comment thread .github/workflows/ci-ruby.yml Outdated
Comment thread .github/workflows/ci-ruby.yml Outdated
Comment thread .github/workflows/ci-ruby.yml Outdated
@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented May 22, 2026

Persistent review updated to latest commit fa50a25

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Streamlines Ruby CI on GitHub Actions runners by switching Ruby workflow execution to be driven by the Bazel “affected targets” list, and consolidating more signal into OS-specific test jobs (instead of a broader unit-test OS matrix).

Changes:

  • Pass Ruby affected targets from ci.yml into ci-ruby.yml and gate Ruby CI on those targets being present.
  • Replace the prior Ruby CI job layout with OS-specific test runs using Bazel tag filters, plus a smaller Ruby interpreter unit-test matrix.
  • Update Bazel tags for Ruby unit and lint targets to support the new filtering approach.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
rb/spec/tests.bzl Retags Ruby unit test targets to support CI tag filtering.
rb/BUILD.bazel Retags the Ruby lint target to be filterable via Bazel tag filters.
.skipped-tests Removes a previously skipped Ruby BiDi test label from the skip list.
.github/workflows/ci.yml Emits Ruby affected targets (rb_targets) and passes them into the Ruby reusable workflow.
.github/workflows/ci-ruby.yml Restructures Ruby CI to run OS-specific Bazel tests over affected targets with tag filters and a reduced unit-test matrix.

Comment thread .github/workflows/ci-ruby.yml
Comment thread .github/workflows/ci-ruby.yml Outdated
Comment thread .github/workflows/ci-ruby.yml Outdated
@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented May 22, 2026

Persistent review updated to latest commit 2f0d00f

Comment thread .github/workflows/ci-ruby.yml Outdated
@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented May 22, 2026

Persistent review updated to latest commit 035aaf6

@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented May 22, 2026

Persistent review updated to latest commit 7e9d57a

@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented May 22, 2026

Persistent review updated to latest commit 6c8325e

@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented May 22, 2026

Persistent review updated to latest commit 2f4b024

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented May 23, 2026

Persistent review updated to latest commit e6d425f

@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented May 23, 2026

Persistent review updated to latest commit 6d6cb6f

@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented May 23, 2026

Persistent review updated to latest commit f0a30f7

@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented May 23, 2026

Persistent review updated to latest commit 02d2d5c

Comment thread .github/workflows/ci-ruby.yml Outdated
@titusfortner titusfortner marked this pull request as draft May 25, 2026 13:31
@titusfortner titusfortner force-pushed the ci-ruby branch 3 times, most recently from c9df9f6 to f05ec3d Compare May 26, 2026 20:47
@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented Jun 3, 2026

Code review by qodo was updated up to the latest commit e883e27

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Comment thread .github/workflows/ci-ruby.yml
Comment thread .github/workflows/ci.yml
Comment thread .github/workflows/ci-ruby.yml
Comment thread .github/workflows/ci-ruby.yml
@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented Jun 3, 2026

Code review by qodo was updated up to the latest commit 8f2902e

@titusfortner titusfortner merged commit 13c5344 into trunk Jun 5, 2026
31 checks passed
@titusfortner titusfortner deleted the ci-ruby branch June 5, 2026 16:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

B-build Includes scripting, bazel and CI integrations C-rb Ruby Bindings

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants