Skip to content

ci: shard java tests #6337

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gz merged 4 commits into from
Jun 28, 2026
Merged

ci: shard java tests #6337

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
db42fea
ci: cache GHA artifacts via in-cluster MinIO; cap parallelism
gz Jun 28, 2026
b363509
adapters: gate heap profiling behind with-heap-profiling feature
gz Jun 28, 2026
a5ab793
[SQL] clean up udf/stubs after tests; isolate cargo check target
gz Jun 28, 2026
afefb36
ci: bump dev image to sha-a5ab793b8d261068867b9fdfad3f04157b2536fc
gz Jun 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions .github/actions/feldera-minio-cache/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: 'Feldera MinIO cache'
description: |
Wrapper around runs-on/cache that targets the in-cluster MinIO bucket used
for GitHub Actions caches (default: gha-cache). The caller is responsible
for providing AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, SCCACHE_REGION,
and SCCACHE_ENDPOINT in the job or workflow env — they double as sccache
config in every workflow that uses this action.

inputs:
path:
description: 'Newline-separated list of paths to cache'
required: true
key:
description: 'Primary cache key'
required: true
restore-keys:
description: 'Newline-separated fallback restore keys'
required: false
default: ''
bucket:
description: 'MinIO bucket name'
required: false
default: 'gha-cache'

runs:
using: composite
steps:
- uses: runs-on/cache@88d90644011a3a9957fd141a106f5a94f9794203 # v5.0.7
env:
AWS_REGION: ${{ env.SCCACHE_REGION }}
AWS_ENDPOINT_URL: http://${{ env.SCCACHE_ENDPOINT }}
AWS_S3_FORCE_PATH_STYLE: "true"
RUNS_ON_S3_BUCKET_CACHE: ${{ inputs.bucket }}
# Defend against AWS_SESSION_TOKEN leaking from a configure-aws-credentials
# step run earlier in the job — MinIO rejects requests carrying it.
AWS_SESSION_TOKEN: ""
with:
path: ${{ inputs.path }}
key: ${{ inputs.key }}
restore-keys: ${{ inputs.restore-keys }}
2 changes: 1 addition & 1 deletion .github/workflows/build-docker-dev.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:
docker_platform: linux/arm64
runs-on: ${{ matrix.runner }}
container:
image: ghcr.io/feldera/feldera-dev:sha-3613839bd73108aee58017bf75dda42f7a3d0bb4
image: ghcr.io/feldera/feldera-dev:sha-a5ab793b8d261068867b9fdfad3f04157b2536fc

steps:
- name: Show Kubernetes node
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/build-docker.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ jobs:
docker_platform: linux/arm64
runs-on: ${{ matrix.runner }}
container:
image: ghcr.io/feldera/feldera-dev:sha-3613839bd73108aee58017bf75dda42f7a3d0bb4
image: ghcr.io/feldera/feldera-dev:sha-a5ab793b8d261068867b9fdfad3f04157b2536fc

steps:
- name: Show Kubernetes node
Expand Down
43 changes: 38 additions & 5 deletions .github/workflows/build-java.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,27 @@ on:
workflow_call:
workflow_dispatch:

env:
# MinIO credentials and endpoint — feldera-minio-cache reads these to talk
# to the in-cluster gha-cache bucket. AWS_* are also unused by upload-jar
# (configure-aws-credentials overrides them via $GITHUB_ENV).
AWS_ACCESS_KEY_ID: "${{ secrets.CI_K8S_MINIO_ACCESS_KEY_ID }}"
AWS_SECRET_ACCESS_KEY: "${{ secrets.CI_K8S_MINIO_SECRET_ACCESS_KEY }}"
SCCACHE_REGION: ${{ vars.SCCACHE_REGION }}
SCCACHE_ENDPOINT: ${{ vars.SCCACHE_ENDPOINT }}
# Pin Maven's local repo so it matches the cache path regardless of
# whatever HOME the GHA container runner sets — JVM's user.home reads
# /etc/passwd, not $HOME, and we don't want to rely on that match.
MAVEN_OPTS: -Dmaven.repo.local=/home/ubuntu/.m2/repository

jobs:
build-jar:
name: Build Compiler
runs-on: [k8s-runners-amd64]
permissions:
id-token: write
contents: read
container:
image: ghcr.io/feldera/feldera-dev:sha-3613839bd73108aee58017bf75dda42f7a3d0bb4
image: ghcr.io/feldera/feldera-dev:sha-a5ab793b8d261068867b9fdfad3f04157b2536fc
steps:
- name: Show Kubernetes node
if: always()
Expand All @@ -22,14 +34,16 @@ jobs:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6

- uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
- name: Cache JVM build dependencies
uses: ./.github/actions/feldera-minio-cache
with:
path: |
/home/ubuntu/.gradle
/home/ubuntu/.m2/repository
./sql-to-dbsp-compiler/calcite
key: sql2dbsp-java-cache-${{ runner.os }}-${{ hashFiles('sql-to-dbsp-compiler/calcite_version.env') }}
key: build-java-jvm-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('sql-to-dbsp-compiler/calcite_version.env', 'sql-to-dbsp-compiler/**/pom.xml') }}
restore-keys: |
sql2dbsp-java-cache-${{ runner.os }}-
build-java-jvm-${{ runner.os }}-${{ runner.arch }}-

- name: gradle version
run: |
Expand Down Expand Up @@ -58,6 +72,25 @@ jobs:
path: build-artifacts
retention-days: 7

# S3 upload runs in a separate job so the OIDC session token never enters
# the build-jar environment, where it would poison the cache save phase
# of runs-on/cache (MinIO rejects requests carrying a stray AWS_SESSION_TOKEN).
upload-jar:
name: Upload JAR to S3
needs: build-jar
runs-on: [k8s-runners-amd64]
permissions:
id-token: write
contents: read
container:
image: ghcr.io/feldera/feldera-dev:sha-a5ab793b8d261068867b9fdfad3f04157b2536fc
steps:
- name: Download build artifact
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
with:
name: feldera-sql-compiler
path: build-artifacts

- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@00943011d9042930efac3dcd3a170e4273319bc8 # v5.1.0
with:
Expand Down
40 changes: 24 additions & 16 deletions .github/workflows/build-rust.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,13 @@ env:
SCCACHE_REGION: ${{ vars.SCCACHE_REGION }}
AWS_ACCESS_KEY_ID: "${{ secrets.CI_K8S_MINIO_ACCESS_KEY_ID }}"
AWS_SECRET_ACCESS_KEY: "${{ secrets.CI_K8S_MINIO_SECRET_ACCESS_KEY }}"
# sccache cannot cache incremental rustc invocations; release already
# disables incremental, but doc tests run in debug mode.
CARGO_INCREMENTAL: 0
# Cap cargo parallelism at the k8s runner core budget so arm64 pods don't
# OOM from too many parallel linkers. Override via the CI_RUNNER_CORES repo

@mythical-fred mythical-fred May 28, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tiny consistency nit: RUST_TEST_THREADS in test-unit.yml and test-adapters.yml is set at the env level; CARGO_BUILD_JOBS here is also env-level. Good. The fallback || 20 is duplicated five times across workflows — fine for now, but if the answer ever needs to change you’ll touch five files. A repo-level vars.CI_RUNNER_CORES set to 20 would let you drop the || 20 literal everywhere.

# variable; the fallback matches the current 20-core pod.
CARGO_BUILD_JOBS: ${{ vars.CI_RUNNER_CORES || 20 }}

jobs:
build-rust:
Expand All @@ -33,7 +40,7 @@ jobs:
runs-on: ${{ matrix.runner }}

container:
image: ghcr.io/feldera/feldera-dev:sha-3613839bd73108aee58017bf75dda42f7a3d0bb4
image: ghcr.io/feldera/feldera-dev:sha-a5ab793b8d261068867b9fdfad3f04157b2536fc

steps:
- name: Show Kubernetes node
Expand All @@ -45,27 +52,26 @@ jobs:
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6

- name: Cache Cargo registry and index
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
uses: ./.github/actions/feldera-minio-cache
with:
# registry/src excluded — cargo regenerates it from registry/cache.
path: |
~/.cargo/registry
~/.cargo/git
key: cargo-registry-${{ runner.os }}-${{ matrix.target }}-${{ hashFiles('**/Cargo.lock') }}
~/.cargo/registry/cache
~/.cargo/registry/index
~/.cargo/git/db
key: build-rust-cargo-${{ runner.os }}-${{ matrix.target }}-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
cargo-registry-${{ runner.os }}-${{ matrix.target }}-
build-rust-cargo-${{ runner.os }}-${{ matrix.target }}-

# Thanks to rust cargo non-sense it's too hard to split this into test job
#
# limiting jobs with --test-threads 18: doc test call the linker for every test, on arm machines
# this can lead up to 128 parallel linkers being used OOM'ing the pod
# https://github.com/rust-lang/cargo/issues/10702
# Doc tests invoke the linker per test; cap concurrency at the runner
# core budget so arm64 pods don't OOM. See rust-lang/cargo#10702.
- name: Run Rust doc tests
if: matrix.arch == 'x86_64'
run: |
# Don't build the webconsole again for rust tests
export WEBCONSOLE_BUILD_DIR="$(mktemp -d)"
touch $WEBCONSOLE_BUILD_DIR/index.html
cargo test --locked --doc --workspace -- --test-threads 18
cargo test --locked --doc --workspace -- --test-threads ${{ vars.CI_RUNNER_CORES || 20 }}

- name: Build Rust binaries
run: |
Expand Down Expand Up @@ -167,12 +173,14 @@ jobs:
- name: Cache Cargo registry and index
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
with:
# registry/src excluded — cargo regenerates it from registry/cache.
path: |
~/.cargo/registry
~/.cargo/git
key: cargo-registry-${{ runner.os }}-${{ matrix.target }}-${{ hashFiles('**/Cargo.lock') }}
~/.cargo/registry/cache
~/.cargo/registry/index
~/.cargo/git/db
key: build-rust-fda-native-cargo-${{ runner.os }}-${{ matrix.target }}-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
cargo-registry-${{ runner.os }}-${{ matrix.target }}-
build-rust-fda-native-cargo-${{ runner.os }}-${{ matrix.target }}-

- name: Build fda
run: cargo build --release --locked -p fda --target=${{ matrix.target }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci-post-release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ jobs:
adjust-versions:
runs-on: [k8s-runners-amd64]
container:
image: ghcr.io/feldera/feldera-dev:sha-3613839bd73108aee58017bf75dda42f7a3d0bb4
image: ghcr.io/feldera/feldera-dev:sha-a5ab793b8d261068867b9fdfad3f04157b2536fc
steps:
- name: Show Kubernetes node
if: always()
Expand Down
49 changes: 36 additions & 13 deletions .github/workflows/ci-pre-mergequeue.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,27 +12,42 @@ env:
SCCACHE_REGION: ${{ vars.SCCACHE_REGION }}
AWS_ACCESS_KEY_ID: "${{ secrets.CI_K8S_MINIO_ACCESS_KEY_ID }}"
AWS_SECRET_ACCESS_KEY: "${{ secrets.CI_K8S_MINIO_SECRET_ACCESS_KEY }}"
# sccache cannot cache rustc invocations when incremental is on.
CARGO_INCREMENTAL: 0

jobs:
# This job needs to be called main (the same as the ci.yml workflow)
# because of how merge queues work: https://stackoverflow.com/a/78030618
main:
container:
image: ghcr.io/feldera/feldera-dev:sha-3613839bd73108aee58017bf75dda42f7a3d0bb4
image: ghcr.io/feldera/feldera-dev:sha-a5ab793b8d261068867b9fdfad3f04157b2536fc
runs-on: [k8s-runners-amd64]
steps:
- name: Show Kubernetes node
if: always()
run: |
echo "K8S node: ${K8S_NODE_NAME}"

# Compute the internal-PR boolean once so individual steps can reference
# it; cheaper to grep and less fragile when triggers expand.

@mythical-fred mythical-fred May 29, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice consolidation — steps.pr-check.outputs.internal is much clearer than the 3-pronged expression. Tiny nit: in the heredoc, ${{ github.event.pull_request.head.repo.full_name }} is interpolated by Actions, not the shell, so an attacker-controlled fork repo name (e.g. foo" && touch /tmp/pwned && echo ") would land verbatim inside the [[ ... ]]. Fork repo names are validated by GitHub (alphanumerics, hyphens, periods, underscores, slashes), so this is safe today, but using env: indirection and quoting "$HEAD_REPO"/"$ACTOR" is the standard hardening for this pattern and matches the pull_request_target hardening rule already documented in the repo.

- name: Determine if this is an internal PR
id: pr-check
run: |
if [[ "${{ github.event_name }}" == "pull_request" \
&& "${{ github.event.pull_request.head.repo.full_name }}" == "${{ github.repository }}" \
&& "${{ github.actor }}" != "dependabot[bot]" ]]; then
echo "internal=true" >> "$GITHUB_OUTPUT"
else
echo "internal=false" >> "$GITHUB_OUTPUT"
fi

- name: Disable sccache for fork/dependabot PRs (secrets unavailable)
if: ${{ github.event.pull_request.head.repo.full_name != github.repository || github.actor == 'dependabot[bot]' }}
if: ${{ steps.pr-check.outputs.internal != 'true' }}
run: |
echo "RUSTC_WRAPPER=" >> "$GITHUB_ENV"

- name: Generate GitHub App token
if: ${{ github.event.pull_request.head.repo.full_name == github.repository && github.actor != 'dependabot[bot]' }}
if: ${{ steps.pr-check.outputs.internal == 'true' }}
id: app-token
uses: actions/create-github-app-token@fee1f7d63c2ff003460e3d139729b119787bc349 # v2
with:
Expand All @@ -41,34 +56,42 @@ jobs:
permission-contents: write

- name: Checkout (internal PRs)
if: ${{ github.event.pull_request.head.repo.full_name == github.repository && github.actor != 'dependabot[bot]' }}
if: ${{ steps.pr-check.outputs.internal == 'true' }}
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
# This needs to be set to a token to trigger a follow-up workflow
# in case some changes were corrected.
token: ${{ steps.app-token.outputs.token }}

- name: Checkout (fork/dependabot PRs)
if: ${{ github.event.pull_request.head.repo.full_name != github.repository || github.actor == 'dependabot[bot]' }}
if: ${{ steps.pr-check.outputs.internal != 'true' }}
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6

# Caching is skipped for fork/dependabot PRs: MinIO secrets aren't
# available in those contexts.
- name: Cache uv pre-commit environments
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
if: ${{ steps.pr-check.outputs.internal == 'true' }}
uses: ./.github/actions/feldera-minio-cache
with:
path: |
~/.cache/pre-commit
~/.cache/uv
key: pre-commit-uv-1|${{ hashFiles('.pre-commit-config.yaml') }}
key: ci-pre-mergequeue-precommit-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.pre-commit-config.yaml') }}
restore-keys: |
ci-pre-mergequeue-precommit-${{ runner.os }}-${{ runner.arch }}-

- name: Cache Cargo registry and index
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
if: ${{ steps.pr-check.outputs.internal == 'true' }}
uses: ./.github/actions/feldera-minio-cache
with:
# registry/src excluded — cargo regenerates it from registry/cache.
path: |
~/.cargo/registry
~/.cargo/git
key: cargo-registry-${{ runner.os }}-${{ matrix.target }}-${{ hashFiles('**/Cargo.lock') }}
~/.cargo/registry/cache
~/.cargo/registry/index
~/.cargo/git/db
key: ci-pre-mergequeue-cargo-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
cargo-registry-${{ runner.os }}-${{ matrix.target }}-
ci-pre-mergequeue-cargo-${{ runner.os }}-${{ runner.arch }}-

- run: |
# Don't build the webconsole for rust checks
Expand All @@ -89,7 +112,7 @@ jobs:
run: sccache --show-stats
- uses: stefanzweifel/git-auto-commit-action@b863ae1933cb653a53c021fe36dbb774e1fb9403 # b863ae1933cb653a53c021fe36dbb774e1fb9403
# Only attempt auto-fix commits for internal branches (forks and dependabot can't be pushed to)
if: ${{ always() && github.event.pull_request.head.repo.full_name == github.repository && github.actor != 'dependabot[bot]' }}
if: ${{ always() && steps.pr-check.outputs.internal == 'true' }}
with:
commit_message: "[ci] apply automatic fixes"
commit_user_name: feldera-bot
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/publish-crates.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ jobs:
environment:
name: ${{ inputs.environment || 'release' }}
container:
image: ghcr.io/feldera/feldera-dev:sha-3613839bd73108aee58017bf75dda42f7a3d0bb4
image: ghcr.io/feldera/feldera-dev:sha-a5ab793b8d261068867b9fdfad3f04157b2536fc
steps:
- name: Show Kubernetes node
if: always()
Expand Down
7 changes: 5 additions & 2 deletions .github/workflows/test-adapters.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ jobs:
target: aarch64-unknown-linux-gnu
runs-on: ${{ matrix.runner }}
container:
image: ghcr.io/feldera/feldera-dev:sha-3613839bd73108aee58017bf75dda42f7a3d0bb4
image: ghcr.io/feldera/feldera-dev:sha-a5ab793b8d261068867b9fdfad3f04157b2536fc
services:
postgres:
image: postgres:15
Expand Down Expand Up @@ -94,7 +94,10 @@ jobs:
shell: bash
env:
RUST_BACKTRACE: 1
RUST_TEST_THREADS: 16
# Cap libtest parallelism at the runner core budget. Adapter tests
# share the pod with postgres/redpanda/redis sidecars; if those start
# starving we can lower CI_RUNNER_CORES or switch this to its own var.
RUST_TEST_THREADS: ${{ vars.CI_RUNNER_CORES || 20 }}
REDIS_URL: redis://localhost:6379/0
DYNAMODB_ENDPOINT: http://localhost:8000
POSTGRES_URL: postgres://postgres:password@localhost:5432
Expand Down
6 changes: 5 additions & 1 deletion .github/workflows/test-integration-platform.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,7 @@ jobs:
-e FELDERA_AUTH_ISSUER="${{ vars.OIDC_TEST_ISSUER }}" \
-e RUST_LOG=info \
-e RUST_BACKTRACE=1 \
-e CARGO_BUILD_JOBS=${{ vars.CI_RUNNER_CORES || 20 }} \
${{ vars.FELDERA_IMAGE_NAME }}:sha-${{ github.sha }} \
--enable-https \
--https-tls-cert-path /home/ubuntu/test-tls/tls_test.crt \
Expand Down Expand Up @@ -173,7 +174,7 @@ jobs:
FELDERA_HOST: https://localhost:8080

container:
image: ghcr.io/feldera/feldera-dev:sha-3613839bd73108aee58017bf75dda42f7a3d0bb4
image: ghcr.io/feldera/feldera-dev:sha-a5ab793b8d261068867b9fdfad3f04157b2536fc
services:
pipeline-manager:
image: ${{ vars.FELDERA_IMAGE_NAME }}:sha-${{ github.sha }}
Expand All @@ -187,6 +188,9 @@ jobs:
FELDERA_ENABLE_HTTPS: true
# Needed by test_update_runtime.py
FELDERA_UNSTABLE_FEATURES: "testing"
# Cap cargo parallelism inside the manager: each pipeline submit
# triggers a Rust build, and pytest workers can submit concurrently.
CARGO_BUILD_JOBS: ${{ vars.CI_RUNNER_CORES || 20 }}
options: >-
--health-cmd "curl --insecure --fail --request GET --url https://localhost:8080/healthz || exit 1"
--health-interval 10s
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test-integration-runtime.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ jobs:
CI_K8S_MINIO_ACCESS_KEY_ID: ${{ secrets.CI_K8S_MINIO_TEST_BUCKET_ACCESS_KEY_ID }}
CI_K8S_MINIO_SECRET_ACCESS_KEY: ${{ secrets.CI_K8S_MINIO_TEST_BUCKET_SECRET_ACCESS_KEY }}
container:
image: ghcr.io/feldera/feldera-dev:sha-3613839bd73108aee58017bf75dda42f7a3d0bb4
image: ghcr.io/feldera/feldera-dev:sha-a5ab793b8d261068867b9fdfad3f04157b2536fc
steps:
- name: Show Kubernetes node
if: always()
Expand Down
Loading