This directory contains build, test, utility, and release packaging scripts for the ETL-SQL engine. To support developers across Windows, Linux, and macOS, core developer workflows are provided in both PowerShell (.ps1) and Bash (.sh) versions.

Gate F is intentionally operator-run because the spill-backed billion-row scenario can take hours. Start it with ./scripts/Test-GateF.ps1; inspect progress from another shell with Get-Content ./certification-results/gate-f-1b/status.json. Re-running the same command skips completed scenario artifacts; use -Force only when intentionally replacing prior results. | test-service-capacity.mjs | Node.js | Cross-platform | Runs stepped Portal-user and Orchestrator-job capacity workloads from a JSON configuration and writes JSON/Markdown reports. | | test-capacity-workload-configs.mjs | Node.js | Cross-platform | Validates all checked-in capacity workload JSON files with the capacity harness --validate-only mode. | | compare-capacity-results.mjs | Node.js | Cross-platform | Compares two service-capacity reports for p95 latency, throughput, and error-rate regressions. | | test-service-capacity-smoke.mjs | Node.js | Cross-platform | Runs the capacity harness against local mock endpoints to verify report generation without deployed services. | | Test-PreRelease.ps1 / test-pre-release.sh | PowerShell / Bash | Cross-platform | Runs the local-first pre-release validation gate with resumable phases, optional Docker/Standard-scale/installer checks, and JSON/Markdown reports under release-validation/. | | Test-VulnerablePackages.ps1 | PowerShell | Cross-platform | CI gate that fails when any NuGet package (direct or transitive) has a known vulnerability. Requires a prior dotnet restore; response procedure in SECURITY.md §13. | | sync-assets.ps1 / sync-assets.js | PowerShell / JavaScript | Cross-platform | Synchronizes canonical shared browser assets from src/ETL-SQL.ReportRuntime to dependent shell host directories (VS Code extension, portal, player). | | generate-syntax-index.js | JavaScript | Cross-platform | Regenerates the canonical token inventory appendix inside Docs/Syntax_Index.md from LanguageMetadata.cs. | | install.ps1 / install.sh | PowerShell / Bash | Cross-platform | Boostrapping workstation installers that download, unpack, and register the ETL-SQL SDK to the user's PATH. | | Master-Release.ps1 | PowerShell | Windows | Orchestrator script to validate tools, compile release binaries, execute smoke tests, build the UI, bundle cross-platform packages, and construct installer setups. | | publish_release.ps1 | PowerShell | Windows | Publishes the CLI applications for different platforms (Windows, Linux, macOS) targeting x64 and arm64 architectures. | | build_msi.ps1 | PowerShell | Windows | Bundles Windows published binaries into an MSI setup package using the WiX candle/light compilation toolset. | | build_linux_packages.sh | Bash | Linux / WSL | Packages published binaries into Debian (.deb) and RedHat (.rpm) distributions using standard linux packaging utilities. | | build_mac_dmg.sh | Bash | macOS | Compiles a macOS application bundle and outputs a signed, notarized DMG mount disk image. | | build_vsix.ps1 / build_vsix.sh | PowerShell / Bash | Cross-platform | Builds and packages the VS Code extension bundle into a publish-ready .vsix file using vsce. | | publish_vsix.ps1 / publish_vsix.sh | PowerShell / Bash | Cross-platform | Packages a platform-targeted .vsix with bundled native binaries for a specific runtime identifier (win-x64, linux-x64, osx-x64, osx-arm64). | | generate-third-party-inventory.js | JavaScript | Cross-platform | Scans npm and NuGet locks to build the third-party dependency licensing inventory report. | | test_sdk_build.ps1 / test_sdk_build.sh | PowerShell / Bash | Cross-platform | Verifies compilation of self-contained single-file SDK CLI tools and runs validation smoke tests on them. | | Set-Version.ps1 / set-version.sh | PowerShell / Bash | Cross-platform | Updates the version string across all canonical locations (build props, manifests, docs, scripts). Does not modify CHANGELOG.md. |

To compile the .NET engine projects, Vite React UI components, and the VS Code TypeScript extension:

• Windows (PowerShell):

  .\scripts\build-debug.ps1

• Linux / macOS (Bash):

  ./scripts/build-debug.sh

To build while bypassing the extension's unit testing phase, pass --skip-tests (or -SkipTests).

To run targeted smoke tests (Categories: Core Language, Path Security, Reporting, Portal):

• Windows (PowerShell):

  .\scripts\test-smoke.ps1 -Lane all

• Linux / macOS (Bash):

  ./scripts/test-smoke.sh --lane all

Runs groups of tests mapped to standard pipeline stages:

• Windows (PowerShell):

  .\scripts\test-lane.ps1 -Lane fast

• Linux / macOS (Bash):

  ./scripts/test-lane.sh --lane fast

Supported lanes are: smoke, fast, engine, portal, integration, perf, release, full, benchmarks, slt.

To see what each lane currently contains without running tests:

.\scripts\Get-TestLaneInventory.ps1
.\scripts\Get-TestLaneInventory.ps1 -Format Json -OutFile test-inventory.json

The inventory is a static visibility report and includes fast-lane exclusion gaps. Use test-lane.ps1 for authoritative pass/fail execution.

Run this before pushing release tags or building release installers. It is designed to catch failures locally before spending GitHub-hosted runner time.

• Windows (PowerShell):

  .\scripts\Test-PreRelease.ps1

• Linux / macOS (Bash):

  ./scripts/test-pre-release.sh

Useful options:

# PowerShell
.\scripts\Test-PreRelease.ps1 -Resume
.\scripts\Test-PreRelease.ps1 -Explain -IncludeSlt
.\scripts\Test-PreRelease.ps1 -Quick -IncludeSlt
.\scripts\Test-PreRelease.ps1 -IncludeSlt
.\scripts\Test-PreRelease.ps1 -IncludeDockerIntegration
.\scripts\Test-PreRelease.ps1 -IncludeStandardScale
.\scripts\Test-PreRelease.ps1 -BuildInstallers -Platforms win-x64

# Bash
./scripts/test-pre-release.sh --resume
./scripts/test-pre-release.sh --explain --include-slt
./scripts/test-pre-release.sh --quick --include-slt
./scripts/test-pre-release.sh --include-slt
./scripts/test-pre-release.sh --include-docker-integration
./scripts/test-pre-release.sh --include-standard-scale
./scripts/test-pre-release.sh --build-installers --platforms linux-x64

Reports and logs are written to release-validation/. Use -Resume / --resume after fixing a failed phase; the script only reuses completed phases when the source fingerprint still matches, unless -ForceResume / --force-resume is supplied.

-Explain / --explain prints the phase list without running it. -Quick / --quick skips Node, scale, Docker, and installer phases. -IncludeSlt / --include-slt adds the SQL Logic Test lane to the local release gate.

The PowerShell and Bash gates run the same phases in the same order. The Bash gate reuses the canonical PowerShell helpers for a few phases (dependency-audit self-test, NuGet dependency audit, cert-baseline regression checks) via pwsh, so those phases require PowerShell 7+ (pwsh) on PATH even on Linux/macOS. This keeps a single source of truth rather than a parallel Bash port.

The full plan with -IncludeSlt -IncludeDockerIntegration -IncludeStandardScale -BuildInstallers -Platforms win-x64 is: asset drift check; secret scan; restore; dependency-audit self-test; NuGet dependency audit; SBOM generation; release build; format verify (auto-fixes drift); smoke lane; fast lane; N→N+1 upgrade-path drill; sample scripts; SLT lane; VS Code npm ci/audit/compile/VSIX-package/unit tests; smoke scale certification and baseline check; Docker integration lane; standard scale certification and baseline check; publish artifacts; Windows MSI.

Windows MSI packaging requires WiX Toolset v3.x (candle.exe and light.exe). On a clean Windows CI runner, install it before build_msi.ps1:

choco install wixtoolset -y --no-progress --skip-if-installed

Runs the SQLite Logic test suite, generating timestamped output folders with standard teed console logs and TRX test results files:

• Windows (PowerShell):

  .\scripts\Test-SltCorpus.ps1 -CorpusOnly

• Linux / macOS (Bash):

  ./scripts/Test-SltCorpus.sh --corpus-only

• TRX Summarizer: You can run .\scripts\Parse-SltResults.ps1 in Windows PowerShell afterward to parse the resulting TRX file and print execution metrics directly to your shell.

To refresh the generated appendix in Docs/Syntax_Index.md after changing LanguageMetadata.cs:

• Cross-platform:

  node ./scripts/generate-syntax-index.js

To verify that the generated appendix matches the source of truth without writing the file, pass --check.

To update the version string across all canonical locations before tagging a release:

• Windows (PowerShell):

  .\scripts\Set-Version.ps1 -Version "0.9.0"

• Linux / macOS (Bash):

  ./scripts/set-version.sh 0.9.0

The scripts update Directory.Build.props, the VS Code extension manifest and lock file, README.md, all release scripts, user-facing docs, and architecture doc headers. After running, manually add a ## [X.Y.Z] entry to CHANGELOG.md, then commit and tag when ready.

To keep report rendering layers identical across host environments (such as the VS Code extension, Portal instance, and standalone Player), the canonical assets are hosted in src/ETL-SQL.ReportRuntime/Resources/Shared/.

Never edit synced host assets directly. Edit the canonical source file first, then synchronize using:

node ./scripts/sync-assets.js

To verify asset synchronization in a CI build environment, run:

node ./scripts/sync-assets.js -Check