The TypeScript "parser wrapper" is a Node.js program launched from the JavaScript extractor in order to extract TypeScript code.
The two processes communicate via a command/response protocol via stdin/stdout.
To debug the parser script:
-
Launch an extraction process with the environment variable
SEMMLE_TYPESCRIPT_NODE_FLAGSset to--inspect, or--inspect-brkif you wish to pause on entry. -
Open VSCode and choose "Debug: Attach to Node process" from the command palette, and choose the process that looks like the parser-wrapper.
The wrapper uses the exact typescript@7.1.0-dev.20261015.1 package for both
compilation/type checking and the native runtime. Its integrity and those of all
platform-specific optional dependencies are recorded in package-lock.json.
npm ci
npm run build
npm test
node build/main.js --versionNode.js 16.20.0 is the minimum supported runtime, matching the pinned TypeScript
package's engine requirement. The existing Bazel toolchain uses Node.js 18.15.0
and does not need an upgrade. Build scripts invoke TypeScript's
lib/tsc.js launcher explicitly because these Node.js versions cannot load its
extensionless ESM bin/tsc entry point. The installed wrapper must include its
package.json to retain the ESM package boundary.
The existing TSLint rules require the legacy JavaScript compiler API, not the native TypeScript 7 API. If compatible TSLint tooling is not already installed, run it separately without replacing the wrapper's build/runtime dependency:
npm exec --yes --package=tslint@6.1.3 --package=typescript@5.9.2 -- tslint --project .SEMMLE_TYPESCRIPT_HOME still overrides the runtime dependency. It must point to
the installed TypeScript package directory (containing package.json), with the
native typescript/sync and typescript/ast entry points and matching executable.
A legacy 5.x module or a directory from which the bundled package merely happens
to be resolvable is not a valid override.
The Bazel build produces two archives: :typescript (javascript.zip) contains
the compiled ESM wrapper and JavaScript API; :native (native.zip) contains the
matching native packages for the target CodeQL distribution:
| Distribution | Native packages |
|---|---|
linux64 |
@typescript/typescript-linux-x64 |
linux-arm64 |
@typescript/typescript-linux-arm64 |
osx64 |
@typescript/typescript-darwin-arm64 and @typescript/typescript-darwin-x64 |
win64 |
@typescript/typescript-win32-x64 |
The build installs the selected packages explicitly with --force --no-save,
using the pinned versions and integrities in the lockfile. Merely passing
--force to npm ci still filters optional dependencies by the build host.
package-runtime.mjs verifies the native package versions and marks executables
as 0755 before ZIP creation. The common CodeQL archive contains the wrapper;
the architecture archive installs native packages into
tools/typescript-parser-wrapper/node_modules/@typescript. Both archives must
be installed, with executable permissions preserved. No separate TypeScript
installation is needed.
Run the installed-package smoke test with:
bazel test //javascript/extractor/lib/typescript:package-testIt checks all distribution selections, mismatched runtime rejection, common/arch
ZIP separation and executable metadata, then installs the merged archives with
the production ripunzip and parses TS, TSX, and malformed source through the
installed JSON protocol. It runs on the host platform and uses Bazel's Node.js
runtime. The Java extractor test suite also installs both ZIPs and restores
executable permissions using their central-directory metadata.
All four distribution runtime ZIPs have been cross-assembled and inspected. Installed parsing has been verified with Node.js 18.15.0 on macOS arm64 and on macOS x64 through Rosetta. CI integration tests have also passed on Linux x64, Linux arm64, and Windows x64, including Windows 2022.
Each file owns a fresh native API/process. AST projection and JSON serialization finish before the program is disposed and the API is closed, including on failure. No native process or compiler cache is retained between files, during stdout backpressure, or across reset/quit. This deliberately favors bounded resource lifetime over reusing a compiler process; the measured cost is described below.
SEMMLE_TYPESCRIPT_MEMORY_THRESHOLD remains a positive megabyte budget, defaulting
to 1000 when the Java extractor does not supply it. At native-process creation,
the wrapper subtracts Node's current RSS (not just its V8 heap) from that budget
and passes the remaining bytes, with a minimum of one, as the child's
GOMEMLIMIT. The parent's original GOMEMLIMIT is restored after spawning.
This is a Go garbage-collector soft limit, not a hard combined RSS ceiling:
Node allocations during extraction, native non-Go allocations, and oversized
individual files can exceed the budget. Closing the native process after every
file prevents native caches and heap capacity from accumulating across files.
Node RSS exceeding the budget produces a warning on stderr.
Native stderr is inherited separately from the JSON stdout protocol. Syntax diagnostics remain successful AST responses; native, I/O, and protocol failures are reported on stderr and terminate the wrapper with a nonzero exit status. Reset, quit, EOF, and failure invalidate queued prefetch work; quit also closes stdin without requiring the sender to close its pipe.
The wrapper targets the native typescript/sync and typescript/ast entry
points, using the same pinned dependency in the production ESM build.
npm test builds the production wrapper with the pinned compiler and runs
test/wrapper.test.mjs and test/lifecycle.test.mjs against its compiled output,
copied with its ESM package boundary into temporary installations. No separate
probe installation or legacy compiler is needed for these tests.
AST nodes are explicitly projected into plain objects before their native
program is disposed. The projection preserves the Java converter's property
names, including optional member tokens, type-parameter defaults, import
attributes, and namespace flags. EOF metadata retains the EndOfFileToken
spelling; the native TypeOfExpression, TypeOfKeyword, InstanceOfKeyword,
and KeyOfKeyword names already match the Java converter.
The tests cover AST projection, metadata, UTF-16 source positions, syntax diagnostics, TSX, repeated parses, native process disposal, the child memory budget, reset/quit/EOF, cancelled prefetch, runtime overrides, and fatal native failures. Run them with Node.js 16.20.0 to verify the minimum supported runtime.
Native interface extends and class implements elements are type references.
The wrapper projects them into the expression-shaped heritage nodes consumed by
the Java converter, preserving generic arguments and qualified names.
The native parser still produces ASTs for legacy assert { ... } import
assertions, but adds diagnostic 2880. The wrapper excludes only that diagnostic
to preserve previously accepted syntax; genuine syntax diagnostics are unchanged.
Import-attribute keys and values are now explicitly serialized. Their absence
from the old wrapper's JSON was a bug: the Java converter expects key/value,
but the old filter exposed neither correctly. The corresponding TRAP expectation
was regenerated; existing QL expectations did not need changes.
Rescanning preserves token text and offsets while retaining the original pre-rescan token kinds on the wire. In particular, regex and template-tail tokens keep the classifications expected by existing TRAP tests.
Source-phase imports (import source wasm from "./module.wasm") are accepted.
Their declaration, imported path, and variable bindings use the existing import
representation; the schema does not yet distinguish the source phase. The new
TypeScript/SourceImports QL test covers these bindings.
Validation on October 19, 2026 passed the pinned-compiler build/type check, existing TSLint rules, parser compatibility and lifecycle tests, 500 Java tests, 68 TypeScript QL tests, the syntax-error query test, both extraction diagnostic integration tests, and the installed-package smoke test. The internal-error integration test deliberately terminates the native child rather than relying on the old parser's stack overflow.
All 19 production-output regression tests also pass on Node.js 16.20.0, the minimum supported runtime.
For QL tests, install the newly built extractor and explicitly select it; otherwise the CLI may silently exercise an older installed extractor:
# From the repository root, with the Java extractor build dependencies available:
bazel run //javascript:install
codeql resolve languages --search-path=javascript/extractor-pack:. --format=json
codeql test run -j4 --search-path=javascript/extractor-pack:. \
javascript/ql/test/library-tests/TypeScript \
javascript/ql/test/query-tests/LanguageFeatures/SyntaxErrorOn macOS arm64 with Node.js 18.15.0, a comparison against the pre-migration
wrapper from 1971c368e1d compiled with TypeScript 5.9.2 gave these medians over three
runs:
| Workload | 5.9.2 time | 7.1 time | 5.9.2 combined RSS | 7.1 combined RSS |
|---|---|---|---|---|
| 103 TypeScript fixtures (33,589 bytes) | 0.237 s | 1.349 s | 124.1 MiB | 113.9 MiB |
| Five parses of TypeScript's 588,128-byte declaration file | 0.991 s | 1.116 s | 269.3 MiB | 249.1 MiB |
Timing covers prepare-files, extraction, serialization, and protocol responses after the metadata handshake, not Java conversion or database import. Combined RSS is the maximum sampled sum of Node and its native descendants, sampled every 5 ms; short-lived peaks may be missed. The per-file native process costs about 5.7x on tiny files and 1.13x on the declaration workload, while measured memory is slightly lower. The bounded lifecycle is retained deliberately; this migration does not claim a speedup.