feat: lift the pure subset of foreign values into IR primops (#178) - #210
Merged
Conversation
Polymorphic hot code bottoms out in opaque curried foreigns — intAdd, ordIntImpl, refEq, boolConj — that the IR optimizer cannot see through, because to the IR a foreign body is text. This lifts the pure return-tree subset of such a body, taken from the actual fork source parsed by the Lua foreign parser (#173), into IR primops that the existing rewrites (beta reduction, case-of-known-constructor #177, inlining, constant folding) can then specialize. - IR.Types: PrimBinOp / PrimNot nodes and the PrimOp enum, defined as the matching Lua operators so lowering is the identity and lifting its inverse (Note [IR primops]). Equality stays the existing Eq node. - Lua.fromIR: lower each primop onto the same-named Lua BinOp / UnOp. - IR.Optimizer: constant-fold primops over literals following Lua 5.1 semantics (integer ±2^53 guard, finite-only floats, divisor-signed modulo, string-only concat, numeric comparisons, boolean short-circuit). - Lua.ForeignLift: translate the liftable subset (curried single-parameter lambdas, if/elseif/else return trees, operators, header-local inlining) to IR, gated by a hard allowlist — a listed export that fails to lift is a compile error, never a silent performance cliff. Lifted exports move out of the foreigns into ordinary bindings, so DCE prunes the now-unread source rows from the emitted FFI. - Backend.compileModules runs the lifter before the optimizer.
- IR.Optimizer.Spec: unit tests for every folding rule (integer range guard, Lua-5.1 modulo sign, finite-float, string concat, numeric comparisons, boolean short-circuit) and the non-folding cases. - Lua.ForeignLift.Spec: the liftable shapes (curried operator, header-local alias, if/elseif/else through a header local, not, concat) and the declined ones (multi-parameter, table index, non-return-tree, else-less if, missing export), plus allowlist membership. - IR.Gen: emit PrimBinOp / PrimNot in both generators, so the optimizer soundness properties fuzz folding for scope/GUC preservation. - Golden.Primops: a runnable module exercising lifted arithmetic, comparison, equality, concat and booleans, with a hand-verified eval oracle. - Golden harness: lift foreigns in compileCorefn too, so the .ir goldens reflect the same pipeline; regenerate the structural goldens (every eval oracle is unchanged — semantics preserved).
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends the PureScript→Lua compiler pipeline by lifting a strictly pure subset of allowlisted foreign exports into new IR primop nodes, enabling downstream IR optimizations (including constant folding) to “see through” previously-opaque foreign definitions and generate more direct Lua operator code.
Changes:
- Introduces IR primop nodes (
PrimBinOp+PrimOpenum, andPrimNot) and lowers them directly to Lua operators. - Adds Lua foreign lifting pass (allowlist-driven) and wires it into both
Backend.compileModulesand the golden-test pipeline. - Implements Lua-5.1-semantics constant folding for primops, expands IR fuzz generators accordingly, and updates/adds tests and golden fixtures (including a new runnable
Golden.Primops).
Reviewed changes
Copilot reviewed 65 out of 74 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| test/ps/src/Golden/Primops/Test.purs | New runnable golden exercising lifted primops via user-level PureScript code. |
| test/ps/output/Golden.Primops.Test/golden.lua | New golden Lua output for Golden.Primops. |
| test/ps/output/Golden.Primops.Test/eval/golden.txt | New eval oracle (hand-verified stdout) for Golden.Primops. |
| test/ps/output/Golden.Primops.Test/eval/.gitignore | Tracks eval harness output file. |
| test/ps/output/Golden.Uncurry.Test/golden.lua | Golden updated to reflect lifted foreigns/primops reducing foreign dispatch. |
| test/ps/output/Golden.TailRecM2Shadow.Test/golden.lua | Golden updated to reflect lifted ordIntImpl and arithmetic primops. |
| test/ps/output/Golden.TailRecM2Shadow.Test/golden.ir | Golden IR updated to include primop nodes and lifted bindings. |
| test/ps/output/Golden.StringCodePoints.Test/golden.lua | Golden updated for lifted Eq/Ord/Semiring/Semigroup/HeytingAlgebra foreigns. |
| test/ps/output/Golden.ProfunctorDictLens.Test/golden.lua | Golden updated to show direct operator lowering rather than foreign calls. |
| test/ps/output/Golden.ProfunctorDictLens.Test/golden.ir | Golden IR updated to include primops in dictionaries and call sites. |
| test/ps/output/Golden.MaybeChainModule.Test/golden.lua | Golden refreshed due to upstream optimizer/pipeline changes. |
| test/ps/output/Golden.MaybeChainModule.Test/golden.ir | Golden IR refreshed due to upstream optimizer/pipeline changes. |
| test/ps/output/Golden.MaybeChain.Test/golden.lua | Golden refreshed due to upstream optimizer/pipeline changes. |
| test/ps/output/Golden.MaybeChain.Test/golden.ir | Golden IR refreshed due to upstream optimizer/pipeline changes. |
| test/ps/output/Golden.Loopification.Test/golden.lua | Golden updated for lifted foreigns and primop lowering (e.g. subtraction worker). |
| test/ps/output/Golden.Loopification.Test/golden.ir | Golden IR updated to reflect new primop nodes and lifted bindings. |
| test/ps/output/Golden.LongWriterBind.Test/golden.lua | Golden refreshed due to upstream optimizer/pipeline changes. |
| test/ps/output/Golden.LongWriterBind.Test/golden.ir | Golden IR refreshed due to upstream optimizer/pipeline changes. |
| test/ps/output/Golden.LongReaderBind.Test/golden.lua | Golden updated to reflect lifted arithmetic foreigns into primops/workers. |
| test/ps/output/Golden.LongReaderBind.Test/golden.ir | Golden IR updated to include primops/lifted bindings. |
| test/ps/output/Golden.LongCallbackChain.Test/golden.lua | Golden updated to reflect lifted ordIntImpl and arithmetic primops. |
| test/ps/output/Golden.LongCallbackChain.Test/golden.ir | Golden IR updated to include primops/lifted bindings. |
| test/ps/output/Golden.LongBindFlipped.Test/golden.ir | Golden IR updated to include string concat primop and arithmetic primops. |
| test/ps/output/Golden.LongApplyChain.Test/golden.ir | Golden IR updated to include string concat primop and refreshed temps. |
| test/ps/output/Golden.Issue37.Test/golden.lua | Golden refreshed (identifier renames/optimizer churn). |
| test/ps/output/Golden.Issue37.Test/golden.ir | Golden IR refreshed (identifier renames/optimizer churn). |
| test/ps/output/Golden.GenericEqTwoTypes.Test/golden.lua | Golden updated to reflect lifted boolean/equality foreigns. |
| test/ps/output/Golden.GenericEqTwoTypes.Test/golden.ir | Golden IR updated to include primops for boolean ops and lifted Eq. |
| test/ps/output/Golden.FloatIn.Test/golden.lua | Golden updated to remove foreign dispatch for basic arithmetic. |
| test/ps/output/Golden.FloatIn.Test/golden.ir | Golden IR updated to include primops in Semiring dictionaries. |
| test/ps/output/Golden.FieldCaching.Test/golden.lua | Golden updated for lifted Eq/Ord/Ring/Semiring foreigns and primop lowering. |
| test/ps/output/Golden.FieldCaching.Test/golden.ir | Golden IR updated to include primops/lifted bindings and simplified Eq check. |
| test/ps/output/Golden.Fibonacci.Test/golden.lua | Golden updated to use direct + and a lifted subtraction worker. |
| test/ps/output/Golden.Fibonacci.Test/golden.ir | Golden IR updated to include primops and lifted sub$w. |
| test/ps/output/Golden.DerivedFunctor.Test/golden.lua | Golden updated to remove foreign dispatch for arithmetic. |
| test/ps/output/Golden.DerivedFunctor.Test/golden.ir | Golden IR updated to include primops in Semiring dictionaries. |
| test/ps/output/Golden.CharLiterals.Test/golden.lua | Golden updated for lifted Eq/Ord foreigns (direct operators in compare/equality). |
| test/ps/output/Golden.CharLiterals.Test/golden.ir | Golden IR updated to include lifted comparisons and simplified equality. |
| test/ps/output/Golden.BugListGenericEq.Test/golden.lua | Golden updated to reflect lifted boolean/equality foreigns. |
| test/ps/output/Golden.BugListGenericEq.Test/golden.ir | Golden IR updated to include primops for boolean ops and lifted Eq. |
| test/ps/output/Golden.ArrayPatternMatch.Test/golden.lua | Golden updated to remove foreign dispatch for arithmetic dictionaries. |
| test/ps/output/Golden.ArrayPatternMatch.Test/golden.ir | Golden IR updated to include primops in Semiring/Ring dictionaries. |
| test/ps/output/Golden.ArrayOfUnits.Test/golden.lua | Golden updated to remove foreign dispatch for arithmetic dictionaries. |
| test/ps/output/Golden.ArrayOfUnits.Test/golden.ir | Golden IR updated to include primops and refreshed identifiers. |
| test/Main.hs | Registers the new ForeignLift test suite. |
| test/Language/PureScript/Backend/Lua/Golden/Spec.hs | Wires foreign lifting into the golden IR pipeline (mirrors compileModules). |
| test/Language/PureScript/Backend/Lua/ForeignLift/Spec.hs | New unit tests for lifting behavior and allowlist membership. |
| test/Language/PureScript/Backend/IR/Optimizer/Spec.hs | Adds unit tests for primop constant folding and its decline cases. |
| test/Language/PureScript/Backend/IR/Gen.hs | Extends IR generators to emit primops for property-based fuzzing. |
| pslua.cabal | Adds the new Lua.ForeignLift module to library and test-suite stanzas. |
| lib/Language/PureScript/Backend/Lua.hs | Lowers IR primops directly to Lua BinOp/UnOp operators. |
| lib/Language/PureScript/Backend/IR/Types.hs | Adds primop AST nodes + helpers, and integrates them into traversal/alphaEq. |
| lib/Language/PureScript/Backend/IR/Optimizer.hs | Implements Lua 5.1–semantics constant folding for primops. |
| lib/Language/PureScript/Backend.hs | Wires ForeignLift.liftForeigns into the main compilation pipeline. |
| exe/Main.hs | Adds ForeignLift error handling to the CLI entrypoint. |
Every `Ord` comparison and `/=` decays to `case … of Ctor -> false; _ -> true`, which compiles to `if p then False else True`; #178's lifted comparison bodies made this the dominant residual shape (spotted in review). The IR optimizer now folds it: - `if p then True else False` -> `p` - `if p then False else True` -> `not p` (a `PrimNot`, a node the IR only gained with #178's primops) - `if not p then a else b` -> `if p then b else a` (drop the negation and swap the branches; runs before the two literal rules so `if not p then False else True` normalises to `p` rather than stalling at `not (not p)`) - `not (not e)` -> `e` The Lua backend additionally emits `not (a == b)` as `a ~= b` (Lua's `~=` is exactly the negation of `==`), the form luacheck expects. Together these collapse e.g. the tailRecM loop guard from a nested IIFE plus a boolean round-trip to a single swapped-branch if (net -105 lines across 16 goldens). Every eval oracle is unchanged.
Unisay
marked this pull request as ready for review
July 8, 2026 13:57
- test/ps/src/Golden/Primops/Test.purs:9: reword the comment — arithmetic lifts to direct operators, while `n <= 0` still routes through the lifted compare (dispatch collapse is #180) (#210 (comment))
The Copilot-review reword in 2d1031c added two comment lines to Golden/Primops/Test.purs but did not regenerate the committed CoreFn, so every sourceSpan in corefn.json was two lines stale. Re-emit it from the current source. The structural goldens (golden.ir / golden.lua) carry no source line numbers, so they are unaffected.
The #178 codegen change moved Bench.BindChain's inner continuation closure onto LuaJIT's hot-count boundary: its function entry root-traces in roughly one process out of five and stays interpreted in the rest, a coin flip weighted by LuaJIT's entropy-seeded penalty PRNG. bench/ci generated each trace report twice and compared byte for byte, so that marginal spot made the two runs differ about a third of the time and flaked CI. trace_report.lua now runs the workload in N independent luajit subprocesses (default 9, BENCH_TRACE_TRIALS overrides) and reports only the abort sites and end states that every trial agrees on. Decisive spots (hot loops, blacklisted entries, reliably root-traced workers) are stable across every run, so the intersection keeps them; the boundary noise drops out. The canonical report is stable by construction, so bench/ci runs each trace report once instead of twice-and-compare. The FNEW census is a pure static function of the artifact, so it keeps its twice-and-compare byte-stability proof. Accept the bench goldens moved by #178. The headline wins: Bench.Fib and Bench.CurriedStep lose every "NYI: bytecode FNEW" trace abort (arithmetic is native primops now, no per-call closures) and neither blacklists anything. The FNEW census drops accordingly: Fib function-body FNEW 2 to 0, CurriedStep total FNEW 9 to 2.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #178. Builds on the foreign parser (#173) and case-of-known-constructor (#177); unblocks the dictionary-method inlining of #180.
What
Polymorphic hot code bottoms out in opaque curried foreigns (
intAdd,ordIntImpl,refEq,boolConj) that the IR optimizer cannot see through, because to the IR a foreign body is text. This lifts the pure return-tree subset of such a body, taken from the actual fork source the foreign parser (#173) already gives us, into IR primops the existing rewrites can specialize.IR.Types):PrimBinOp ann PrimOp a bandPrimNot ann a, with aPrimOpenum of the twelve binary operators. Each node is defined as the matching Lua operator, so lowering is the identity and lifting its inverse (Note [IR primops]). Equality stays the existingEqnode; the lifter maps==onto it and~=ontonot (a == b).Lua.fromIR): each primop lowers to the same-named LuaBinOp/UnOp.IR.Optimizer): fold primops over literal operands in Lua 5.1 semantics (Note [Folding primops follows Lua 5.1]). Integer+/-/*fold with a ±2^53 exactness guard, floats only to a finite result, integer modulo copies the divisor-signeda - floor(a/b)*b, concat is string-with-string, comparisons fold on two numbers, andand/or/notfold including the short-circuit identities.IR.Optimizer):if p then True else Falsebecomesp,if p then False else Truebecomesnot p,if not p then a else bbecomesif p then b else a(drop the negation, swap the branches), andnot (not e)becomese. EveryOrdcomparison and/=decays to theif … then False else Trueshape, so once the comparison bodies lift this is the dominant residual; thenot prewrite is only expressible because the primops above give the IR aPrimNot. The Lua backend then emitsnot (a == b)asa ~= b, the form luacheck expects.Lua.ForeignLift): translate the liftable subset of an export's parsed Lua AST. Curried single-parameter lambdas become nestedAbs,if/elseif/elsereturn trees becomeIfThenElse, operators become primops, and a reference resolves either to a parameter or to an inlined headerlocal(which is howordIntImpl = (unsafeCoerceImpl)and therefEqaliases lift). Loops, mutation, varargs, multi-parameter functions, table constructors, and calls leave the export opaque.intAdd/intMul/intSub,ordIntImpl/ordCharImpl,boolConj/boolDisj/boolNot, therefEqfamily,concatString). It is a hard contract: an allowlisted export that fails to lift, because a fork changed its shape, is a compile error rather than a silent performance cliff. An export not on the list stays opaque even when it would lift.Backend.compileModulesruns the lifter before the optimizer, and the golden harness'scompileCorefndoes the same, so the.irgoldens go through the real pipeline.Design points
Lifted exports become ordinary bindings, not foreigns. The first cut replaced the accessor in place inside
uberModuleForeigns, and it miscompiled. DCE's reachability graph only understands theForeignImport/ObjectPropshapes among the foreigns, so a lifted lambda has no vertex there, gets read as unreachable, and is dropped while every reference to it survives dangling (attempt to call field 'Data_Ring_intSub' (a nil value)). Moving the lifted export intouberModuleBindingsfixes it structurally: DCE handles it as a normal binding, and once the accessor is gone it prunes the export's name from the module'sForeignImport, since nothing reads it off theforeignimport anymore, so the source row drops out of the emitted FFI. When every export of a module lifts, itsForeignImportdisappears: see the Fibonacci diff, where theData_Semiring_foreign/Data_Ring_foreigntables are gone andintAddis a native+.Folding tracks Lua 5.1. The target is all-doubles, so the rules watch it: integer folds bail beyond ±2^53 (a folded literal has to reparse to the value the runtime computes), float folds stop at a finite result (Lua has no inf/nan numeric literal), modulo copies the formula whose sign follows the divisor, and concat is string-with-string only (Lua's number-to-string on concat is build-dependent). Comparisons fold on two numbers; strings and chars are left alone, since Lua orders strings by bytes while the IR literal is semantic
Text.Semantics from the source, not a registry. The original plan was a hand-written qualified-name to IR registry (the purs-backend-es approach). Deriving the semantics from the parsed fork source instead makes registry drift against the package set impossible by construction. That is the payoff of Lua's tiny grammar, and the reason the allowlist can be a hard contract instead of best-effort.
Verification
~=peephole, and for the lifter (each liftable shape, each declined shape, allowlist membership).PrimBinOp/PrimNot, so the optimizer's well-scopedness, GUC, free-reference, and idempotence properties fuzz constant folding over primops. Unlike Fold ReflectCtor and DataArgumentByIndex over known constructor applications (case-of-known-constructor) #177, widening the generator is in scope here, since folding correctness is what benefits from the fuzz.Golden.Primopsmodule exercises lifted arithmetic, comparison, equality, concat, and booleans, with a hand-verified eval oracle.tailRecMloop guard collapses from a nested IIFE plus a boolean round-trip to a single swapped-branch if.Notes for reviewers
Scope vs the 42x number. #178 gives you the primops and the lifter, and direct arithmetic collapses fully (the Fibonacci golden is native
+and subtraction now, no foreign dispatch). The generalgreaterThanOrEq(ordInt)(a)(b)tonot (a < b)collapse also needs dictionary-method inlining (#180), which this unblocks: theordIntImplbody is already lifted into primops insideordInt.compare, but theData_Ord_compare(dict)method dispatch stands until #180 inlines it.Twice-used primops can surface as a shared worker. A lifted primop used at two or more saturated sites can come out as an uncurried worker instead of an inline operator (Fibonacci's
sub$w(v, k)forn - 1andn - 2). The chain: purs sharessub ringIntas a top-level binding when-is used more than once,reduceObjectPropstrips theJust Alwaysoff the lifted lambda as it resolves through the dict projection (by design, so foreign accessors don't over-inline), and uncurry then splits the surviving curried binding into worker and wrapper. Correct, and still far cheaper than the old foreign dispatch of a table index plus two curried calls. Inlining trivial primop-bodied bindings regardless of use count would close the gap; it is a reasonable low-priority follow-up, filed as #211 and out of scope here.