feat(codegen): lower Effect/ST loop combinators to native Lua loops (#233) - #299
Merged
Conversation
A run of foreachE/forE/whileE (and the ST twins foreach/for/while) previously compiled to a foreign higher-order combinator handed a closure it called once per iteration. The codegen now recognises the saturated application run by magic-do — by qualified name, in both the imported-reference and dissolved-accessor head forms — and emits the native for/while loop instead, inlining a literal body lambda as the loop body with its parameter as the loop variable. Non-atomic arguments pre-bind to block-scoped locals so the loop never re-evaluates them, preserving the foreign call's once-per-argument evaluation order; a thunk body too large to splice under Lua's active-locals cap keeps its per-iteration call. A same-named combinator outside Effect/Control.Monad.ST.Internal, or an unapplied first-class combinator, keeps the ordinary call — pinned by the NativeLoopsGuard eval golden, whose deliberately one-step fakes would print differently if a name-only match ever fired. Closes #233
The lowering matches a combinator head as a direct imported reference or a dissolved foreign-accessor read, but resolves no top-level alias hop — magic-do's chain-head recognition resolves one. On ordinary code the optimizer dissolves such an alias before codegen, so the gap is unreachable; an inline-never directive pins the alias undissolved and exposes it. Golden.NativeLoopsAliasPin pins the resulting UNLOWERED shape. It is a missed optimization, never a miscompile: the foreign combinator is what stays, and the eval oracle holds across either shape, so only golden.lua discriminates. When recognition moves into the IR (#239 needs the loop visible to the optimizer), this golden should flip to a native for loop — that flip proves the move closed the gap.
This was referenced Jul 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #233. Built on top of the #297 fix (#298), which the ST golden of this PR exposed at baseline.
What it does
A run of an Effect/ST loop combinator —
foreachE/forE/whileEand the ST twinsControl.Monad.ST.Internal.foreach/for/while— previously stayed a call into a foreign higher-order combinator handed a Lua closure it called once per iteration: one closure allocation and one call per step, on top of the combinator's own call. The codegen (Language.PureScript.Backend.Lua.NativeLoop) now recognises the saturated application run by magic-do and emits the native loop the foreign implementation ran, inlining a literal body lambda as the loop body with its parameter as the loop variable. Before:after:
and a
whileEwith a non-thunk condition pre-binds it once and calls it per iteration, with the body statements inlined flat:Design points
EffectRunArg-terminated spine, in both head forms a foreign takes at codegen time (imported reference, dissolved accessor read). A partial application or a first-class combinator value keeps the foreign call — only a run lowers.$-namespace locals in application order (the loop never re-evaluates the array or a pre-built body function), and each pre-binding run is scopeddo … endwith its loop so temporaries do not accumulate againstLUAI_MAXVARSacross a magic-do chunk's ~150 statements.Returns rewritten to evaluation statements viastatementize); a body declaring more than 40 block-level locals keeps the per-iteration thunk call — the cost the foreign implementation had — so a spliced magic-do chunk cannot overflow Lua's active-locals cap.fromIR, so aforeachEin the tail of aforEbody becomes a nested nativefor.local _ = <run>) emits the bare loop; a named binder (x <- foreachE …,x :: Unit) declareslocal xafter the loop — the run yields no values, so the binder reads nil exactly as it did from the foreign call.Tests
Golden.NativeLoops.Test(Effect): literal and non-literal bodies,whileEover aRefcounter, a nestedforE/foreachEpair in tail position; hand-written eval oracle pins iteration count, order, and the half-openforEbound.Golden.NativeLoopsST.Test(ST):for/foreach/whileinsideST.run, eval oracle55/10/5.Golden.NativeLoopsGuard.Test— the soundness guard: same-named user combinators (each deliberately performing at most one step) stay ordinary calls; a name-only match would iterate for real and fail the eval golden.The eval oracles were written and verified against the FFI implementations before the lowering landed, and pass unchanged against the lowered output — the semantic proof the issue asks for.
cabal test all: 1067 examples, 0 failures, twice (second run with a fixed alternate seed); no pre-existing golden moved.Known limitation, pinned
Recognition resolves no top-level alias hop, where magic-do's chain-head recognition resolves one. On ordinary code the optimizer dissolves a bare-
Refalias to a combinator before code generation, so the gap is unreachable; aninline neverdirective pinning such an alias undissolved leaves the foreign call in place. Measured on the live compiler:myFor = forE, two call sitesEffect.forE alwaysGolden.NativeLoopsAliasPin.Test.myFor neverThe automatic paths do not reach it: purs CSE floats only synthesized dictionary applications (
forEis a plain foreign import, not a dictionary — unlike thediscardfloat of #297), andshareForeignAccessorsproduces the direct-reference form, which is handled. It is a missed optimization, never a miscompile — the foreign implementation is what stays — so no eval oracle can catch it and onlygolden.luadiscriminates.Golden.NativeLoopsAliasPinpins that unlowered shape as a canary rather than papering over it with a resolver here: recognition is due to move into the IR as the opening step of #239, which needs the loop visible to the optimizer, and the alias question disappears once a lift keys off the foreign import itself. That golden should flip to a nativeforwhen the move lands — the flip is the regression test that it closed the gap. Full design record: #233 (comment)