Skip to content

feat(codegen): lower Effect/ST loop combinators to native Lua loops (#233) - #299

Merged
Unisay merged 2 commits into
mainfrom
issue-233/native-loop-combinators
Jul 25, 2026
Merged

feat(codegen): lower Effect/ST loop combinators to native Lua loops (#233)#299
Unisay merged 2 commits into
mainfrom
issue-233/native-loop-combinators

Conversation

@Unisay

@Unisay Unisay commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

Closes #233. Built on top of the #297 fix (#298), which the ST golden of this PR exposed at baseline.

What it does

A run of an Effect/ST loop combinator — foreachE/forE/whileE and the ST twins Control.Monad.ST.Internal.foreach/for/while — previously stayed a call into a foreign higher-order combinator handed a Lua closure it called once per iteration: one closure allocation and one call per step, on top of the combinator's own call. The codegen (Language.PureScript.Backend.Lua.NativeLoop) now recognises the saturated application run by magic-do and emits the native loop the foreign implementation ran, inlining a literal body lambda as the loop body with its parameter as the loop variable. Before:

local _ = Effect_forE(1)(4)(Golden_NativeLoops_Test_logShow)()
local _ = Effect_foreachE({ [1] = 10, [2] = 20, [3] = 30 })(function(n_S_1)
  return Effect_Ref_modify__S_w(function(v_S_2) return v_S_2 + n_S_1 end, sum_S_0)
end)()

after:

do
  local _S_f2 = Golden_NativeLoops_Test_logShow
  for _S_i3 = 1, 3 do _S_f2(_S_i3)() end
end
do
  local _S_xs4 = { [1] = 10, [2] = 20, [3] = 30 }
  for _S_i5 = 1, #(_S_xs4) do
    local n_S_1 = _S_xs4[_S_i5]
    Effect_Ref_modify__S_w(function(v_S_2) return v_S_2 + n_S_1 end, sum_S_0)()
  end
end

and a whileE with a non-thunk condition pre-binds it once and calls it per iteration, with the body statements inlined flat:

do
  local _S_cond6 = Effect_functorEffect.map(function(v0_S_5) return v0_S_5 < 3 end)(Effect_Ref_read(counter_S_4))
  while _S_cond6() do
    local n0_S_6 = Effect_Ref_read(counter_S_4)()
    local _ = Effect_Console_log(Data_Show_showIntImpl(n0_S_6))()
    Effect_Ref_modify__S_w(function(v1_S_7) return v1_S_7 + 1 end, counter_S_4)()
  end
end

Design points

  • Recognition is by qualified name on the EffectRunArg-terminated spine, in both head forms a foreign takes at codegen time (imported reference, dissolved accessor read). A partial application or a first-class combinator value keeps the foreign call — only a run lowers.
  • Evaluation order and sharing are preserved: non-atomic arguments pre-bind to fresh $-namespace locals in application order (the loop never re-evaluates the array or a pre-built body function), and each pre-binding run is scoped do … end with its loop so temporaries do not accumulate against LUAI_MAXVARS across a magic-do chunk's ~150 statements.
  • Body inlining is budgeted: a literal thunk body splices as statements (tail Returns rewritten to evaluation statements via statementize); a body declaring more than 40 block-level locals keeps the per-iteration thunk call — the cost the foreign implementation had — so a spliced magic-do chunk cannot overflow Lua's active-locals cap.
  • Nested loops lower recursively: the body-run compilation feeds back through fromIR, so a foreachE in the tail of a forE body becomes a nested native for.
  • The statement position (local _ = <run>) emits the bare loop; a named binder (x <- foreachE …, x :: Unit) declares local x after the loop — the run yields no values, so the binder reads nil exactly as it did from the foreign call.

Tests

  • Golden.NativeLoops.Test (Effect): literal and non-literal bodies, whileE over a Ref counter, a nested forE/foreachE pair in tail position; hand-written eval oracle pins iteration count, order, and the half-open forE bound.
  • Golden.NativeLoopsST.Test (ST): for/foreach/while inside ST.run, eval oracle 55/10/5.
  • Golden.NativeLoopsGuard.Test — the soundness guard: same-named user combinators (each deliberately performing at most one step) stay ordinary calls; a name-only match would iterate for real and fail the eval golden.

The eval oracles were written and verified against the FFI implementations before the lowering landed, and pass unchanged against the lowered output — the semantic proof the issue asks for. cabal test all: 1067 examples, 0 failures, twice (second run with a fixed alternate seed); no pre-existing golden moved.

Known limitation, pinned

Recognition resolves no top-level alias hop, where magic-do's chain-head recognition resolves one. On ordinary code the optimizer dissolves a bare-Ref alias to a combinator before code generation, so the gap is unreachable; an inline never directive pinning such an alias undissolved leaves the foreign call in place. Measured on the live compiler:

scenario result
bare alias myFor = forE, two call sites optimizer dissolves the alias → lowers
Effect.forE always lowers
Golden.NativeLoopsAliasPin.Test.myFor never alias survives → does not lower

The automatic paths do not reach it: purs CSE floats only synthesized dictionary applications (forE is a plain foreign import, not a dictionary — unlike the discard float of #297), and shareForeignAccessors produces the direct-reference form, which is handled. It is a missed optimization, never a miscompile — the foreign implementation is what stays — so no eval oracle can catch it and only golden.lua discriminates.

Golden.NativeLoopsAliasPin pins that unlowered shape as a canary rather than papering over it with a resolver here: recognition is due to move into the IR as the opening step of #239, which needs the loop visible to the optimizer, and the alias question disappears once a lift keys off the foreign import itself. That golden should flip to a native for when the move lands — the flip is the regression test that it closed the gap. Full design record: #233 (comment)

Unisay added 2 commits July 24, 2026 20:06
A run of foreachE/forE/whileE (and the ST twins foreach/for/while)
previously compiled to a foreign higher-order combinator handed a
closure it called once per iteration. The codegen now recognises the
saturated application run by magic-do — by qualified name, in both the
imported-reference and dissolved-accessor head forms — and emits the
native for/while loop instead, inlining a literal body lambda as the
loop body with its parameter as the loop variable.

Non-atomic arguments pre-bind to block-scoped locals so the loop never
re-evaluates them, preserving the foreign call's once-per-argument
evaluation order; a thunk body too large to splice under Lua's
active-locals cap keeps its per-iteration call. A same-named combinator
outside Effect/Control.Monad.ST.Internal, or an unapplied first-class
combinator, keeps the ordinary call — pinned by the NativeLoopsGuard
eval golden, whose deliberately one-step fakes would print differently
if a name-only match ever fired.

Closes #233
The lowering matches a combinator head as a direct imported reference or
a dissolved foreign-accessor read, but resolves no top-level alias hop —
magic-do's chain-head recognition resolves one. On ordinary code the
optimizer dissolves such an alias before codegen, so the gap is
unreachable; an inline-never directive pins the alias undissolved and
exposes it.

Golden.NativeLoopsAliasPin pins the resulting UNLOWERED shape. It is a
missed optimization, never a miscompile: the foreign combinator is what
stays, and the eval oracle holds across either shape, so only golden.lua
discriminates. When recognition moves into the IR (#239 needs the loop
visible to the optimizer), this golden should flip to a native for loop
— that flip proves the move closed the gap.
@Unisay
Unisay merged commit 07894d6 into main Jul 25, 2026
2 checks passed
@Unisay
Unisay deleted the issue-233/native-loop-combinators branch July 25, 2026 11:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Lower ST/Effect loop combinators (foreachE/forE/whileE) to native Lua for/while

1 participant