Skip to content

fix(optimizer): canonicalize Effect/ST heads through top-level aliases (#297) - #298

Merged
Unisay merged 1 commit into
mainfrom
issue-297/canonicalize-through-toplevel-aliases
Jul 24, 2026
Merged

fix(optimizer): canonicalize Effect/ST heads through top-level aliases (#297)#298
Unisay merged 1 commit into
mainfrom
issue-297/canonicalize-through-toplevel-aliases

Conversation

@Unisay

@Unisay Unisay commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

Fixes #297.

What was broken

A module that instantiates discard with two different Bind dictionaries — Effect or ST plus any second monad; the repro uses ST + Effect, and Effect + Maybe produces the same shape — miscompiled: purs CSE floats the shared partial application in two stages, a shape neither canonicalization tier of #182 matched, so the Effect/ST chains stayed nested past magic-do, and once post-magic-do call-site inlining exposed the canonical pair, tier 2 manufactured a reference to a foreign accessor binding DCE had already removed. The generated Lua read a never-assigned module-table field and crashed:

local M = {}
...
return M.Effect_bindE(Effect_Console_log("st:"))(...)()  -- nil call at runtime

The fix

Tier 2 resolves head positions through top-level aliases (canonicalizeEffectAppInModule). A reference denotes a row's name if it is that name, or if the top-level binding it names has a right-hand side that denotes it (visited-bounded). This is closed under any CSE split of the structurally bounded spines (bind/pure: two nodes, discard: three), not just the observed two-stage float. Because tier 2 runs in the pre-magic-do fixpoints, the chains flatten again — the repro's stCount now compiles to a flat thunk:

local Golden_MixedDiscardFloat_Test_stCount = Control_Monad_ST_Internal_foreign.run(function(  )
  local r = Control_Monad_ST_Internal_foreign.new(1)()
  local _ = Control_Monad_ST_Internal_foreign.map_(function()
    return Data_Unit_unit
  end)(Control_Monad_ST_Internal_foreign.write(2)(r))()
  return Control_Monad_ST_Internal_foreign.read(r)()
end)

Tier 2 declines a rewrite whose produced reference would dangle: the rule now consults the module's top-level bindings (CanonEnv, built per optimizeModule run from the Standalone bindings plus the foreigns) and leaves the dictionary application in place when the canonical accessor is gone — sound, since the application it leaves is executable.

The compiler refuses to emit code for a non-closed module (lintDanglingImports, new ResultDanglingImports failure): any future resurrect-after-DCE fails the build instead of shipping Lua that nil-calls at runtime. The check runs unconditionally in compileModules and on every golden module in the harness. It is deliberately not a pass-boundary invariant: unit fixtures legitimately reference external names, so closedness is only meaningful for linked modules.

Tests

  • Golden.MixedDiscardFloat.Test — the minimal reproduction (ST discard + Effect discard in one module); its hand-written eval oracle crashed at baseline and passes now, and the goldens pin the flattened output.
  • Optimizer.Spec: the canonicalization block now runs against a CanonEnv (the rule's new contract), plus two new cases — resolution through the two-stage float, and declining when the canonical target has no binding.
  • Linter.Spec: lintDanglingImports coverage (binding/foreign resolution, Prim exemption, per-site reporting).

cabal test all: 1055 examples, 0 failures; no existing golden moved.

When one module discards over both ST and Effect, purs CSE floats the
shared partial application in two stages (discard = discard discardUnit;
discard1 = discard bindST), a shape neither canonicalization tier
matched: chains stayed nested past magic-do, and once call-site inlining
exposed the canonical pair, tier 2 manufactured a reference to a foreign
accessor DCE had already removed — a nil call at runtime.

Tier 2 now resolves head positions through the module's top-level
bindings before matching the table, and declines a rewrite whose
produced reference the module can no longer resolve. The compiler
additionally refuses to emit code for an optimized module with dangling
imported references (lintDanglingImports); the golden harness applies
the same closedness check to every golden module.

Fixes #297
@Unisay
Unisay force-pushed the issue-297/canonicalize-through-toplevel-aliases branch from d9bd03b to 74c4fd6 Compare July 24, 2026 17:35
@Unisay
Unisay merged commit f59f1a8 into main Jul 24, 2026
2 checks passed
@Unisay
Unisay deleted the issue-297/canonicalize-through-toplevel-aliases branch July 24, 2026 17:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Mixed ST/Effect discard chains miscompile: canonicalization resurrects a DCE'd foreign accessor (runtime nil call)

1 participant