Lua 5.4 in pure Java — no GC of its own; the JVM does the collecting.
Run existing Lua projects on the JVM without changing a single line of their Lua code.
This project is a module-by-module port of the Lua 5.4 reference implementation
(lua-5.4.x C sources) to Java. It is not a Lua-inspired language that merely looks
similar — it is Lua 5.4, with the memory-management layer removed and handed to the
host runtime.
| Target | Full Lua 5.4 language semantics + complete standard library |
| Method | Direct, module-by-module translation of the C reference sources |
| GC | Not implemented. Object lifetime is entirely the JVM's job (G1/ZGC are concurrent collectors already) |
| Runtime | Java 17+ (coroutines are plain stacks, no threads — see below) |
| Compatibility | Pure-Lua projects; C extension modules (.so / luaopen_*) are out of scope for now |
lgc.c is one of the most intricate parts of the C implementation: tri-colour
incremental marking, generational mode, write barriers, finalizers, weak tables.
All of it exists to do what the JVM already does — concurrently, and better tuned
than a hand-rolled collector written in Java would be. The file is deleted
outright. Only the semantics that depend on collection happening are rebuilt on
JVM primitives:
| Lua concept | C mechanism | This implementation |
|---|---|---|
| Object lifetime | lgc.c tri-colour marking |
JVM GC (G1 / ZGC) |
Weak tables (__mode = "k"/"v"/"kv") |
weak links swept in lgc.c |
WeakReference + ReferenceQueue |
__gc finalizers |
luaC_checkfinalizer / GCTM |
java.lang.ref.Cleaner |
| Short-string interning | lstring.c table pinned against collection |
weak intern pool; entries die when unreferenced |
Everything else — lexer, parser, code generator, VM, metamethods, coroutines, standard library — is a direct port.
The package layout mirrors the C sources one-to-one, so a reader holding
lua-5.4.x side by side can find their way around:
| Lua 5.4 C source | Java |
|---|---|
lobject.h / lobject.c |
LuaValue, LuaString, LuaTable, Proto, Closure, Upvalue |
lstring.c |
StringPool |
ltable.c |
LuaTable |
lfunc.c |
Proto, Closure, Upvalue |
lstate.c |
LuaState, CallInfo, GlobalState |
lzio.c |
Zio |
llex.c |
Lexer |
lparser.c |
Parser (hand-written recursive descent, as upstream) |
lcode.c / lopcodes.h |
FuncState, Codegen, OpCode |
lvm.c |
Interpreter |
ldo.c |
call frames, protected calls, error propagation |
ltm.c |
Meta (metamethod dispatch) |
ldebug.c |
Debug, error messages, stack traces |
lapi.c / lauxlib.c |
operations on LuaState, AuxLib |
lundump.c / ldump.c |
bytecode load / dump |
lbaselib.c, lcorolib.c, loadlib.c, lstrlib.c, ltablib.c, lmathlib.c, liolib.c, loslib.c, lutf8lib.c, ldblib.c |
corresponding lib.* classes |
lgc.c |
absent — see above |
The auxiliary layer (lauxlib.c and friends) comes along, because the standard
library is written against it.
These are the places where translating C to Java is not mechanical, and where a wrong call costs correctness that is expensive to recover later.
A Lua string may contain \0; #s counts bytes; s:byte(i) returns a byte.
Java String is UTF-16 and rejects lone surrogates. Any real-world Lua project
that touches binary data, UTF-8 text, or embedded \0 will break if strings are
backed by String. Lua strings are therefore backed by byte[], and every string
library function is implemented byte-wise.
Lua 5.4 distinguishes 1 (integer) from 1.0 (float): they compare equal, but
math.type differs, // differs from /, 1/2 is 0.5 while 1//2 is 0,
and coercion to string yields 1 vs 1.0. Values are a mutable slot type
(a type tag plus long, double and an Object payload) mirroring C's TValue
— this avoids boxing and lets the VM write stack slots in place.
Ported from ltable.c, including the migration logic that decides whether an
integer key is dense enough to live in the array part. Integer keys and string keys
are distinct entries: t[1] and t["1"] are different, as in Lua.
Lua coroutines can yield from arbitrarily deep stacks — inside pcall, inside a
metamethod, inside a for iterator. The plan was to give each one a Java virtual
thread. That turned out to be the wrong call, and phase 5 dropped it.
A Lua coroutine is cooperative: exactly one runs at a time, and the suspension
point is stated explicitly by coroutine.yield. C models this with one stack per
lua_State plus longjmp out to lua_resume. Java has the same two ingredients
— one stack per LuaState, and an exception (LuaYield) that unwinds the Java
call stack. Nothing needs a thread, because all the Lua state lives on the
heap: unwinding the Java frames leaves the stack, the CallInfo chain and the
open upvalues exactly as they were, ready to continue on resume.
Virtual threads were also too permissive. A C function that calls back into Lua
(table.sort's comparator, string.gsub's replacement function) cannot yield in
real Lua — it raises attempt to yield across a C-call boundary. On a virtual
thread it would simply work, and that difference is a bug, not a feature, for a
project whose goal is "run existing Lua code unchanged."
This also drops the Java floor from 21 to 17, which is what makes the local build work today.
The machinery that replaced threads is a faithful port of ldo.c:
lua_resume / unroll / finishCcall / finishpcallk / precover, plus
luaV_finishOp, which completes the half-executed instruction that a yield
interrupted. C functions that can be interrupted (pcall, xpcall) register a
continuation — the Java equivalent of lua_KFunction.
lua_error becomes a thrown error carrying the Lua error value. pcall / xpcall
catch it, as does the coroutine boundary. Care is needed so that non-string error
values, error objects carrying __tostring, and errors crossing a coroutine
boundary all behave as upstream does.
The Lua reference parser is hand-written recursive descent. Porting it directly removes the build-time code-generation step, along with the generated-parser drift problem the previous architecture had.
__index, __newindex, __call, __concat, __len, __eq, __lt, __le,
__unm, __add … __idiv, __close, __gc, __mode, __name, __metatable,
__pairs, __ipairs. Including the parts that are easy to get wrong: __le
degrading to not (b < a) when only __lt exists; __eq firing only for
comparable types; __len on tables ignoring non-integer results; and __index
chain traversal.
Each phase ends with tests green. Nothing advances on a red phase.
| Phase | Content | Upstream reference |
|---|---|---|
| 0 | Foundations: LuaValue, LuaString + interning, LuaTable, LuaState, error type, Zio |
lobject, lstring, ltable, lstate, lzio |
| 1 | Lexer + recursive-descent parser; emits directly as upstream does, with no separate tree | llex, lparser |
| 2 | Bytecode: full Lua 5.4 instruction set, code generator, constant folding, upvalue resolution | lopcodes, lcode, lfunc |
| 3 | Interpreter main loop: calls/returns, arithmetic & comparison with coercion, metamethod dispatch | lvm, ldo, ltm |
| 4 | Closures & open/closed upvalues, goto/labels, varargs, multiple assignment and returns |
lparser, lvm |
| 5 | Coroutines: coroutine.*, yield/resume/unroll, yield & error across boundaries |
lcorolib, ldo, lvm |
| 6 | Collection-dependent semantics: weak tables, __gc, weak string pool |
(replaces lgc.c) |
| 7 | Standard library: base, coroutine, package, string, table, math, io, os, utf8, debug |
l*lib.c, loadlib.c |
| 8 | load / loadstring / dump, require searchers, package.path / cpath |
lundump, ldump, loadlib |
| 9 | Conformance: upstream test suite + differential testing against a real lua5.4 binary |
— |
A rewrite that merely looks like Lua is worthless. Four gates, in increasing order of strength:
- Upstream test suite. The
lua-5.4.x/tests/*.luafiles (alls.lua,big.lua,strings.lua,coroutine.lua,pm.lua, …) run unmodified. This is the primary gate — upstream's own suite is far more adversarial than anything written by hand here. - Differential testing. The same script runs under this implementation and
under a real
lua5.4binary; stdout must match byte for byte. Wired into CI withlua5.4installed. - Differential fuzzing. Generated programs compared against the reference binary — this is what catches the "no crash but wrong answer" class.
- Coverage gate (JaCoCo line ratio).
A phase is done only when the gates that apply to it are green.
Phase 10 — the upstream lua-5.4.x/tests suite runs green end to end. 19
files whose output is deterministic (api, bitwise, calls, closure,
code, coroutine, db, errors, events, gengc, goto, nextvar, pm,
strings, tpack, utf8, vararg, verybig, …) match a real lua5.4
byte for byte. Five more (cstack, gc, literals, locals,
constructs) print things that vary with the implementation or with a random
seed, so they are only asserted to run to completion and print OK — the reason
for each is listed under Phase 10 below. 874 tests green, JaCoCo line coverage
0.96 against a 0.95 gate. (The upstream-suite gate needs LUA54_TESTS or
target/lua54-tests; without a test directory that whole class skips itself,
so a green run on a machine without it is 25 tests weaker than it looks.)
Phase 9 — complete, including debug.sethook. The previous fakescript
language implementation has been removed from master; see Migration below.
Phases 0 through 9 are in: the compiler produces bytecode that is
field-for-field identical to a real lua5.4's, and the VM executes it —
stdout matches a real lua5.4 byte for byte across 158 + 121 + 60 + 32 + 25
- 16 + 21 + 36 differential scripts and 2100 generated programs, and error
messages match line for line.
string.dump's output is additionally compared as raw bytes, not just as behaviour. All ten standard libraries are in (base, coroutine, package, string, utf8, table, math, os, io, debug),debugincluded in full — hooks and all.
Phase 6 is the one phase where "no GC of our own" costs real ground. Weak tables,
__gc and the string pool are all rebuilt on JVM reachability, which is not the
same thing as Lua's mark-and-sweep; three places where it cannot be made to match
are listed below under Known divergences.
Landed so far:
| Piece | Upstream | Notes |
|---|---|---|
Lua type tags |
lua.h, lobject.h |
integer/float subtypes kept as distinct variants |
LuaValue |
TValue |
mutable slot, so the VM can write stack slots in place |
LuaString |
TString |
byte[]-backed; \0, arbitrary and invalid-UTF-8 bytes all preserved |
StringPool |
lstring.c strt |
interned weakly, so unused strings do not pin the heap |
LuaTable |
ltable.c |
array part + hash part, rehash, #t borders, integral-float key normalisation |
LuaError |
lua_error |
carries a Lua value; only this type is caught by pcall |
Zio |
lzio.c |
byte input stream; EOZ is -1, bytes are unsigned |
LCType |
lctype.c |
the luai_ctype_ table copied byte for byte, so lislalpha treats _ as a letter exactly as upstream does |
Numeral |
lobject.c luaO_str2num |
int-first-then-float; strict decimal syntax, so 1d/inf/nan are rejected the way C's strtod rejects them |
Lexer |
llex.c |
full token set, all escapes incl. \z / \xXX / \u{XXX} / \ddd, long strings & comments, byte-exact error messages |
OpCode |
lopcodes.h/.c |
all 83 opcodes; order, names and luaP_opmodes verified mechanically against upstream headers |
Proto |
lobject.h, lfunc.c |
code / constants / nested protos / upvalues / locals / line info |
Arith |
lobject.c, lvm.c |
the arithmetic used by constant folding — same code path as the VM will use |
ExpDesc, FuncState, DynData |
lparser.h |
expdesc's union flattened; block/goto/variable bookkeeping |
Codegen |
lcode.c |
instruction emission, register allocation, jumps & patching, constant folding, NEWTABLE/SETLIST |
Parser |
lparser.c |
hand-written recursive descent, emitting code as it parses |
UpVal |
UpVal |
two-state (open = stack slot reference, closed = private copy) |
LClosure |
LClosure |
proto + upvalue array |
CallInfo |
CallInfo |
frame record; CIST_FRESH decides where a Lua→Lua return stops |
LuaState |
ldo.c, lstate.c |
stack, call/return, pcall, upvalue open/close, all luaG_* errors |
LuaTm |
ltm.c |
metamethod lookup & dispatch, __le fallback, to-be-closed |
Vm |
lvm.c |
the interpreter main loop; all 83 opcodes |
BaseLib, MathLib |
lbaselib.c, lmathlib.c, part of lstrlib.c |
minimal but differentially-verified |
Diff (test) |
— | shared differential driver: runs one script on both sides |
ConformanceTest, CodegenTest, ParserTest, RuntimeTest |
— | run a real lua5.4 binary as the oracle |
ClosureTest (test) |
— | Phase 4 gate: closures, goto, varargs, multiple assignment |
CClosure |
CClosure |
a C function bound to values — needed by coroutine.wrap |
LuaThread value (Lua.VTHREAD) |
LUA_TTHREAD |
a LuaState reachable as a Lua value |
CoroutineLib |
lcorolib.c |
create / resume / yield / status / wrap / isyieldable / running / close |
resume / unroll / finishCcall / finishpcallk / precover |
ldo.c |
the yield/resume state machine, no threads |
LuaKFunction |
lua_KFunction |
continuation for a C function interrupted by a yield |
LuaYield |
luaD_throw(L, LUA_YIELD) |
unwinding exception; deliberately not caught by pcall |
Vm.finishOp |
luaV_finishOp |
completes the half-executed instruction a yield interrupted |
CoroutineTest (test) |
— | Phase 5 gate: 60 differential scripts, incl. yield inside every metamethod |
FuzzTest (test) |
— | 1500 seeded programs + 600 more run inside a coroutine with random yields |
LuaTable.WeakRef |
iscleared in lgc.c |
a weak payload is the stored form; keeps the identity hash, since identityHashCode(null) is 0 and would collapse every dead key onto bucket 0 |
LuaTable.purge / rewrap |
cleartable / traverseweakvalue |
dead entries are dropped at the end of a collection round, and re-wrapped when __mode changes |
LuaState.clearDeadStackSlices |
traversethread's atomic branch |
nils [top, stack_last) on every thread, inside collectgarbage |
GcSentinel |
GCTM in lgc.c |
sentinel whose finalize resurrects the table onto a pending queue; drained on the Lua thread |
LuaState.close |
lua_close → callallpendingfinalizers |
runs __gc for every object that still has one, at exit |
WeakTableTest (test) |
— | Phase 6 gate: 20 differential scripts, split into a collection-dependent half and an always-true half |
GcTest (test) |
— | Phase 6 gate: 12 scripts — once-only, errors swallowed, late __gc ignored |
LuaUserdata |
Udata |
payload + per-object metatable; luaL_testudata needs the latter |
Undump |
lundump.c |
reads a precompiled chunk; header check, varint sizes, the binary string chunk-name rule |
Dump |
ldump.c |
writes one; string.dump, strip |
LoadTest (test) |
— | Phase 8 gate: 25 scripts — modes, chunk names, _ENV, reader functions, loadfile / dofile |
DumpTest (test) |
— | Phase 8 gate: 16 — runtime stdout and string.dump output byte for byte |
StringPack |
part of lstrlib.c |
string.pack / unpack / packsize; the only platform-dependent corner of the library |
PackTest (test) |
— | 21 scripts — integers, floats, strings, alignment, wide integers, errors, key set |
DebugLib |
ldblib.c + part of ldebug.c / ldo.c / lauxlib.c |
getinfo / locals / upvalues / traceback / registry / metatables / user values / setcstacklimit / sethook / gethook |
LuaHook |
lua.h lua_Hook |
the hook callback; DebugLib's hookf is the one sethook installs |
LuaLibs |
linit.c |
luaL_openlibs; the ten libraries in loadedlibs order |
DebugTest (test) |
— | Phase 9 gate: 36 scripts — levels, options, locals, upvalues, traceback, deep stacks, and 14 hook scripts |
LuaLibsTest (test) |
— | openAll really opens all ten, and they are the same tables the globals see |
The compiler is validated by differential comparison, not by self-declared
expectations. src/test/resources/fkdump.c is a small dumper compiled against
the reference lua5.4 sources; for every script it prints every instruction with
every field (a, b, c, k, Bx, sBx, Ax, sB, sC, sJ), plus the
line, the constant table and the upvalue table, recursively for nested functions.
CodegenTest compares that against ProtoDump's rendering of our Proto,
requiring the two to be identical.
Two corpora: ~50 hand-written scripts covering every construct, and the 33 files
of the upstream lua-5.4.7 test suite (target/lua54-tests, fetched in CI),
compiled but not run. All of them match.
Error messages are gated the same way: ParserTest writes broken sources to disk,
runs them through a real lua5.4, and compares the message byte for byte —
'end' expected (to close 'if' at line 1), label 'a' already defined on line 1,
<goto l> at line 1 jumps into the scope of local 'x', and so on. The lexer half
was already gated like this: a table of deliberately-broken sources is compared
verbatim, including the near '<token>' part whose quoting rules differ between
<eof> and everything else.
Two decisions in this phase were settled by running the reference implementation rather than by reading the manual:
- an integral float used as a key is normalised to an integer on write
(
t[1.0]yieldsmath.type(next(t)) == "integer"and#t == 1); - writing a NaN key raises
table index is NaN, while reading one silently returnsnil.
Both are locked by tests that run against real Lua 5.4 in ConformanceTest.
Each of these looks like a bug and is not; all three are locked by tests.
>>is a logical shift.intop(>>,x,-y)casts tolua_Unsignedfirst, so-8 >> 2is4611686018427387902, not-2.- Upstream's constant table contains duplicates.
ls->h— the "constant → index" cache — is shared by the whole chunk, while each function numbers its own constants from 0. A nested function overwrites cached indices with its own small ones; back in the outer function the index no longer points at the right constant, so the entry is rejected and a duplicate is appended. - Integral floats get a perturbed cache key. Lua compares table keys with
==, so100000and100000.0are the same key; withoutluaK_numberK'sr + r*2^-52trick they would evict each other from the cache and a constant would be re-created on every use.
RuntimeTest is the Phase 3 gate. Every case is a script that runs twice —
once under this implementation, once under a real lua5.4 — and the two stdout
streams must be equal. Error cases are gated on the message line, including the
(local 'x') / (constant '10') / (method 'nope') suffixes.
The stack is modelled with one invariant that everything else depends on:
A stack slot
LuaValueis never replaced.LuaStatealways doesstack[i].set(v), neverstack[i] = v.
That single rule is what makes "an open upvalue holds a reference to the stack slot" mean the same thing in Java as "an open upvalue holds a pointer to the stack slot" means in C. Break it once and closures silently stop seeing updates.
Running it today:
LuaState L = new LuaState();
BaseLib.open(L);
MathLib.open(L);
L.run("print('hello from Lua')");Each of these was guessed wrong from the manual or from intuition first; all six are locked by differential tests.
- Arithmetic does not coerce strings, and
"10" + 1 == 11does not come from the VM.lvm.c's arithmetic macros usetonumberns/tointegerns, which ignore strings. The coercion lives inlstrlib.c, which installs__add(and friends) on the string metatable — firstlua_stringtonumberboth sides, thenlua_arith. Skip those metamethods and you get a plausible but wrongattempt to perform arithmetic on a string value. - A C function's
nreturn values are the topnpushed values.ipairsauxpushesi, then pushest[i], and returns1whent[i]is nil — so the value seen by the genericforis that trailing nil, noti. Written the intuitive way (return i),for _, v in ipairs(t)never terminates, because the loop only stops when the first result is nil. - Every arithmetic opcode is followed by an
OP_MMBIN*. On success the VM doespc++to skip it; on failure it falls through and the metamethod gets dispatched. Miss onepc++and a working expression runs its metamethod too. OP_NEWTABLE'sEXTRAARGis unconditional (becausek == 0⟺Ax == 0), whileOP_SETLIST's is conditional. Reading it conditionally producesmalformed opcode EXTRAARGon every table constructor.- C's
%gstrips trailing zeros; Java's does not.1.5printed as1.5000000000000untilformatGwas written by hand. luaG_runerrorandluaL_errortake their position differently. The former usesisLua(ci)on the current frame; the latter isluaL_where(L, 1)and walks back one level to the caller. A library function that uses the former reports nosrc:line:prefix at all, because its own frame is a C frame.
FuzzTest generates 2100 programs from a small grammar and diffs each one
against the reference — 1500 plain, and 600 more whose main chunk runs inside a
coroutine.create with coroutine.yield sprinkled at random points and driven
by a bounded resume loop. Two constraints keep it from becoming a nuisance
generator:
- It must terminate. Only bounded
for/ipairsloops, nowhile, norepeat, nogoto; a call may only target a function that finished being defined earlier, so the call graph is a DAG and recursion is impossible. - It must be reproducible. Seeds increment from 0 — no
Math.random. A failure is reproducible from "seed 84".
The generator knows nothing about how we think Lua behaves, which is the point: it is the only gate that finds "does not crash, but the value is wrong".
luaF_newtbcupvalchecks__closeat assignment time, not at scope exit.OP_TBCruns right after the local receives its value, solocal x <close> = 5 print(x)raisesvariable 'x' got a non-closable valuebefore printing anything.nilandfalseare exempt (l_isfalse), solocal x <close> = nilis legal.- An error inside
__closeduring unwinding replaces the original error.error('orig')followed by a__closethat doeserror('in close')makespcallreturnin close. Keeping the original instead is wrong. - The code generator folds
OPR_MINUSandOPR_BNOTonly — neverOPR_LEN. So#7.5is a runtime error, not a load-time one. Folding it would make a script fail to compile that upstream compiles and only fails when run. - Two harness rules, learned the hard way. When a script errors, the stdout
it already produced must still be compared (the reference's stdout and stderr
are captured together, so "print ten lines, then crash" has both in its
output); and the
lua.cprogram-name prefix must be stripped everywhere, not just at the start, because it appears mid-output once stdout precedes it.
pcall,xpcall, metamethods andforiterators are yieldable; C-library callbacks are not.table.sort's comparator andstring.gsub's replacement function raiseattempt to yield across a C-call boundary, becauseltablib.c/lstrlib.ccall back throughlua_call(luaD_callnoyield).__tostringinvoked byprintis the same story. This is precisely the behaviour virtual threads would have "fixed" — and broken.- The main thread is never yieldable.
lstate.c'slua_newstatedoesincnny(L)unconditionally, socoroutine.isyieldable()isfalseon the main thread even with no C call in sight. A coroutine starts at zero, and everyresumeresets it to zero (lua_resumecopies only the low 16 bits ofnCcallsfrom the resumer). resume's arguments are the return values ofyield.lua_resume'sposcall(ci, nargs)is what makescoroutine.yield(1)see99afterresume(co, 99)— the number of values moved back is the number passed in, not the number yielded out.coroutine.wrapreports its own name, notcreate's.luaB_cowrapcallsluaB_cocreatedirectly, andluaL_argerrorderives the name from how the current frame was called — socoroutine.wrap(nil)saysbad argument #1 to 'coroutine.wrap'.luaV_finishOphas a branch per interruptible opcode, andOP_CONCAT's is not optional. Concatenation proceeds in batches, so after a__concatyield the metamethod result has to be moved back one slot,toppulled in by one, and the remaining elements concatenated — possibly yielding again. Skipping it silently yields the operand table instead of the concatenation result.
- The GC-visible stack ends at
L->top, and that is load-bearing.lgc.c: traversethreadmarks only[stack, top)and then, in the atomic phase, nils out[top, stack_last)— the source comment calls it "clear dead stack slice". So a value sitting in a register abovetopis invisible to the collector, and gets collected. Two shapes depend on this: a value pushed by a C function such asnext(the call returns,topdrops back, the slot still holds the key), and the table built by the last iteration of a loop (for i = 1, 5 do t[i] = {i} endleaves{5}in a register whileL->toponly reaches the call's arguments). Ourstackis a Java array, so every slot is a live reference —LuaState.clearDeadStackSlicesreproduces that clearance, and has to run insidecollectgarbage, not on everytopchange (see the next bullet). - …but clearing eagerly on every
topchange breaks two things. C only clears during the atomic phase. Doing it inline silently destroyed bothluaT_callTMres's result (the destination register can be the slot being popped) and the error object ofpcall(LuaError.valuealiases a live stack slot, becauseerror('boom', 0)throwsL.arg(0)itself — sopcallstarted returningfalse nil). Waiting untilcollectgarbageis both faithful and exactly as precise. - A weak table's storage has to be weak, not merely carry a
WeakReferencebeside it. The payload in the slot is theWeakReference; a strong payload means nothing is ever released. That has a consequence:LuaTable.getmust hand back a copy for weak tables, so the write fast path (OP_SETTABLE's "slot found, write into it") cannot be used on them — writing into a copy is a no-op, andt.f = gwould leavet.fequal tof. __modeis read dynamically, and re-wraps everything that is already in the table. Building a table, filling it, and then callingsetmetatable(t, {__mode = 'k'})still lets those entries be collected — measured. Since our weakness is a storage form, changing the mode has to walk the table and re-wrap in place, and a weak table must keep everything in the hash part (array slots are plainLuaValues and cannot hold aWeakRef).__gcneeds the object alive, which no JVM reference type can give you. Every "it's dead" notification (WeakReferencecleared,Cleaner,PhantomReference) arrives after the object is unreachable, and the reference cannot be recovered. Onlyfinalize()runs whilethisis still usable, and it is allowed to resurrect. So a table with__gcgets a sentinel that only it points to; when both go unreachable the sentinel'sfinalizepushes the table onto a pending queue, and the nextcollectgarbagedrains that queue on the Lua thread — never on the JVM finalizer thread, where calling into aLuaStatewould race and any exception would vanish without a trace.
These are not oversights; each is a place where the JVM has no primitive to match, and each is documented rather than papered over.
- Ephemeron tables only handle the case where the value is the key.
__mode='k'witht[k] = kclears the entry (matching the reference), and the usual cache shapecache[obj] = {fresh data}keeps its value (also matching). Butt[k] = {k}— value contains key, key otherwise unreachable — keeps the entry where Lua would drop it. The two halves are mutually exclusive in Java: holding the value strongly keeps the key alive; holding it weakly loses the value whenever the key is alive but the value is not otherwise referenced, which is exactly the common cache. Data loss is worse than retention, so we retain. - A weak entry whose key has a
__gcis dropped one cycle early. Lua marks finalizable objects, so the entry survives the cycle that schedules the finalizer. Ours keys offWeakReferenceclearing, which happens in the same cycle. - Objects with
__gcare only reclaimed whencollectgarbageruns. The JVM never tells us an object is unreachable except by collecting it, so there is no way to schedule a finalizer without an explicit cycle.
string (StringLib), table (TableLib), math (MathLib), utf8
(Utf8Lib), os (OsLib), io (IoLib) and package (PackageLib) are
ported, each with its own differential gate — StringLibTest (29),
MathLibTest (26), TableLibTest (18), Utf8LibTest (13), OsLibTest (24),
IoLibTest (35), PackageLibTest (22).
- A shared
lauxlib.clayer has to come before any library. Argument checking (checkString/checkInteger/argError/ …) is not scaffolding: the message text is part of the observable contract, and six hand-rolled copies would drift apart within a week. One detail is only right if the parameter numbering is 1-based:s:rep(nil)reportsbad argument #1, becauseluaL_argerrordecrements for a method call. math.max/math.minuselua_compare, not a numeric comparison. Somath.max("a","b")is"b", andmath.max(1,"b")raisesattempt to compare number with stringvia the failed__ltpath. They also preserve the operand's subtype:math.max(1, 2.0)is2.0(a float).math.fmod's float path is C'sfmod, not Lua's%. They differ in the sign of the result:math.fmod(-7.5, 2)is-1.5(sign of the dividend) while-7.5 % 2is0.5(sign of the divisor). Reusing the VM'sArith.modfhere is the easy mistake and it is wrong by a sign.math.log's base-2 and base-10 branches are genuinely different computations. C callslog2/log10;log(x)/log(2)differs fromlog2(x)on 9 of 43 sampled inputs, andlog(x)/log(10)fromlog10(x)on 25 of 43. Java has nolog2, so it is reconstructed ase + log(m)/ln2from the mantissa decompositionx = m * 2^e, which lands within 1 ulp on 190 of 192 inputs.table.inserton an object with__newindexmust call it.ltablib.cdoes not useluaH_setint; it useslua_seti, which goes throughluaV_finishset. Writing raw leavestable.insert(ud, v)silently doing nothing — no error, no write.os.executemust spawnsh, not/bin/sh. C'ssystem()isexecl("/bin/sh", "sh", "-c", cmd)— argv[0] issh— and the shell prefixes its diagnostics with argv[0]. Spelling the full path turnssh: 42: command not foundinto/bin/sh: 42: command not found, which the differential gate sees immediately.- A userdata's metatable lives on the object, not on its type.
luaL_testudataasks "is this userdata's metatable the one registered underFILE*?", so file handles cannot share a per-type metatable with the internal ones (the PRNG state,gmatch's state) without those being mistaken for files.LuaUserdatawas added for this, and with it__gc— otherwise a file left un-close()d by a script never gets closed at all. ioon a real file must be block-buffered, andio.writemust not be. C stdio buffers file output: afterio.output(p):write("x"), readingpback gives nothing, because the bytes are still in theFILE*. Standard streams are the opposite — they are line/unbuffered, andio.write's output has to reach whichever stream the host installed. SoIoLibbuffersRandomAccessFilewrites (flushing onflush,seek, any read, and close) and writes straight through for stdin/stdout/stderr.require's searcher result is tested withlua_isstring, not "is a string". A searcher that returns the number42has its value concatenated into the not-found message; only a non-string, non-number result is treated as "this searcher said nothing" and drops the\n\tprefix again.- Load-time errors run the chunk name through
luaO_chunkidtoo. The module file is compiled as"@" + path, butluaG_addinfostrips the@before printing, so a syntax error in a required module reads/tmp/m.lua:1: syntax error near 'is'— not@/tmp/m.lua:1: .... - Every library has to be registered in
_LOADED, not just in_G.coroutinewas the one that slipped through:require("coroutine")reportedmodule 'coroutine' not found, because the table was only in the globals.
sin / cos / tan / asin / acos / atan / exp / log / log10 /
sinh / cosh / tanh / pow are not bit-identical to the reference.
Java's Math is fdlibm; the reference binary links the platform libm (macOS
libsystem_m here). Both are correct to within 1 ulp of the true value, but
they round differently in the last place: of 43 typical inputs, sin differs on
1, tan on 12, cosh on 6. Closing that gap means shipping a libm of our own,
which neither fits the goal nor matches what Lua itself guarantees — Lua's own
results shift with the platform. The differential cases therefore compare these
functions to 14 significant digits (1 ulp is ~2.2e-16 relative). Everything
that can agree does agree: sqrt, abs, floor, ceil, frexp, ldexp
and the log branches above are bit-exact.
file (%p).f_tostringprints a pointer, soprint(io.stdout)can never match byte for byte.IoLibemits a same-shaped hex value and the differential harness normalises0x…away.stdinseeking. The reference's stdin in the harness is redirected from/dev/null, which is seekable (lseekreturns 0); ours is a byte array, which is not.io.stdout:seek()agrees (nil, "Illegal seek", 29on a pipe);io.stdin:seek()is not compared, because the difference is in where the stream comes from, not in whatiodoes.- Writing to an update stream straight after reading it. Where the bytes
land is C's read-ahead buffer, and ISO C leaves it undefined. Measured: 13 on
a 12-byte file, 4097 on a 6002-byte one, 5001 after
read(5000). We write at the logical position. On the well-defined path (with an interveningseek) the two agree. See Theio.openmode matrix below. - The position reported after writing to an append file. The reference gives
base + nwhile the data sits in the buffer andEOF + nonce it is flushed (the boundary is 4096); we always giveEOF + n, which is the POSIX answer. The bytes written are identical either way. See the same section.
loadlib.c's fallback branch is the honest one here: there is no dlopen on
the JVM (reaching it would mean JNI or FFI — a different project), so the C
searcher behaves the way C does on a platform built without dynamic loading.
package.loadlibreturnsnil, "dynamic libraries not enabled; check your Lua installation", "absent". The reference binary is built withdlopen, so it returns adlerror()string and"open".requireof a module whose.soactually exists stops at "dynamic libraries not enabled" instead of loading it. Modules that only exist as.lua, andpackage.preloadentries, are unaffected.
load / loadfile / dofile (BaseLib) and the binary-chunk pair
lundump.c / ldump.c (Undump / Dump) are in, with two differential
gates: LoadTest (25) and DumpTest (16).
DumpTest compares at two granularities, and the second one is the real
gate for this phase:
- Runtime. Each script's stdout is compared byte for byte, as everywhere else.
- Artefact.
string.dump(f)is written to a file by each side and the two files are compared byte for byte. Runtime-only comparison would let "the compiler emits something slightly wrong that still happens to run" through; the artefact comparison pins down constant ordering, the line tables, local and upvalue debug info, andmaxstacksizeas well.
Eight shapes are checked that way — plain, nested, vararg, upvalue, goto,
loop, assorted constants, and a constant string longer than
LUAI_MAXSHORTLEN (which takes the non-interned long-string path on both the
dump and the undump side).
luaU_undumprewrites the chunk name before it ever reports an error. A leading@or=is dropped; a name that starts with the signature byte (i.e. the chunk name is the binary chunk, which is whatload(s)uses when no second argument is given) becomes the literalbinary string. Soload("\27Lua")reportsbinary string: bad binary format (truncated chunk)— not an ESC-prefixed mess.f_parserconsumes the first byte on the binary path but hands it back on the text path.luaU_undump's comment says so directly: "skip 1st char (already read and checked)". Ungetting it beforeundumpmisaligns the whole header by one byte and turnstruncated chunkintonot a binary chunk.luaO_chunkidhas a third form, and it is the oneloadhits. Only@xand=xare stripped; anything else is wrapped as[string "x"].load("x++")reports[string "x++"]:1: syntax error near '+'— a "strip the prefix" implementation that skips the wrapping reportsx++:1:, and every load-time error message is then wrong.- Chunk names are byte strings, and
luaO_chunkidtruncates by bytes. Decoding to a JavaStringfirst and truncating there gets the length wrong on non-UTF-8 names.chunkIdtherefore has abyte[]core and theStringoverload goes through it. - A prototype with no source gets no position at all — not even a line
number.
luaG_addinfoisif (src) ... else return msg. This is observable:string.dump(f, true)strips the source, sopcall(load(string.dump(f, true)))reports the bareboom. Falling back to the chunk name not only adds a prefix, it splices the raw binary chunk into the message (invalid UTF-8, so it decodes to replacement characters). lundump'sdefault: lua_assert(0)is a no-op.lua_assertcompiles to((void)0)in a default build, so a chunk whose constant type byte is99loads fine and that constant is justnil. Raising an error there is inventing a semantic of our own. Measured, not assumed: the hand-built chunk loads as afunctionon the reference.
One more, on the dump side: dumpFunction compares sources by reference
(f->source == psource), not by content. Short strings are interned so the two
agree, but a long source name is not interned and C writes it out again for
every nested function. Comparing by content would emit less than C does.
Ported as StringPack (the pack/unpack half of lstrlib.c), with PackTest
(21 scripts). This is the only part of the standard library whose output
depends on the host platform: l / L are C's long, T is size_t, !'s
default maximum alignment is offsetof of LUAI_MAXALIGN, and the default byte
order is the host's. Those are pinned to the platform the reference binary runs
on — LP64 + little-endian — because on a platform where long is 4 bytes the
reference itself emits different bytes.
Two more things the reference binary settled here:
Xconsumes the option after it.getdetailscallsgetoptionto learn the next option's alignment, and that call advances the format string. Sostring.packsize("Xi4")is0, andstring.pack("Xi4", 7)is"". An implementation that peeks without advancing returns4— which looks more sensible and is wrong.str_pack'slua_pushnilmarker is observable. It exists to separate the arguments from the string buffer, but it sits at index 2, so with no second argumentstring.pack("c2")reportsstring expected, got nilrather thangot no value.
ldblib.c is in as DebugLib, together with the parts of ldebug.c that only
the introspection API needs (lua_getstack / lua_getinfo / luaG_findlocal /
luaG_traceexec / luaG_tracecall) and lauxlib.c's luaL_traceback.
DebugTest (36 scripts) is the gate. All sixteen debug functions are in.
LuaLibs.openAll (linit.c's luaL_openlibs) also landed — before it existed,
"open the standard libraries" was ten calls that only the test driver knew the
order of.
sethook is not a library function that can be ported on its own: it is a hook
dispatcher whose call points are spread over the interpreter. Porting it meant
adding, in addition to DebugLib.dHook (ldo.c: luaD_hook):
| Call point | Upstream | Fires |
|---|---|---|
LuaState.precallC |
ldo.c |
call for C functions — before the C function itself runs |
DebugLib.dHookCall from Vm's startfunc and from OP_VARARGPREP |
ldo.c: luaD_hookcall |
call / tail call for Lua functions |
DebugLib.rethook from LuaState.poscall |
ldo.c: rethook |
return for both kinds |
DebugLib.traceExec from Vm's vmfetch |
ldebug.c: luaG_traceexec |
line and count |
plus trap on CallInfo and hook / hookmask / hookcount / allowhook /
oldpc on LuaState. The point of trap is that it is a local variable in
luaV_execute, so with hooks off the main loop does not even branch.
lua_sethookmust callsettraps. The main loop'strapis a local that is only re-read from the frame at a few points (updatetrap). Turning a hook on mid-execution therefore does nothing at all unless every live Lua frame'strapis set to 1 — which is whatsettrapsdoes. Without it,debug.sethook(f, "l")produces zero line events.- A hook error is not caught by the
pcallit is reported for. Thecallhook forpcallfires insideprecallC, beforepcallestablishes its recovery point, sopcall(function() ... end)cannot catch an error raised by acallhook onpcallitself. CIST_HOOKEDis tested beforeisLuainfuncnamefromcall. The frame carrying the mark is often a C frame (thecallhook forpcallis reported onpcall's own C frame), so testingisLuafirst never reaches it — and the traceback line changes fromin hook '?'toin function <file:line>.- Hooks can never yield.
hookfcalls the user function withlua_call, which goes throughluaD_callnoyield, so the non-yieldable counter is non-zero inside a hook. Measured:coroutine.yield()in a hook reportsattempt to yield from outside a coroutineon the main thread andattempt to yield across a C-call boundaryin a coroutine — in both cases the program or coroutine dies. This is why the port does not implementCIST_HOOKYIELDat all: it is unreachable from Lua. (CIST_HOOKYIELDexists upstream for C hooks installed via the C API, which are called directly rather than throughlua_call.)
allowhook must be restored at the protected-call boundary, after the
message handler has run — not before. The handler runs at the throw point, where
allowhook is still 0 (that is what stops a traceback from re-entering the
hook). Restoring it earlier means the traceback triggers the hook again, which
raises again, which applyErrMsg turns into error in error handling. The
symptom: debug.sethook(function() error("x") end, "c") followed by an uncaught
error printed error in error handling instead of a real traceback.
- Level 0 is the C frame of
getinfoitself.lua_getstack(L, 0, &ar)succeeds and yieldswhat = "C", currentline = -1, name = "getinfo". The name comes fromfuncnamefromcallreading the caller's frame, so it answers "what was the caller calling", not "what is this frame called". Getting this off by one shifts everydebug.getinfo(n)result by a level. 'f'and'L'needlua_rotate, not a pop. The result table is pushed afterlua_getinfo, solua_getinfo's own outputs sit underneath it.treatstackoptionislua_rotate(L, -2, 1)followed bylua_setfield(L, -2). DoingL.top--instead pops the result table itself — the symptom isdebug.getinfo(1)returning something that prints as a function, anddebug.getinfo(1).funcholding the table.pushfuncnametries a global name first.errorisglobal 'error'as far asgetobjnameis concerned (it is aGETTABUP _ENV), but a traceback printsfunction 'error'—pushglobalfuncnamelooks the function up by value in_LOADED._Gand wins. Checking onlyar.namewhatgets the qualifier wrong on every C function that has a global name.- The message handler runs before the stack unwinds —
luaG_errormsgcalls it at the throw point, not fromluaD_pcall's recovery. This is the whole reasonxpcall(f, debug.traceback)shows the full traceback instead of just the outer two frames. Java has nolongjmp, but the Lua stack (stack/top/ci) is still intact at thecatch, so running the handler at the top ofpcall's catch is equivalent. LEVELS1/LEVELS2are 10 and 11, not a round 10/10, and the head and tail are not the same length. The "(skipping N levels)" count is off by one otherwise, and only on stacks deep enough to trigger it.
OP_TESTSET's condition was inverted, which brokeand/or. Upstream testsl_isfalse(rb) == k; this port testedtoBoolean() == (k != 0)— and those two are exact opposites overk's two values. It is a quiet bug:TESTSETonly survives when the result register differs fromB(patchTestRegrewrites it toOP_TESTotherwise), soif x thenwas always right whilex and ywas always wrong. The symptom isprint(t and "yes")printing the table, and — worse —cond and a or b, the most common idiom in Lua, taking the wrong branch. The bytecode gate could not see it: the emitted code was field-for-field identical to the reference's.RuntimeTest.logicalOperatorsReturnValuesnow pins the returned value, not just the truthiness.xpcall's handler ran after the call chain had already been restored. See the fourth bullet above;xpcall(f, debug.traceback)'s traceback was two frames long instead of seven.
lua.c runs the script from inside pmain, a C function, so an extra C
frame sits under the main chunk. It shows up as the trailing [C]: in ? in every
traceback and as debug.getinfo(3).what == "C". A host that calls
luaL_dostring directly has no such frame and sees neither. This is therefore
fixed in the differential driver (Diff now mirrors lua.c: pmain +
docall + msghandler), not in the engine — bending the engine to match would
have made luaL_dostring wrong instead.
The heaviest part of the fourth gate: UpstreamSuiteTest runs the upstream
.lua files unmodified and compares their output with the reference binary
byte for byte; UpstreamSmokeTest collects the ones that must not be
compared that way but still have to finish.
| File | Why it cannot be compared byte for byte |
|---|---|
cstack.lua |
each case prints a final count = how much C stack one nesting level eats; Java and C necessarily differ |
gc.lua |
the whole file tests the incremental collector's count / step / setpause; this port has no such layer |
literals.lua |
the trailing locale section needs os.setlocale("pt_BR") to actually take effect; the JVM cannot |
locals.lua |
tracegc writes a progress dot from a finalizer with io.stderr:write'.'; when it runs is up to the JVM |
constructs.lua |
line 319 is _ENV.GLOB1 = math.random(0, 1); two processes can never share a seed |
sort.lua |
it prints sort timings and a comparison count, and the input is generated with math.random |
math.lua |
it prints random seeds, the range of the random draws, and timings |
__closecan still yield after an error. Whenyieldable(L)holds,lua_pcallkdoes not install a local recovery point: the error travels all the way tolua_resume, andprecover→finishpcallk→luaF_close(..., yy = 1)finishes the job — so every__closemay yield, and a new error raised by a__closereplaces the old one for the variables still pending. Inlocals.lua's "yielding inside closing metamethods", three<close>variables yield in turn; the middle one raiseserror(err + 20)after yielding, so the third sees (andpcallfinally returns) 30, not 10. This port uses Java exceptions instead oflongjmp, so the recovery point sits inpcall'scatchandyyhas to be passed down explicitly —pcallpicks it fromyieldable(), which is exactlyk != NULL && yieldable(L).- When a yield interrupts the unwinding, somebody has to keep the error
object. In C it is always on the stack (
luaD_seterrorobjrewrites it before every close, andfinishpcallkreadsL->top - 1). Here it lives in the exception and vanishes with the Java stack, so it is parked inLuaState.closeErrPendinguntil the nextfinishPcallK. It must yield to a newer error, or the third variable would still see the stale 10. luaB_errorhas to be written in the C shape,lua_settop(L, 1)included: that call decides what sits one slot below the top, which is whatluaD_seterrorobj'sdefaultbranch reads. Without it, errors out ofcoroutine.wrap(...)lose theirfile:line:prefix.luaL_whereandluaG_addinfoanswer "no debug info" in opposite ways. The former needscurrentline > 0and otherwise emits nothing; the latter prints?:-1:anyway. Merging them breaks botherror("x")and the^%?:%-1:assertion inerrors.lua.collectgarbagevalidates its option first, and"generational"/"incremental"return the previous mode as a string (lbaselib.c: pushmode), not an integer.luaD_shrinkstack'sstackinuseis the max oftopand everyci->top, nottopalone, andstackErris cleared only when a shrink actually happens.- The main thread starts one slot up (
stack_initreserves it forbase_ci.func),IoLibmustlua_popafter creating the std files, andDiffmust pushmsghandleronto the stack the waylua.c: docalldoes — each is worth one level of recursion inerrors.lua's "expected stack overflow after N calls".
Both files used to crash on entry; they only reach OK after these. None of them
is a "we differ" — all four were plain mistakes on our side:
tunpack's overflow test must compare as unsigned. In C,nislua_Unsigned, so a wrappinge - ibecomes a huge value; a Javalongwraps to a negative one, the signed comparison lets it through, and the loop then pushes fromminItomaxIuntiltable.unpackdies withOutOfMemoryError.top + noverflows incheckStack/checkStackOk. With annof2^31the sum goes negative, so the "overMAXSTACK" test stops firing — sameOutOfMemoryError. Now computed aslong.tmove's test must not be simplified. C askse < LUA_MAXINTEGER + f, note - f < LUA_MAXINTEGER: the two agree only when nothing wraps, and wrapping is exactly what is being tested. Simplified,table.move({}, minI, maxI, 1)reports#4 (destination wrap around)where the reference reports#3 (too many elements to move).luaG_tointerrorcarries avarinfosuffix. Formath.huge << 1the reference saysnumber (field 'huge') has no integer representation.tonumbermust not useString.trim(). Java'strimstrips "characters at or below U+0020", which includes\0; C recognises onlySPACECHARS(" \f\n\r\t\v") and additionally requires the parse pointer to land exactly ons + l. So bothtonumber('1\0')andtonumber('1\0', 2)silently returned1.b_str2intis now written over bytes and re-checks the length.
sort.lua lives in the smoke set — it prints timings, so its output can never be
compared byte for byte — and a later full run failed inside it with
attempt to compare nil with number out of table.sort. The file seeds nothing,
so math.random() hands it fresh data on every run, which makes this a latent
bug: it only surfaces when a partition comes out unbalanced enough for auxsort
to draw a new random pivot.
IdxTisunsigned int, and so is the pivot seed. C'srnd % (r4 * 2)is an unsigned modulo, always in[0, r4*2). Java'sintis signed, and the seed carries exactly the 32 bits C would have produced — negative half the time. A signed modulo then dragspbelowlo, and at the extreme below1, wherea[p]is nil andsort_compreportsattempt to compare nil with number. NowInteger.remainderUnsigned, matching C.rndis passed by value. In C it is a parameter ofauxsort, so re-randomizing inside a recursive call never leaks back to the outer frame. It used to be shared through anint[]here, which meant a seed drawn deep in one subtree went on to steer every remaining partition of the whole sort.
One run in twenty is a bad way to find a bug, so choosePivot now carries its own
deterministic test (TableLibTest.choosePivotStaysInsideTheInterval) that checks
the C lua_assert — lo + r4 <= p <= up - r4 — directly, instead of hoping a
50 000-element sort trips over it.
ldo.c: f_parser reads one byte. If it is the first byte of LUA_SIGNATURE, the
chunk goes to luaU_undump; otherwise to luaY_parser. That single test is why a
caller never has to say whether it is handing over source or bytecode — and why
load, loadfile, dofile and require all accept precompiled chunks for free.
The dispatch used to exist only inside BaseLib's load / loadfile. lua_load
itself (LuaState.load) did not look at the signature, so require of a
precompiled module failed:
error loading module 'mod' from file '/tmp/mod.lua':
/tmp/mod.lua:1: unexpected symbol near '<\27>'
The reference loads it and returns the module. PackageLib reaches the loader
through LuaState.loadFile, which is why this showed up there and not in load.
The fix is at the lua_load layer, not a patch on require:
LuaState.load/loadFilenow dispatch onLua.LUA_SIGNATURE0;loadFilestill runs thelauxlibpreamble first (BOM,#!), because a precompiled chunk never starts with#— order matters;- the binary branch does not push the first byte back:
luaU_undumpsays "skip 1st char (already read and checked)", socheckHeaderstarts at byte two.
BinaryChunkTest covers both directions:
- chunks produced by the real
luac(ReferenceLua.compile, located viaLUAC54_BINor sitting next to the referencelua), run on both sides and compared byte for byte. Self-compiled chunks would be grading our own compiler with our own compiler, so the reference'sluacis what produces them. The corpus covers upvalues, varargs,goto, mixed table constructors, integer and float edges, coroutines, a 300-entry constant table, and two scripts that fail on purpose — the error messages carry line numbers and local-variable names, so losing the debug info inundumpturns them red; - chunks produced by our own
string.dump(require,dofile,loadfile, themodeargument, thebinary stringchunk-name rule, and truncated or tampered chunks, which must be load-time errors rather than crashes).
The generator had been emitting programs that do not compile:
- table constructors wrote
{'key0' = v}. Only a name can be a bare key, so a string key needs[k] = v— the numeric case was bracketed, the string case was not; expressionemitted(select('#', ...))unconditionally, while a function declares...only 40% of the time. Outside a vararg function that iscannot use '...' outside a vararg function.
Measured with the reference luac: 178 of 200 seeds failed to compile. A
program that does not compile produces the same error on both sides, so the
comparison "passes" while proving nothing — the strongest gate had been quietly
vacuous over tens of thousands of seeds. Fixed (keys are always bracketed; the
generator tracks whether the function being built is vararg), and re-measured:
0 / 400.
FuzzTest.assertCompiles now makes that state unrepresentable: it fails the seed
whose reference output has the shape of a compile error. The discriminator is
"the message ends in near 'tok'" rather than "contains expected", because
runtime errors are full of expected (bad argument #1 to 'upper' (string expected, got nil)) and those are supposed to happen.
Ordinary fuzzing only ever takes the text path — Dump and Undump were touched
by a few dozen hand-written cases, which is exactly the blind spot the
require-a-precompiled-chunk bug lived in. generatedChunksSurviveDumpAndUndump
closes it: the generated body is wrapped in a vararg function, string.dumped,
loaded back and run. That pushes thousands of random Protos through
serialization — constant-table order and varint encoding, line info, local and
upvalue debug names, maxstacksize, nested protos. A sentinel print
(dumped true loaded true) is asserted on, so a silent string.dump failure
cannot masquerade as agreement.
The fuzz generator learned table.* and immediately paid for itself: five
divergences, none of which any hand-written case had hit.
l_str2int's accumulator islua_Unsigned. Inlobject.cthe running valueais unsigned, and the overflow guard isa >= MAXBY10. With a signedlong, onceapasses 2^63 it goes negative and the guard stops firing, so the overflow escapes:1 + (math.mininteger .. 0)— the operand is"-92233720368547758080"— wraps to exactly 0 and returns1instead of falling through to the float path (-9.2233720368548e+19). NowLong.compareUnsigned.luaG_runerroraddsfile:line:where it is raised, not later.idivandmoddivide-by-zero go throughluaG_runerror, which callsluaG_addinfoon the spot. Ours threw a bareLuaErrorthat only received its prefix at the protection boundary, by which time the raising frame was gone — socoroutine.resumereturned the error as a value and it arrived without a position:false attempt to divide by zeroinstead offalse file:line: attempt to divide by zero.Arith.rawArithnow takes aLuaStatefor exactly this reason.luaH_newkeyrefuses nil values. Its first line isif (ttisnil(value)) return; /* do not insert nil values */. Assigning nil to a key that does not exist creates no node and triggers no rehash. Creating the node changes the computed array-part size, which is a direct input to#t:table.move({[1]=4.0}, 0, 2, 1)leaves#t == 2in the reference and0otherwise. The nil-key and NaN-key checks sit before that line, sot[nil] = nilmust still raisetable index is nil.lauxlib.c: interrortestslua_isnumber, not the type.luaL_checkintegerthat cannot get an integer reportsnumber has no integer representationwhenlua_isnumber(L, arg)— true for a string that parses as a number — andnumber expected, got stringotherwise. Sostring.format("%d", "3.5")is the former ("3.5"parses, 3.5 just is not an integer) andstring.format("%d", "abc")is the latter. The guard used to bev.isNumber(), which made both the latter.%.14gmust round once, from the exact value. Java'sString.format("%.Nf")does not round the value: it roundsDouble.toString's shortest representation, i.e. twice.math.sqrt(math.maxinteger)is exactly3037000499.976049900054931640625, whose shortest form is3037000499.97605; the 15th digit is a 5, so the second rounding carries and produces3037000499.9761. The reference prints3037000499.976.StringLibalready formatted throughBigDecimal(whosenew BigDecimal(double)is the exact binary expansion);luaO_tostringhad its own copy built onString.format. Two copies of one C conversion is how they drifted, soVm.formatGnow delegates toStringLib.generalDigits.
That last one is worth stating plainly: measured over 4602 printed floats, the
old code was wrong on 12 — 0.26%. No hand-written case was ever going to find
it, and no amount of upstream-suite greenness would have either. It took the
fuzzer. RuntimeTest.floatFormattingMatchesReferenceAcrossMagnitudes now pins it
with a few thousand values, generated by an LCG inside the script so that the
random-number generator itself stays out of the comparison.
The generator then learned utf8.*, gmatch, next, string.pack/unpack,
table.pack, non-string error values, math.ult/fmod/max/min, and
rawequal/rawlen/select. Measured before trusting it: every new branch fires
in 6–29% of programs, so the corpus is actually alive. It immediately found two
more, both in ltable.c — the same file as three of the previous five.
-
luaH_getnanswerslimit - 1before it ever binary-searches. The manual only asks#tto return a boundary, which makes "we found a valid boundary" sound sufficient. It is not: real programs observe which one.ltable.chas a fast path for the common case (t[#t] = nil, then ask for the length again):if (limit > 0 && isempty(&t->array[limit - 1])) { if (limit >= 2 && !isempty(&t->array[limit - 2])) return limit - 1; /* (1a) do not search */ else { /* (1b) */ ... binsearch ... } }
An array of
[1, nil, 3, nil]: binary search returns1(it finds the first boundary), the reference returns3.table.pack(1, nil, 3, nil)leaves#t == 3, not1. Across a systematic sweep of hole patterns — every single hole up ton = 32, every hole pair up ton = 19, every triple up ton = 11, all 32table.packnil combinations — 212 of 4968 results were wrong (4.3%). -
luaH_resizemust let the hash part shrink to dummy. Afterrehash, C'sreinsertmoves the old entries withluaH_set, so any integer key<= asizeis routed into the new array part and occupies no hash slot;luaH_resize(L, t, 4, 0)therefore ends with a dummy (zero-slot) hash. This port sized the hash to fit all the moved entries, so an array of 4 came with 2 stale hash slots. That matters becauseluaH_newkeytestsisdummy(t): with a non-dummy hash, the next insert just takes a free slot and never rehashes, so the array part stops growing.t = {}; t[2]=2; t[3]=3; t[4]=4; t[6]=6; t[7]=7gives the reference an array part of 8 and#t == 7; this port had 4 and 4. The sweep above is what exposed it — the previous corpus only ever inserted keys1..nin order, which never reaches this path.
Both are pinned by RuntimeTest.tableLengthBoundaryMatchesReference, which builds
the hole patterns with loops inside the script (an LCG would put the generator
itself into the comparison) and reads #t twice per table, because C rewrites
alimit as a hint on the first call.
The same corpus was then also run through coroutine.create, which found a bug
that had nothing to do with coroutines:
pcall(math.random, 5, 1)inside a coroutine reportedbad argument #1 to '?' (interval is empty); the reference reportsbad argument #1 to 'math.random' (interval is empty).
-
_LOADEDlives atglobal_Statelevel, not per-thread.lauxlib.c'spushglobalfuncnamereverse-looks-up the current function inpackage.loaded, two levels deep, to turn'?'into a real name:static void pushglobalfuncname (lua_State *L, lua_Debug *ar) { ... lua_getfield(L, LUA_REGISTRYINDEX, LUA_LOADED_TABLE); if (findfield(L, L.top - 1, 2)) { ... }
LUA_LOADED_TABLEis a field of the registry, and the registry belongs toglobal_State— everylua_Stateshares one. This port had madeloadeda per-LuaStatetable, so a coroutine'spackage.loadedwas empty and every diagnostic inside it degraded to'?'. The fix is one line in the constructor:loaded = main.loaded. Note that this only shows up inside a coroutine — the main thread populated its own table, which is why the whole upstream suite and every earlier fuzz seed were silent about it.
RuntimeTest.libraryFunctionNamesResolveInsideCoroutines pins it.
A systematic sweep of string.format("%q", ...) — every byte 0–255, in four
positions — was the first probe that wrote non-UTF-8 bytes to stdout. It found
nothing wrong with the engine, and something wrong with the harness: both sides
captured the reference's and this port's output into a Java String by decoding
UTF-8. Any invalid byte becomes U+FFFD, so 0x80 and 0xEF 0xBF 0xBD compare
equal:
>>> b'"\x80"\n'.decode('utf-8', 'replace') == b'"\xef\xbf\xbd"\n'.decode('utf-8', 'replace')
True
>>> b'"\x80"\n'.decode('latin-1') == b'"\xef\xbf\xbd"\n'.decode('latin-1')
False
Measured rather than assumed: with print deliberately changed to round-trip
through a Java String (i.e. corrupting every non-UTF-8 byte), the new %q
sweep is red under Latin-1 and green under UTF-8 — the old gate let a real
byte-level corruption through. The gate now decodes Latin-1, which is a
byte-for-char bijection, so comparisons are genuinely byte-exact.
RuntimeTest.quoteFormatMatchesReferenceForEveryByte is the pin.
The next probe was a full matrix: every operator × every pair of types, with no
metamethods, comparing the error text. 1651 lines. A second one did the same
with metamethods present — which of the two handlers runs, __le falling back to
not (b < a), __index/__newindex in both table and function form, __concat
right-associativity, __close timing and error propagation (245 lines). The
second one was clean; the first found this:
-
luaG_tointerrorconverts withF2Ieq, not "floor to integer".lvm.hdefinesLUA_FLOORN2IasF2Ieqby default — the name says floor, the semantics are "accepts only integral values":void luaG_tointerror (lua_State *L, const TValue *p1, const TValue *p2) { lua_Integer temp; if (!luaV_tointegerns(p1, &temp, LUA_FLOORN2I)) p2 = p1; /* report the *first* operand if it cannot convert */ luaG_runerror(L, "number%s has no integer representation", varinfo(L, p2)); }
Judging with a rounding mode makes
1.5look convertible, so1.5 & 3reported the right operand:number (local 'b') has no integer representation, where the reference reports the left,(local 'a'). Ten lines of the matrix were wrong. Hand-written cases only ever usedinf/nan, where both modes agree — which is why this survived every earlier gate.
Both matrices are now permanent tests:
RuntimeTest.operatorTypeErrorsMatchReference and
RuntimeTest.metamethodDispatchMatchesReference.
lstrlib.c: str_format is one of the densest functions in the source, so the next
probe swept it: 22 verbs × 10 flag combinations × 5 widths × 9 precisions × 22
value types — 20777 lines. It found two things.
-
The order of the two checks.
str_formatdoes not validate the spec first. Ford i u o x Xit callsluaL_checkintegerbeforecheckformat, and forf e E g Git callsluaL_checknumberfirst:case 'd': case 'i': flags = L_FMTFLAGSI; goto intcase; ... intcase: { lua_Integer n = luaL_checkinteger(L, arg); checkformat(L, form, flags, 1);
(
c,aandsare the other way round.) Validating the spec first meant("%.100d"):format("x")reportedinvalid conversion specificationwhere the reference reportsnumber expected, got string— 6620 lines wrong. -
A precision turns off the
0flag for integer verbs. C99: "Ford,i,o,u,x, andXconversions, if a precision is specified, the0flag is ignored." So("%05.3d"):format(42)is042— the precision pads to042, then the width pads with spaces — not00042. 220 lines. Float verbs have no such rule (%08.3fstill zero-pads), so the fix is confined toemitInt.
RuntimeTest.formatSpecifierMatrixMatchesReference pins both; reverting either
one alone turns it red.
The next sweep was os.date specifiers, os.time argument checking,
utf8.* boundaries, and tonumber across bases 2–36 (854 lines). It found a
familiar shape of bug: luaL_checkinteger had two implementations, and the
older one in BaseLib was missing two things — the function-name prefix
(bad argument #2 (...) instead of bad argument #2 to 'tonumber' (...)) and
the "no integer representation" case, reporting number expected, got number
for 1.5. Four call sites were affected: the base of tonumber, the index of
select, the level of error, and the index of the ipairs iterator. It now
delegates to LuaAux.checkInteger.
That sweep also turned up a second instance of "not our bug, libc disagreeing":
os.date("!%z", t). C's strftime reads the process-global timezone,
not the tm Lua hands it, and with the ! (UTC) flag the two libc's answer
differently — macOS prints the local offset +0800, glibc prints +0000. The
test therefore only exercises %z/%Z without !, where both sides run on
the same machine and the answer is well-defined.
string.pack/unpack/packsize also got a full sweep — every format verb
(including i/I widths 1–16, c/s counts, X and !n alignment, all four
byte orders) × values, plus invalid formats and out-of-range offsets, 706 lines.
It was clean; it is now a test so that "already agrees" stays agreed.
quotefloat formats through C's %a:
int nb = lua_number2strx(L, buff, MAX_ITEM, "%" LUA_NUMBER_FRMLEN "a", n);C99 fixes one hexadecimal digit to the left of the point and says it "is 0 only
if the argument is denormalized" — which leaves how a subnormal is written
unspecified. glibc and Apple libc pick opposite answers:
| value | glibc (CI: ubuntu) | Apple libc / musl |
|---|---|---|
5e-324 |
0x0.0000000000001p-1022 |
0x1p-1074 |
3 * 5e-324 |
0x0.0000000000003p-1022 |
0x1.8p-1073 |
This port matches glibc, because the gate that matters — CI — runs on
ubuntu-latest. StringLibTest.subnormalQuoteFollowsGlibc pins it with a plain
assertion outside the differential table, so that nobody "fixes" it by matching
a macOS reference binary and breaks Linux. (Double.toHexString already produces
the glibc form, so there is nothing to implement.)
Apple libc is simply wrong when %a is given a precision and the first dropped
hex digit is 8. It is not round-to-nearest at all — its decision depends on the
parity of the second dropped digit:
("%.4a"):format(0x1.500082p0) -> 0x1.5000p+0 # 0x82 > 0x80, yet rounded down
("%.4a"):format(0x1.500081p0) -> 0x1.5001p+0 # ...and this one rounds up
glibc's rule, straight from stdio-common/printf_fphex.c plus
include/rounding-mode.h:
bool more_bits = ((next_digit_value & 7) != 0 || precision + 1 < numend - numstr);
do_round_away = round_away (negative, last_digit_value & 1,
next_digit_value >= 8, more_bits, mode);
/* FE_TONEAREST: half_bit && (last_digit_odd || more_bits) */i.e. round-to-nearest, ties-to-even — and when precision == 0 the "last kept
digit" is the one left of the point, so ("%.0a"):format(1.5) is 0x2p+0, not
0x1p+0. StringLibTest.hexFloatPrecisionRoundingFollowsGlibc pins these with
plain assertions; the differential sweep skips every case whose first dropped digit
is 8 (the two sides skip exactly the same set — the predicate is computed from the
unrounded %a, which both agree on).
Same file, one more: inf and nan are not zero-padded. The special branch
does pad(buf, ' ', width) unconditionally and never looks at pad == '0', so
("%020.5a"):format(math.huge) is seventeen spaces followed by inf.
The table sweep — insert / remove / move / concat / unpack / pack /
sort × positions × sizes × offsets × value types, including same-table
overlapping move and cross-table move, 6277 lines — produced 268 diff
lines, and every one of them had an unsettling shape: the table's contents were
identical, only #t differed.
< MV2-dump #0 -2=nil,-1=nil,0=nil,1=nil,2=1,3=nil,4=nil,...
> MV2-dump #2 -2=nil,-1=nil,0=nil,1=nil,2=1,3=nil,4=nil,...
The cause is a missing clause. C decides the copy direction like this:
if (t > e || t <= f || (tt != 1 && !lua_compare(L, 1, tt, LUA_OPEQ))) {
for (i = 0; i < n; i++) { ... } /* forward */
} else {
for (i = n - 1; i >= 0; i--) { ... } /* backward */
}This port had only the first two clauses. The third one is not about positions at
all: when the source and destination are different tables, the copy is always
forward, regardless of how t and f relate. Dropping it sends cases that
should go forward down the backward path.
Order does not change what ends up in the table — but it changes the layout,
which is what #t reads. table.move({1}, -2, 1, -1, {1}) leaves the destination
with only [2] = 1 either way; going forward clears [1] first and leaves
alimit at 1, so luaH_getn's first case ("array[alimit-1] is nil ⇒ binary
search inside the array part only") fires and the answer is 0. Going backward
sets [2] first, pushing alimit to 2, that case no longer applies, and the
answer becomes 2. Hence the confusing symptom: the same sequence of writes
agrees when driven through rawset and disagrees when driven through
table.move, because only the internal layout differs.
Two lessons went into the tests:
- Comparing table contents is not enough —
#thas to be compared too, or layout bugs are invisible. - The sweep is now three permanent tests:
RuntimeTest.tableLibraryMatrixMatchesReference(6277 lines, the one that found it),RuntimeTest.patternMatchingMatrixMatchesReference(24819 lines of pattern matching × subject ×init× replacement form — clean, pinned so that "already agrees" keeps agreeing), andRuntimeTest.tableLengthAcrossInsertionHistoriesMatchesReference(3736 lines that make insertion history itself the sweep dimension: every subset of1..n× three insertion orders, fill-then-punch,move-built layout,insert/removealternation, negative/zero keys, bisection up to 40, tail truncation — also clean).
Reverting the one clause turns the first of them red.
The next sweep — debug.getinfo (14 fields × levels 0–6), getlocal/setlocal,
the upvalue family, traceback (including inside a coroutine), sethook with 10
masks plus the count hook, load (source / reader function / mode / env / binary
chunk), package.path / cpath / config / searchers and require's odd
corners, error at levels 0–4, xpcall, assert, and error propagation through
coroutines, 289 lines — found xpcall had lost its argument check:
static int luaB_xpcall (lua_State *L) {
...
luaL_checktype(L, 2, LUA_TFUNCTION); /* check error function */C checks that the message handler is a function before calling anything. This
port only checked that two arguments were present, so xpcall(f, 5) did not
raise — it called f, and when f errored it tried to use 5 as the handler:
| reference | this port, before | |
|---|---|---|
xpcall(f, 5) |
false bad argument #2 to 'xpcall' (function expected, got number) |
true false error in error handling |
xpcall() |
false bad argument #2 to 'xpcall' (function expected, got no value) |
false bad argument #2 to 'xpcall' (value expected) |
The io sweep also went in — 32 open modes (including every invalid one), every
read format and byte counts 0–24, numeric reads, seek, lines, using a
closed file, io.input/output/read/write/close, append-mode seek
semantics, r+ interleaving, a 5000-byte block, and the same file opened twice,
167 lines. It was clean, and it is worth having anyway: it is the only thing that
exercises the errno layer — a failed io.open must return
"path: No such file or directory" plus errno 2, which C gets from strerror
and we have to match on our own.
The debug matrix was green on the first attempt — with the xpcall bug still in
the code. That was the gate failing, not the code passing.
Diff.assertMatches normalizes by cutting everything from the first
stack traceback: onwards. That is deliberate: when a script crashes, the
message comes first and the traceback last, and our ldebug.c cannot be expected
to match it line for line. But when the traceback is printed by the script —
debug.traceback()'s return value — it lands in the middle, and everything the
script printed afterwards was thrown away. This script prints a traceback in its
fifth section and has 180 lines after it.
The guard now rejects that case: if anything other than traceback lines (each
starts with a tab) follows the traceback, assertMatches fails and says to use
assertFullMatches, which does not strip. Thirteen existing tests turned out to
have tracebacks in their output; the refined check — content after the
traceback, not merely its presence — leaves all of them alone, because in those
the traceback really is last.
This is the second time the gate has been found blind rather than the code wrong (the first was UTF-8 decoding of captured output). A gate that quietly compares less than it claims is worse than no gate: it reports green.
At scale (16500 generated programs) the fuzzer produced a program whose stack overflow was reported on line 67 by the reference and line 65 by this port. The engine was fine; the harness was one stack slot short:
lua_pushcfunction(L, &pmain); /* to call 'pmain' in protected mode */
lua_pushinteger(L, argc); /* 1st argument */
lua_pushlightuserdata(L, argv); /* 2nd argument */
status = lua_pcall(L, 2, 1, 0); /* do the call */lua.c's main passes pmain two arguments, so the stack's initial top
is 3. Diff.runLikeLuaC passed one. One missing slot is carried all the way
down: the recursion that overflows the stack goes one level deeper, so the
innermost frame is a different function, so the reported line differs.
The reason it took a while to see: it does not always differ. Each frame
costs k + 1 slots (k = number of locals) and the depth is
floor((limit - base) / (k + 1)), so a base one slot off only changes the answer
when it happens to cross a division boundary. Measured across frame sizes:
locals k |
0 | 1 | 2 | 3 | 4 | 6 | 8 | 16 | 32 |
|---|---|---|---|---|---|---|---|---|---|
| differed before | yes | yes | yes | no | no | yes | no | no | no |
Trying one example would have looked fine. Fixing the harness makes all nine
match, and RuntimeTest.stackOverflowDepthAndLineMatchReference now sweeps them.
(The other trap in that probe: local function f() return f() end is a tail
call, consumes no stack, and loops forever instead of overflowing — the
reference binary had to be killed after five minutes. The probe uses
return 1 + f().)
Not every sweep finds something. math (every function × 30 boundary values
including ±inf, nan, −0.0, maxinteger, mininteger, 2^53, 2^63, 1/3; both
return values of modf; log's base; six kinds of argument error per function;
the seeded random sequence) plus the deterministic parts of os, 1423 lines,
is clean. It stays in the suite for three reasons:
- The last bit of floating point.
math.sinand friends go throughlibm;Math.sinneed not agree bit for bit. Defaulttostringprints 14 significant digits, which happens to hide it — the sweep proves it is hidden, rather than never having been looked at. math.random's sequence — xoshiro256** pins it exactly, so a seeded run must match number for number.- The
errno/strerrorlayer again, onos.remove/os.rename.
Three things there cannot be tested at all: os.clock (elapsed time), a
math.randomseed with no argument (seeds from the clock), and os.time with no
argument (current time).
One catch while landing it: math.atan and math.atan2 are the same
math_atan in lmathlib.c, so which name shows up in the error message is
decided by whichever key lauxlib.c: findfield runs into first while walking
package.loaded — and that order comes from the string hash, which is seeded by
luai_makeseed (time-based). The reference binary is nondeterministic here:
$ for i in 1 2 3; do lua -e 'print(pcall(math.atan))'; done
false bad argument #1 to 'math.atan2' (number expected, got no value)
false bad argument #1 to 'math.atan2' (number expected, got no value)
false bad argument #1 to 'math.atan' (number expected, got no value)
Six atan2 and two atan out of eight runs. We are deterministic (the hash is
not seeded) and that is a deliberate deviation. Sweeping the zero-argument error
name of every function in math, string, table, os, io, coroutine
and debug, eight runs each, atan is the only one that moves; the script now
normalises it before comparing.
The non-pattern-matching half of the string library — sub, byte, char,
rep, upper, lower, len, reverse, format, tonumber(base), tostring
(pattern semantics are covered separately by the 24819-line sweep) — 3921
lines, and it found two:
__name follows C string rules.
int tt = luaL_getmetafield(L, idx, "__name");
const char *kind = (tt == LUA_TSTRING) ? lua_tostring(L, -1) : ...;
lua_pushfstring(L, "%s: %p", kind, lua_topointer(L, idx));lua_tostring hands back a const char * and "%s" treats it as a C string, so
the name is cut at the first \0: with __name = "A\0B" the reference prints
just A, while we printed the whole "A\0B". Lua strings are not
NUL-terminated, C strings are, and here C wins. Fixed with
LuaString.toCString(), for the places where the C code uses a value as a
const char *.
%p does not zero-pad. The address was formatted 0x%016x; the reference
prints 0x7ab30002c0, with no leading zeros (neither macOS nor glibc pads).
And one thing that cannot be compared at all: with the + or space flag on a
NaN, macOS libc prints no sign (nan) while glibc prints one (+nan). CI's
conformance job runs on ubuntu, this machine runs macOS, so the reference
itself differs between the two. The script strips NaN's sign before comparing;
the implementation stays glibc-compatible.
The previous sweep only tried zero or one argument. This one permutes the second
and third arguments too: for ~70 functions, every combination of {nil, 1, "x"}
from zero arguments up to that function's arity — 1866 lines. It found two,
and both are the same mistake: the order of the argument checks.
string.gsub:
int tr = lua_type(L, 3); /* just record the type */
lua_Integer max_s = luaL_optinteger(L, 4, srcl + 1);
...
luaL_argexpected(L, tr == LUA_TNUMBER || tr == LUA_TSTRING ||
tr == LUA_TFUNCTION || tr == LUA_TTABLE, 3,
"string/function/table");The fourth argument is checked first. So gsub(s, p, nil, "x") reports
bad argument #4 to 'gsub' (number expected, got string); we checked argument 3
first and reported bad argument #3 (string/function/table expected, got nil).
tonumber: the three checks run luaL_checkinteger(2) →
luaL_checktype(1, LUA_TSTRING) → luaL_argcheck(2 <= base <= 36, ...) — the
range check comes last:
| reference | before the fix | |
|---|---|---|
tonumber(nil, 1) |
bad argument #1 to 'tonumber' (string expected, got nil) |
bad argument #2 (base out of range) |
tonumber(1, 1) |
bad argument #1 to 'tonumber' (string expected, got number) |
bad argument #2 (base out of range) |
Three functions are deliberately absent: os.time and os.date read the clock
when their first argument is nil, math.randomseed(nil) seeds from the clock,
and io.lines with no argument reads stdin.
311 lines. package's static structure, searchpath, every shape of
require (module returning nil / false / true / nothing, caching, a module
that raises, circular dependency, preload winning over disk, names with
slashes), and load's mode / env / chunkname / reader functions.
package.loadlib's third return value says where it failed, not that it
failed. ERRLIB (the library could not be opened) gives "open", ERRFUNC
(the symbol is not in it) gives "absent":
lua_pushliteral(L, (stat == ERRLIB) ? "open" : "absent");We had it hard-coded to "absent", copying the fallback branch for builds
without dlopen — but in that branch lookforfunc always returns ERRLIB, so
"open" is what it should have been all along. Code that uses the third value
to decide whether to keep looking picks the wrong branch.
A circular require does not report a bare C stack overflow. It reports:
error loading module '_pk_b' from file './_pk_b.lua':
C stack overflow
because the parser and luaD_call share one counter: #define enterlevel(ls) luaE_incCstack(ls->L). So the 200th level is not reached at a lua_call — it is
reached in the middle of some luaL_loadfile, the error is caught by
luaD_protectedparser, loadfile returns a non-OK status, and checkload
wraps it. Counting call depth and expression depth separately gives the naked
message instead. The sweep pins the exact shape, module name included.
Fixing that exposed the counter's other half: the rollback. luaD_call
increments nCcalls before the try-equivalent, so when luaE_checkcstack
raises, the increment is outside the finally; C gets it back from
luaD_rawrunprotected's L->nCcalls = oldnCcalls at the pcall boundary, and
Java has no longjmp to fall back on. Without the rollback every C-stack
overflow leaks one level forever — measured with a pcall-recursion probe, the
quota walks 197 → 196 → 195 → 194 → 193, and upstream errors.lua starts
reporting error in error handling where the reference prints OK. Same for the
parser: the restore has to sit on lua_load, not on the binary/text dispatch
below it, or a failed load leaks two levels each time.
And one that cannot be compared: package.loadlib's failure message comes
from the platform libc (dlopen(...) on macOS, cannot open shared object file... on glibc), and this implementation does not load dynamic libraries at
all — three different strings, so the sweep compares only that it is a string.
io was the thinnest of the ten standard libraries at 167 lines. This one adds
124 lines of output: sixteen legal open modes and nine illegal ones (which
raise rather than returning nil + message — l_checkmode wants [rwa]%+?b?),
every read format, how write renders integers versus floats, all three
seek origins and their out-of-range cases, lines, setvbuf, closing twice,
io.output / io.input switching, and binary safety with \0 and invalid UTF-8.
It found one thing: a negative target position is EINVAL, not a generic I/O
error.
reference: nil Invalid argument 22 -- seek("set", -1)
before: nil Input/output error 5
fseek gives EINVAL for a negative resulting offset from all three origins
(SET, CUR, END), while Java's RandomAccessFile.seek throws "Negative
seek offset", which falls through errnoOf's catch-all into EIO. Seeking
past EOF is legal and returns the new position — easy to block by accident
while fixing this.
One thing is deliberately absent: print writes to the C-level stdout, so it
is not redirected by io.output — a script that assumes otherwise produces two
different wrong answers.
fopen on a directory used to be listed here as absent too, and that was a
mistake: it is a single POSIX rule (open(dir, O_RDONLY) succeeds, O_WRONLY
gives EISDIR, and the first read fails with EISDIR), it is the same on macOS
and glibc, and the JVM simply refused to model it. It is now modelled — see
The directory bug below. The one part that is genuinely not comparable is
seek("end") on a directory, which returns the directory's own size (64 here),
a filesystem fact rather than a Lua one.
The 854-line sweep covered every os.date conversion specifier. This one covers
the half it did not: os.time with tables (leap-year carry, month/day/hour out
of range, defaults, ignored extra fields), difftime, getenv, remove /
rename, tmpname, execute's exit codes, setlocale.
The fields of a date table are read with lua_tointegerx, which converts
strings.
lua_Integer res = lua_tointegerx(L, -1, &isnum); /* cvt2num: strings too */So os.time{year = "2000", month = 1, day = 1} is the year 2000; we reported
field 'year' is not an integer. Library arguments go through the coercing
luaL_check* family, while the VM's arithmetic instructions deliberately do not
coerce — two different rules, easy to write as one.
The sweep also pins down that month and day are mandatory (field 'month' missing in date table; year defaults and hour defaults to 12), that a field
too large for the int range reports field 'x' is out-of-bound, and that extra
fields (yday, wday, …) are ignored.
One thing is deliberately absent: extreme years such as year = -100, because
mktime's representable range is libc-specific (macOS reports "time result
cannot be represented in this installation").
Removing a directory used to be absent too, on the grounds that ENOTEMPTY
is 66 on macOS and 39 on glibc — but "the number differs per OS" is a reason to
key off the OS, not to skip the case. It is now compared, with ENOTEMPTY
resolved from os.name; see The errno shape table below. (Skipping it had
hidden that the errno was wrong outright: it was hard-coded to 39 everywhere.)
This is the semantically riskiest corner of the whole project: there is no
collector (lgc.c is not ported), so weak tables are rebuilt on
WeakReference and __gc on Cleaner. It deserves more than twelve hand-written
tests. The sweep covers weak keys / values / both, keys that are still strongly
reachable, mutually-referencing weak values, values that are not collectable at
all (strings, numbers, booleans), every spelling of __mode, __gc on one and on
twenty objects, __gc raising, __gc added after setmetatable (which C
ignores — it only checks at that moment), and <close>'s ordering and error
argument.
It found one thing: collectgarbage("generational") / ("incremental") report
isdecGCmodegen, not gckind.
#define isdecGCmodegen(g) (g->gckind == KGC_GEN || g->lastatomic != 0)lastatomic is non-zero after the first real collection, and loading a script
allocates, so user code starts out looking generational even though gckind is
KGC_INC. Five consecutive switches in the reference:
1 generational 2 incremental 3 incremental 4 generational 5 incremental
Tracking only gckind gets the first one wrong. Both switches clear
lastatomic, so afterwards gckind alone decides.
Two things cannot be compared: collectgarbage("count") (nothing to count
without a collector) and collectgarbage("step")'s return value — a single step
in the reference never finishes a cycle, so it is always false; with no
incremental collector one step is a full round, so we return true.
Deliberately: returning false would make the repeat until collectgarbage("step") idiom spin forever.
Both sides had been swept separately (114 and 99 lines), but not the
cross-product: getinfo / getlocal / traceback / sethook on a
coroutine, hook counts (printed as a number instead of one line per event —
the count itself is a strong check on the line table and the call sequence, and it
pins the tail call hook event), registry / uservalue / setmetatable, and the
rest of the coroutine state machine.
It found one thing: _LOADED has to be a field of the registry.
#define LUA_LOADED_TABLE "_LOADED" /* lauxlib.h */package.loaded is not stored somewhere else with a copy in the registry — it
is the registry field, so debug.getregistry()["_LOADED"] must return it. We
kept it in a LuaState field only, so type(debug.getregistry()["_LOADED"])
gave nil where the reference gives table. Now it is read back out of the
registry, so a script that swaps _LOADED swaps ours too.
208 lines on one axis that no earlier matrix had touched: error(msg, level) for
level from -1 to math.maxinteger, error objects of every type (and what
print / tostring / string.format("%s") do to them), the luaL_where
position prefix in every syntactic position that can raise (table constructor,
numeric and generic for, all twelve metamethods, a gsub replacement, a sort
comparator, a coroutine, <close>, a method, an __index chain, a tail call, a
loaded chunk with = / @ / string names), the shape of xpcall, errors
inside hooks, and chunks dumped with strip.
It found two things, both in luaT_objtypename:
const char *luaT_objtypename (lua_State *L, const TValue *o) {
Table *mt;
if ((ttistable(o) && (mt = hvalue(o)->metatable) != NULL) ||
(ttisfulluserdata(o) && (mt = uvalue(o)->metatable) != NULL)) {
const TValue *name = luaH_getshortstr(mt, luaS_new(L, "__name"));
if (ttisstring(name)) /* is '__name' a string? */
return getstr(tsvalue(name)); /* use it as type name */
}
return ttypename(ttype(o)); /* else use standard type name */
}__name is a C string. getstr() hands back a const char * and
luaG_typeerror interpolates it with %s, so it is cut at the first \0:
__name = "A\0B" reports attempt to perform arithmetic on a A value. We
returned the whole byte array. (The same rule was already applied on
luaL_tolstring's path — this was the second copy of the bug.)
Only tables and (full) userdata consult __name. Everything else falls
straight to ttypename, type metatable and all: after
debug.setmetatable(nil, {__name = "NilT"}), nil + 1 still reports
... on a nil value. Our lookup went through luaT_gettmbyobj, which also
returns the type metatable — so we reported NilT.
Four more things the sweep pinned down, none of which were bugs:
debug.tracebackreturns a non-string, non-nilmessage untouched (ldblib.c: db_traceback), soxpcall(f, debug.traceback)with a table error object yields the table, not a traceback. Andlua_tostringdoes not honour__tostring— which is exactly whylua.c'smsghandlerhas to callluaL_callmeta(L, 1, "__tostring")by hand.- The line hook fires before the first instruction of the line, so
print(pcall(f))written on the same line does not protect the hook's error; the call hook fires forprintandpcallthemselves. xpcalldrops the handler's second result:ldo.c: dothecallcalls it asluaD_callnoyield(L, L->top - 2, 1)—nresultsis hard-coded to 1.coroutine.wrap's error carries two position prefixes;pcall(error, "x")has none (a C function has no line); a library error does have one, becauseluaL_error'sluaL_where(L, 1)asks for the caller.
One axis, one rule: a Lua string is a byte string, but every C API that takes a
const char * — strlen, %s, strstr, strchr, strcmp — stops at the
first NUL. 88 lines, and it found thirteen divergences, all of them the same bug
in thirteen different doorways:
cut at the first \0 |
chunk name (luaS_new), module name and package.loaded's key, all four searchpath arguments, io.open / io.lines / os.remove / os.rename / os.getenv filename, fopen's mode, debug.getinfo's what, debug.sethook's mask, luaL_traceback's message, luaL_checkoption (collectgarbage), string.pack's format, and __name |
|---|---|
| not cut | string.format("%s") (goes through luaL_tolstring, which carries the length), error's message (lua_concat), os.date (it iterates s + slen), table keys, gsub, upper |
| rejected instead of cut | %s with modifiers, and string.pack's z — luaL_argcheck(L, l == strlen(s), arg, "string contains zeros") |
Two of them are worth calling out.
os.remove("/x\0y") crashed the whole process. Paths.get throws
InvalidPathException, which is a RuntimeException — not a LuaError, so
pcall cannot catch it (only LuaError is) and it escapes all the way out.
Truncating to C-string semantics removes the case entirely; the catch now also
covers RuntimeException as a backstop.
__name was the second copy. luaL_typeerror and luaG_typeerror both go
through luaT_objtypename, but typeArgError had its own metafieldOf(v, "__name") lookup — which got both rules wrong (no truncation, and no
table/userdata restriction). It now calls LuaTm.objTypeName like the other two.
The two debug ones are the subtlest: debug.getinfo(f, "S\0ln") must not
report invalid option (C stops scanning at the NUL, so it never sees l and
n at all), and debug.sethook(f, "l\0c") installs only the line hook while
"c\0l" installs only the call hook — the counts come out inverted.
Auditing every remaining .toJavaString() in src/main turned up three more
doorways — load's mode (checkmode compares const char *), loadfile's
filename, and the filename handed to io.output / io.input. So
load(src, name, "t\0b") has mode "t", and load(src, name, "b\0t") tries to
undump a text chunk and reports attempt to load a text chunk (mode is 'b').
It also found one place where C and Java simply disagree about a path:
loadfile(""). C's fopen("") fails with ENOENT, giving
cannot open : No such file or directory; Java resolves the empty path to the
current directory and fails while reading, giving Is a directory — which we
turned into Input/output error. The empty path is the only case where the two
judge a path differently, so it gets ENOENT by C's rule.
Directories were claimed here as "fine, must not be fixed" — and that claim was
wrong, which is the point worth keeping: it was written from reading the C
source, not from running the reference. loadfile("/tmp") does succeed at
fopen in C and fail at read — but this implementation was reporting
cannot open /tmp: Input/output error, i.e. the other branch, because Java's
Files.readAllBytes throws at open. Measuring it (see The directory bug) turned
one wrong sentence into a real bug across five entry points.
54 lines on lua_load's lua_Reader, which nothing had swept: one shot versus
one byte at a time versus arbitrary chunk splits, empty strings are not the
end (Zio asks again; only nil terminates, so {"", "", src} compiles as
src while {"", ""} compiles as an empty chunk), a reader returning
true / a table / a function terminates the load, a reader that raises
propagates even when half the source is already consumed, a reader cannot
yield (lua_load uses luaD_callnoyield), and load — unlike loadfile —
does not eat a BOM or a #! line (that belongs to the loading layer, not the
lexer).
It found nothing, which is itself the point: the reader is where a byte-stream translation is most likely to lose or duplicate a byte.
Coroutines (99 lines): which boundaries yield may cross, the semantics of
close, how errors propagate, the four states, and nested resume. The split,
measured and matching:
may yield |
pcall / xpcall, numeric and generic for (custom iterators too), gmatch, __index / __newindex / __add / __lt / __concat / __len / __call / __close |
|---|---|
| may not | a table.sort comparator, a string.gsub replacement function, __tostring inside print — attempt to yield across a C-call boundary |
It also pins down that a coroutine is dead (not suspended) after yielding
inside __eq, that closing yourself reports cannot close a running coroutine,
that <close> variables are closed in reverse order of declaration, and that
an error raised inside __close replaces the original error.
Syntax and scope (121 lines): sixty-odd fragments pushed through load(), so
compile-time errors are part of the comparison too — goto into a local's
scope, duplicate labels, assigning to a <const>, break outside a loop,
malformed numerals and escapes, the various shapes of function names. Nothing a
generated program will ever produce. The runtime half covers "the right-hand side
is evaluated first" in multiple assignment, table-constructor evaluation order and
key overwriting, varargs with nil mixed in, malformed uses of :, upvalue
sharing, repeat's condition seeing locals declared in the body, and operator
precedence and associativity.
debug and utf8 (114 lines): every what letter of getinfo alone and in
combination, out-of-range getlocal / setlocal / getupvalue / setupvalue,
all four sethook masks, the shape of traceback (not compared wholesale —
it contains chunk names and addresses), and the boundary cases of all five utf8
functions.
Numeric for and coercion (551 lines): all four forprep / forloop
branches (integer up/down, float up/down), including maxinteger bounds,
strings as limits, a zero step, and float steps; plus //, %, bitwise
operations and string coercion. Three things worth remembering: in
for i = 0, 1, 0.5 the first value is an integer 0 (with 1.0 as the
limit it is float throughout); bitwise operators do not coerce strings
("10" // 2 is 5 but "10" & 1 errors); and for i = 2^53, 2^53+2 /
for i = 1e300, 1e300 never advance under float precision — the probe has a
20000-iteration guard so both sides bail out at the same place.
This one is different from every sweep above: those look for divergences inside
functions I already wrote. Cross-checking base_funcs[] in lbaselib.c against
BaseLib.open line by line found that warn was not implemented at all —
with 828 tests green, because not one of them ever called it. A behaviour matrix
is blind to an absent function by construction.
So there are now two things: the implementation, and a permanent gate that
compares the symbol set (190 lines: every global, every key of all nine
libraries, the string metatable, package.loaded, each printed as name type).
Values are not compared — they are addresses, and after normalisation nothing is
left, which would quietly turn the test into the empty case assertFullMatches
already rejects.
The same audit caught _VERSION, a second whole gap. It is "Lua 5.4", not
"Lua 5.4.7" (LUA_VERSION has no release number; LUA_RELEASE is what
lua -v prints). It matters more than it looks: real projects probe
if _VERSION == "Lua 5.1" then …, so a missing _VERSION takes the else
branch — silently wrong rather than loudly broken.
The third hit, arg, is not a library gap: lua.c builds it in createargtable
after luaL_openlibs. Since the differential driver already imitates lua.c's
shape (pmain, msghandler, the extra stack slot), it now imitates this too —
otherwise no script that reads arg[1] can ever be compared.
Implementing warn meant porting lauxlib.c's three-state machine, and the
probe caught three things that a warn("@on") test can never reach:
| rule | symptom if missed |
|---|---|
checkcontrol requires tocont == 0 — so warn("@on", "x") prints @onsecond literally |
"on" swallowed as a control message |
only OFF and ON consult checkcontrol; warnfcont writes straight through — warn("a", "@on") prints a@on |
a second Lua warning: prefix inside one message |
the message is a const char * and lua_writestringerror writes bytes |
warn("@on\0junk") stops turning warnings on; warn("\255") comes out as 0xC3 0xBF |
That last pair is the NUL rule and the byte rule from earlier sections, arriving together in one function — which is the reason to sweep by contract rather than by example.
The gate had a hole of its own: it compared the io table, so it never saw the
seven file methods — close, flush, lines, read, seek, setvbuf,
write. They live in getmetatable(io.stdout).__index (alongside __gc,
__close, __name = FILE*, __tostring), not in io. A symbol audit has to
follow __index to be worth anything; it now does.
Probing those methods then turned up a real divergence:
fwrite(s, 1, 0, f) returns 0, which equals l, so writing zero bytes always
succeeds — f:write("") on a read-only file returns the file itself rather
than nil, "Bad file descriptor", 9. Our write checked !writable before
looking at the length.
Only one cell of the grid exposes it, which is the argument for sweeping grids rather than cases:
| writable file | read-only file | |
|---|---|---|
f:write("") |
file (both) | file / nil, msg, 9 |
f:write("a") |
file (both) | nil, msg, 9 (both) |
Read is asymmetric and was already correct — and must stay that way: f:read(0)
does not go through read_chars but through test_eof (getc then ungetc),
so it does fail on a write-only stream, and read short-circuits on
done >= len before it ever checks readable.
Also measured and matching: flush, setvbuf and seek all succeed on a
read-only stream; io.stdout:close() gives nil, "cannot close standard file".
The f:write("") bug was a return-arity bug, and it was found by hand. So
there are now two mechanical sweeps that apply one uniform input to all ~130
library functions and compare success/failure, the type of every return value,
and the error text:
- zero arguments (~120 lines) — the "missing argument" path;
- one wrong-typed argument (
{},nil,1; 359 lines) — the "type error" path, which is a different C function inlauxlib.c(luaL_check*versusluaL_opt*treat absence differently).
The zero-argument sweep found nothing new. The wrong-argument sweep came back clean too. They are kept because they pin ~370 call shapes at once: change any error text, any result count, or drop one argument check and they go red, in corners no hand-written matrix visits.
Four things have to be excluded or normalised, each learned the hard way:
| why | |
|---|---|
os.exit |
really kills the process |
io.*, dofile, loadfile |
touch the default streams or read stdin, which is not the same on both sides |
debug.debug() |
enters the interactive prompt (lua_debug>) and reads commands |
math.atan / math.atan2 |
the same C function ({"atan", math_atan} and {"atan2", math_atan}), so which name findfield reports depends on lua_next order — and the string-hash seed is randomised per process. Six reference runs: three atan, three atan2. The reference disagrees with itself; the name is normalised before comparing. |
Values are never printed, only type(): os.time(), os.date() and
math.random() differ on every run otherwise.
The existing hook tests were one mask per test; this one is the full grid — eight masks, three counts, four bodies, plus per-thread installation — because several semantics only show up in a cell:
makemaskusesstrchrand knows onlyc/r/l:"x"and"CC"are empty masks (and it is case-sensitive). Reading only"crl"never reveals this.lua_sethookturns the hook off entirely whenmask == 0— sodebug.sethook(f, "")installs nothing anddebug.gethook()returnsnil, rather than a live hook that never fires. Likewisedebug.sethook(f, "x").countjoinsLUA_MASKCOUNTonly when> 0:0and-1produce no count events, while""with count1is count-only.- hooks are per thread: a line hook on the main thread never fires inside a
coroutine, and
debug.sethook(co, …)is that coroutine's alone. gethook's mask string excludes count (unmakemaskemits onlyc/r/l); count is the third return value.
Two shape constraints matter as much as the assertions. Events are collected and
printed at the very end — with hooks on, every print produces its own
call / return pair and drowns the result. And the hook body spans several
lines and calls a function: if allowhook re-entry protection were broken, this
would be infinite recursion rather than a comparison failure.
It came back clean.
In 5.4 a "number" is one of two subtypes, and math.type can see the
difference. This is the single most likely thing to get wrong when porting 5.4:
a "everything is a double" implementation runs most scripts fine and diverges
quietly in //, %, string.format("%d") and math.tointeger. So every door
in the standard library that can hand back a number is swept, comparing only the
subtype — values are never printed (os.time and math.random differ per run).
156 lines, in two halves: everyday shapes (the math library's
"integer in, integer out; float in, float out", four ways of writing tonumber,
the bitwise operators, every string.unpack width, io's seek, os.date's
fields) and the boundaries (maxinteger + 1, mininteger // -1,
math.fmod(1, 0) and 1 // 0 raise rather than returning nan,
math.floor(-0.0) is an integer, math.tointeger(2^63) is nil).
Every line is wrapped in pcall — that is how the two raising cases were found.
Without it the script dies at the first one and the remaining hundred lines are
never compared, while the test still looks green.
The first version of this sweep had a bug worth recording: merging the two probe
scripts left the second half calling the first half's helper, which took a
value instead of a thunk — so 72 of 156 lines silently read tag nil and the
test was green anyway. A matrix whose lines are all nil proves nothing;
check the reference output for uniformly-empty lines before believing a clean run.
io.popen, io.tmpfile and setvbuf were the thinnest part of the io library —
one or two cases each. Four scripts (read/write pipes, mode validation, tmpfile,
every setvbuf mode) widened that, and the very first run bit:
e3 true nil exit 3 <- reference
e3 true false exit 3 <- ours
luaL_execresult pushes luaL_pushfail when the child exited non-zero, and in
5.4 that macro is literally lua_pushnil
(lauxlib.h: #define luaL_pushfail(L) lua_pushnil(L)) — it has been since 5.4
split "fail" from false. This was invisible for the whole project because
os.execute has its own copy of the logic and got it right; only io.popen's
teardown goes through the shared execResult, so popen was the only door that
could show it. Two implementations of the same helper means the bug hides in
the one nobody tested.
Three things here are measured as uncomparable, and each was measured rather than assumed away:
- A child's stdout / stderr has to be redirected inside the command. The
inherited descriptors do not end up in this implementation's capture buffer (a
Java child inherits the JVM's real stderr), so the two sides cannot line up.
Hence
cat > /dev/nulland2>/dev/null, notcat. - Closing an undrained pipe is a race. The reference gets
nil, "signal", 13(echodied of SIGPIPE); we gettrue, "exit", 0. Which one you get depends on whether the child finished writing first — it is not stable run to run and must not be compared. setvbufwith a negative or huge size is libc-defined. C hands the size to libc as asize_tand whether the buffer can be allocated is libc's call: on the same machine2^31fails while2^40succeeds, and"no"with-1succeeds while"line"with-1fails. Not modelable on the Java side.
The two earlier sweeps over every library function asked two questions: does the
function exist (the symbol audit), and what does it do when you hand it the
wrong arguments (the arity and wrong-type sweeps). Both go through
luaL_check* / luaL_opt* — the checking branch. Neither touches the code that
runs after the checks pass. A string.rep that computes the wrong thing would
have stayed green under both.
So there is now a third matrix: hand every library function a set of valid
arguments and compare the return values, not their types. 209 lines in three
scripts — string and math (68), table/utf8/os/coroutine (44),
io/debug/the globals (97).
Values are described by a d() that keeps the comparison deterministic without
weakening it: a number carries its math.type subtype, a string carries its
length, a table is expanded two levels with keys sorted (so neither pairs order
nor an address can leak in). Only a handful of things compare type instead of
value — os.time, os.clock, collectgarbage("count"), and tostring of a
table.
Two results worth having: seeded math.random matches (the xoshiro256**
sequence is right, not just its range), and getmetatable("") dumps the entire
string metatable in one line — __index and all seventeen methods.
The matrix's sensitivity was measured, not assumed. Breaking one line of
string.upper (shifting 'a'…'z' to 'B'…'Z') turns the run red at
s.upper [2] q(3)[Abc]. A matrix that cannot fail is worse than no matrix, and
the only way to know is to break something and watch it go red.
209 lines, byte-identical, first run. Which is also the suspicious outcome — see
the note above about matrices whose lines are uniformly nil. Here 205 of 209
lines carry real values, so the matrix is not vacuous.
l_checkmode accepts [rwa]%+?b* — and the tail is strspn(mode, "b") == strlen(mode),
not "at most one b". So "rbb" is valid while "rb+" is not, and "r b", "br",
"R", "r\t" all fail. 34 mode strings × (read / seek / write / position / close /
resulting file content) — 79 lines.
It found three divergences. All three are stdio's call, not Lua's, and all three were measured before being set aside:
- On an update stream, writing straight after reading — with no intervening
positioning call, which ISO C leaves undefined — lands wherever C's read-ahead
buffer left the physical offset. Measured: a 12-byte file writes at 13 (EOF); a
6002-byte file writes at 4097 (
BUFSIZ+ 1); butread(5000)writes at exactly 5001, because a large read bypasses the buffer. The matrix now puts an explicitseekbefore every write, which sidesteps the UB path — and on the well-defined path (read → seek → write) the two agree byte for byte. - Where the position lands after writing to an append file.
seek("set", 0)then write one byte: the reference reports 1 (base + n), we report 13 (EOF + n— ours is the POSIX answer). Measuring further showed it is buffer- defined too:n ≤ 4095givesbase + n,n ≥ 4096givesEOF + n. The matrix flushes before reading the position, which collapses it onto the one POSIX- defined answer so the two sides are comparable. The bytes written are identical either way — only whatseek("cur", 0)reports differs. seekon a standard stream depends on whether the real descriptor is seekable: 0 when redirected to a file or/dev/null,"Illegal seek"on a pipe or a tty. This implementation has no access to that descriptor, so it is not compared.
Two of the three are the same shape as the setvbuf sizes: C hands the decision to
libc and there is nothing sensible to model. The difference is that here the
observable is not just an errno but a file offset — which is why they are written
down rather than merely excluded.
The package gate used to exercise only the pure-Lua half of the searcher chain,
by populating package.preload in memory. PackageLib was still the second-
thinnest class by coverage, so there is now a matrix that writes real modules to
disk and makes require find them: 57 lines covering what a module may return
(nil, false, true, a number, nothing, multiple values), a module that raises, one
with a syntax error, one that sets package.loaded itself, dotted names turning
into directories, the cache on a second require, a circular require (the
reference gives C stack overflow, not a sentinel), package.loaded pre-set to a
value / false / nil, package.preload given a function / a function returning
nil / a raising function / a non-function, searchpath's sep and rep, and the
searchers chain — its length, each entry called directly, and the whole chain
replaced.
One thing it pins down that would otherwise rot quietly: every
module 'X' not found message prints the entire candidate list, so the default
package.path has to match the reference binary's compiled-in LUA_PATH_DEFAULT
character for character.
57 lines, byte-identical on the first run — so, again, sensitivity measured rather
than assumed: changing searchpath's default sep from "." to "-" turns it
red at sp_dots.
Picking a target by class-level miss ratio only goes so far — after the mode and
require matrices the top holes barely moved, because a class's uncovered
instructions are mostly catch (IOException e) handlers that no Lua program can
reach. So the next pass went one level down: open
target/site/jacoco/<pkg>/IoLib.java.html (the XML report has no line detail, the
HTML does), collect the class="nc" line numbers, and ask of each one "can a Lua
script reach this?"
Of 63 uncovered lines in IoLib, most are genuinely unreachable error handling.
The rest were reachable behaviours that had simply never been run — 39 lines:
tostringof a closed handle is"file (closed)"— notio.type's"closed file"; two different strings, and only one of them was being tested.io.lines()with no argument, and with an explicitnil— these go through the registry'sIO_INPUT, a different path fromio.lines(filename).- Every shape of
f:read("n"): exponents (1e5, and1e999givinginf), hex (0x10,0x1p4,0x.8p1), a leading zero (0777— decimal, not octal), truncated prefixes (0x,1egive nil), and an over-long digit string (nil). - Closing the file during
linesiteration: the iterator then raises"file is already closed". - A large write (5000 bytes), which takes the branch that bypasses the write buffer and goes straight to the file.
All 39 came back byte-identical — those branches were correct but never
executed. Their value is as a regression net: change read("n")'s prefix
handling or the write-buffer threshold and this goes red immediately.
Reading PackageLib's uncovered lines and asking "can a Lua script reach this?"
led to searcher_C's fallback and, from there, to a module name that collides
with a directory. That turned out to be one rule mis-modelled in five places:
POSIX lets open(dir, O_RDONLY) succeed. fopen(dir, "r") therefore succeeds;
fopen(dir, "w") gives EISDIR; and the first fgetc / fread fails with
EISDIR. The JVM refuses to open a directory at all, so this implementation got
all five entry points wrong in one consistent way:
| reference | before | |
|---|---|---|
loadfile(dir) |
nil, "cannot read <p>: Is a directory" |
nil, "cannot open <p>: Input/output error" |
io.open(dir, "r") |
a file handle | nil, "<p>: No such file or directory", 2 |
io.open(dir, "w") |
nil, "<p>: Is a directory", 21 |
nil, "<p>: No such file or directory", 2 |
package.searchpath |
finds it (returns the path) | nil, "no file '<p>'" |
require(name) |
error loading module … cannot read … |
module '…' not found |
The fix is one idea applied at four sites: openInto lets a read-only open of a
directory through (and reports EISDIR for anything writable), LuaFile.getc /
read fail with 21, seek / tell treat it as a zero-length file (a directory
descriptor is seekable — f:seek("set", 5) returns 5, and f:seek("set", -1)
is still EINVAL), PackageLib.readable accepts directories, and loadfile /
searcher_Lua take the cannot read branch instead of cannot open.
18 lines, byte-identical after the fix. Two things about how this was found are worth more than the bug:
- The README previously said directories were "fine and must not be fixed" —
written from reading
lauxlib.c, not from running the reference binary. It was wrong. Measure, then write. - The same sentence in the code comment (
BaseLib) had asserted the behaviour was already correct. A comment agreeing with the code is not evidence.
LuaAux still had 15 uncovered lines, and most of them were errnoOf / strError
branches that no Lua script had ever reached — ENOTEMPTY, EACCES, and friends.
22 lines aimed at them found five divergences, four of them real bugs:
ENOTEMPTY's number is chosen by the OS, not by libc: 39 on Linux, 66 on macOS/BSD. It was hard-coded to 39, soos.remove(nonempty_dir)had the right message and a number 27 off. It now comes fromos.name.strErrormust know both numbers. Fixing only the number turned the line into "errno 66 correct, messageInput/output error" — the numeric constant and its text have to move together.os.remove("")gaveEIOwhere C givesENOENT— the same "C and Java judge a path differently" corner asloadfile("").os.rename's whole shape table was wrong. File → existing directory isEISDIR(21); directory → existing file isENOTDIR(20); directory → empty directory succeeds (it replaces); file → existing file also replaces. Every "target already exists" case was reportingEEXIST, because the code calledFiles.movewithoutREPLACE_EXISTING.
(os.remove("/etc") used to be skipped as machine-dependent — it is EPERM
here because of SIP. It is no longer skipped: see The errno shape table below,
where the whole class of cases became derived instead of hard-coded, and
therefore portable.)
The general lesson is the one from the directory bug again: "the value differs per platform" is a reason to derive it from the platform, not to leave the case untested — skipping it is what let a plain wrong constant survive.
Undump had 11 branches no script had ever reached — truncated chunk, integer
overflow, bad format for a constant string, version / format / size mismatch.
Hand-picked bad samples only cover the positions you think of, so this sweep is
byte-by-byte mutation instead: every truncation length, every byte position
mutated to each of four values, varint-limit injections, and trailing garbage,
run over the plain dump, the stripped dump, and a dump with upvalues and nested
protos. 865 lines, 21 distinct outcomes, 542 lines that are not ok. All 11
branches come out, two of them in places no hand-picked sample would have hit.
Two traps first, because either one silently voids the whole sweep:
string.dumpoutput carries the source name, and the source name defaults to the script path. The reference runs/tmp/x.lua, ours runstarget/rt/case.lua, so the dumps differed by 7 bytes and every offset shifted — the first run showed 40 "divergences" that were all this one cause. The probe pins the chunk name to"=X", after which the two dumps are byte-identical.- The varint-limit injection has to land after
nupvals. The header is 31 bytes, not 30; byte 32 is the upvalue count.
The sweep found exactly one real divergence, and it was a branch we invented:
loadSize'sv > Integer.MAX_VALUE → integer overflowdoes not exist in C. The old comment asserted that "C would allocate and raiseLUA_ERRMEM" — a conclusion read off the C source, and measured to be wrong: with a source-name length of2^35-1(varint7f 7f 7f 7f ff) the reference reportstruncated chunk, not a memory error. malloc of 34GB succeeds on 64-bit, so C goes ahead and reads those 34GB and hits end-of-stream. The branch is gone; long strings are now accumulated while reading — how much is buffered depends on how many bytes the stream actually has, not on how many it claims, and hitting the end still reportstruncated chunk.
The reverse check afterwards — swapping loadUnsigned's limit >>>= 7 for the
Long.MAX_VALUE >>> 7 its own comment warns about — turned up a second defect
in passing:
loadCode'snew byte[n * 4]can overflow to a negative number and throwNegativeArraySizeException— a Java exception, whichpcallcannot catch, so one malformed chunk takes down the host process. It also paid twice: C allocatesninstructions, this allocatednplus another4nbytes, i.e. double whatever an untrusted input asked for. Reading instruction by instruction from a reused buffer removes the overflow and allocates onlyn. The same sabotage re-run afterwards: no crash, and 13 red lines — the matrix is sensitive to errors this small.
The lesson is the standing one: a comment agreeing with the C source is not evidence. The more plausible that comment looked, the easier it was to read the mismatch as a deliberate deviation — and a branch that exists nowhere in C stayed quiet until a machine checked it.
Picking targets from uncovered lines has a trap, and this is where it surfaced:
JaCoCo marks the line that calls an always-throwing method as not covered.
The instruction after the call — the pop that discards the return value — is
never reached, so the whole line is recorded as nc.
Measured: Undump's ten error() branches fired 848 times during the suite
(counted by adding a print inside error()), every line inside error() is
fc, and all nine call sites are nc. Following the report would have meant
writing more cases for code that was already covered.
The test: the line is an xxxError(...) call or starts with throw, and the
method it calls is itself covered — that is a fake gap. Filtering on it makes
Codegen's 15 lines, Parser's 21 and Undump's 10 disappear entirely;
what is left is real.
Of the real gaps, LuaTable had 16 lines, all paths that only run for narrow key
ranges: hashFloat's -0.0 normalisation, landsInArray's float branch, rehash
when the hash part needs more slots. A handful of hand-picked keys never reaches
them, so this is a key × key square: 20 keys each written once, then read back
with all 20, plus 2048 integers on each side of math.maxinteger swept by step.
It turned up exactly one real divergence, and it is not in LuaTable:
rawequal(math.maxinteger, math.maxinteger + 1.0)should befalse. The two numbers are genuinely different (math.maxintegeris 2^63−1, the right side is the float 2^63), but(double)(2^63−1)rounds to2^63, so "cast both to double and compare" calls them equal. Lua 5.4 compares two numbers of different variants the other way: convert both with F2Ieq and compare integers; if the float side does not convert, they are not equal. The fix is inLuaValue.numberEqual, which backs==,rawequal, and the table's key comparison — sot[math.maxinteger]andt[2^63]were previously the same entry.
The duplicatefloatToIntExactinLuaTableis now folded intoLuaValue, so the two copies of that rule cannot drift apart.
The two rules only diverge over roughly the thousand integers before
math.maxinteger, which is why the stepped sweep at the end of the script
matters: without it the fix is worth 6 lines; with it, reverting turns 77 red.
next's order depends on the hash function and need not match, so key sets
are always sorted before printing — the set is compared, not the order.
After filtering out the fake gaps, DebugLib still had 22 lines, all paths that
only run when the stack has a particular shape: currentLine returning -1 for
non-Lua frames, funcNameFromCallInfo's ci.hooked branch, the
getinfo-with-a-function (rather than a level) path, and traceback's
(...tail calls...). Two hand-picked examples never reach them, so this is a
frame-shape matrix: top level, Lua frames, C frames (looking outward from
table.sort and pcall callbacks), inside and outside a coroutine, a tail-call
chain, and a long function under a line hook. 106 lines.
Two things worth recording:
- The hook frame is at level 3. Inside a hook, level 1 is the
pcallwrapping the call, level 2 is the printing helper, and the hook's own frame is level 3. Stop at level 2 and thenamewhat = "hook"branch is never reached — and on the report it looks exactly like every other uncovered line. - One thing is not compared: the
"(C temporary)"of the outermost C frame is whatever the host pushed when callingpmain. The reference pushes(argc, argv), so it reads back2and a userdata; this driver pushes the script text. That is the host's calling convention, not Lua semantics, so the script only goes up to level 2.
Result: 106 lines match byte for byte, no real divergence, and three never-taken
paths are now covered — getupvalue on a C closure, getinfo with a function
plus a thread, and traceback's tail-call marker. What remains uncovered is
defensive (HOOKKEY missing, hook not a function) plus one line that only a
host can reach: debug.gethook's "external hook" — sethook from Lua always
installs HOOKF, so no script gets there.
Every entry point that C drives from inside a callback has its own nny
(C's nyci) bookkeeping, and a single missing +1 is invisible until something
yields at exactly that spot. So this is a matrix over where the yield
happens: pcall, xpcall, xpcall's message handler, table.sort's
comparator, string.gsub's replacement function, load's reader, eight
metamethods (including __close), all four kinds of generic-for iterator, a
debug hook, coroutine.wrap, a nested coroutine, resume on a dead and on an
errored coroutine, and a suspended coroutine resuming itself. Each row also
asks coroutine.isyieldable() separately, so "may I" and "what happens if I do"
are two measured answers rather than one. 67 lines.
It found one real divergence, and it is worth stating precisely because the wrong behaviour looks more capable, not less:
run xpcall_msgh true false error in error handling <- reference
status xpcall_msgh dead
run xpcall_msgh true in msgh <- this implementation
status xpcall_msgh suspended
LuaState.applyErrMsg was calling the handler with the yieldable pcall;
C uses luaD_callnoyield. The reason matters more than the symptom: being able
to yield inside the error-handling continuation means an
"error half-delivered" intermediate state can be suspended — and C has no
such state. Fixed with callNoYield plus a catch (LuaError) that stops
backstopping the handler's own failure, which is what produces
"error in error handling".
Two more things this matrix pinned down:
pairsorder over string keys is randomly seeded, so it cannot be compared. String keys go throughluaS_hash, whose seed comes fromlstate.c: luai_makeseed—time(NULL)mixed with three ASLR addresses. Running the same script against the reference five times gavein pairs:a,a,b,a,b. Any probe that prints raw key order is a fake-red generator; this one drains the coroutine to death and prints the collected yields sorted instead. (This implementation has no seed at all, so its order is fixed and deterministic — a deliberate, safe asymmetry: the reference's own order is not observable either, since nothing can depend on a value that changes every run.)load's reader must terminate. A reader that never returnsnilmakesloadread forever — the first version of this probe hung the reference binary and had to bepkilled.
Reverse verification: swapping callNoYield back to call turns exactly two
lines red, and the test fails.
<close> is the only new syntax in 5.4 and it has the most edges, so this is a
matrix over how the block is left (falling off the end, nested, return,
break, goto, error with a string and with a table) crossed with what
__close itself does (log, raise, raise a non-string, return several values,
yield, be a number, be a callable table, be absent), then crossed again with
pcall / xpcall / coroutines / coroutine.close. 56 lines.
It found two divergences with one root cause: an error raised inside __close
must go through the message handler too.
msgh orig | close a err=H | msgh E:a | xpcall -> false H <- reference
msgh orig | close a err=H | xpcall -> false E:a <- this implementation
luaD_pcall restores L->errfunc on its last line, and
luaD_closeprotected runs before that line — so the handler is still installed
for the whole time __close runs, and error() inside it reaches
luaG_errormsg just like any other error. The practical consequence is easy to
miss: the final result is the handler's return value, not __close's error.
The same fix produces "error in error handling" in the case where the handler
itself raises — that string comes from the __close error making a second pass
through a handler that is already failing.
The second divergence is the coroutine side of the same rule:
finishpcallk restores L->errfunc after luaF_close, so the handler must
run before the closing methods, not after. Getting this backwards shows up as
close a err=after | msgh h2 after instead of msgh h2 after | close a err=H-h2
— the closing method sees the raw, unhandled error.
Moving that call earlier exposed a second, independent trap: the handler must be
called before top is pulled back to fIdx + 1. luaG_errormsg pushes its
call frame at the current top of the stack, where the protected function's
locals still live; pull top back first and it starts writing at fIdx + 2,
which is exactly the first local slot — the <close> variable — so __close
gets overwritten with the handler itself and the next call reports
attempt to call a nil value.
One combination is deliberately not in the matrix: a message handler that
raises. luaG_errormsg recurses there until the C stack overflows, producing
several hundred lines whose count varies with stack depth. The matrix pins the
result only.
Hooks are the one place where this implementation interrupts the VM's normal
flow with a Java exception, so events are emitted from three different points
(precall, poscall, the instruction loop) with allowhook, nCcalls and
ci each in a different state. This is a matrix over the eight mask
combinations (c / r / l and their unions) crossed with count
(0, 1, 2, 3, 5), crossed with frame kind (Lua, C, tail call, coroutine), crossed
with what the hook itself does (disable itself, swap itself, raise, raise a
non-string, yield, read debug.gethook). 56 comparable lines.
Result: byte for byte identical, no real divergence. It is still worth
keeping, because it is measured to be sensitive — renaming the "tail call"
event to "tailcall" turns 14 lines red.
The interesting thing it pinned down is not a bug but a trap in writing such probes:
- Whether a hook error can be caught depends on where
sethookwas called. The very nextcallevent aftersethookis forpcallitself — thecmask counts C functions — and that fires beforepcallhas established its protected region, so the error escapes and kills the script. The reference reportslua: hk1with[C]: in function 'pcall'in the traceback and stops there. To test catching,sethookhas to run inside thepcall; the escaping variant can only go last in the script, or everything after it is silently uncompared while the test stays green. - Reading the stack from inside a hook needs level 3. Level 2 is the
printing helper; level 3 is the frame with
namewhat = "hook".
lua_setmetatable dispatches three ways: tables and userdata store the
metatable on the object, everything else installs it in global_State's mt[]
where the whole type shares one. This matrix walks all eight types through
debug.setmetatable / debug.getmetatable. 48 lines, byte for byte identical,
and it is sensitive — deleting one line of the userdata branch turns 6 lines red.
The reason that branch had never been covered is worth stating: setmetatable
from Lua requires its first argument to be a table —
lbaselib.c starts with luaL_checktype(L, 1, LUA_TTABLE) — so userdata can
only be given a metatable through debug.setmetatable, and no script in the
suite ever did that. (The error message names the type via __name:
bad argument #1 to 'setmetatable' (table expected, got FILE*).)
Two things it pins down:
- Only tables and full userdata honour
__name. Give numbers,nil, booleans, functions, threads or strings a type metatable with__nameand the runtime still saysnumber/nil/boolean/ … ; only the userdata becomesattempt to index a M_ud value. - The string type metatable is the string library. Replace it and
("x"):upper()stops working, so the probe has to save and restore it — and it must save it before the section that installs an empty metatable on every type, or it saves the wrong table and the script dies halfway through.
PackageLib was the thinnest class by coverage (0.89), so this is a matrix over
what loadlib.c actually does with a path: 18 template shapes ("", ";",
"a;", ";;a;;", …) crossed with ? substitution, sep / dirsep corner
cases, embedded NULs, argument checking, all four searchers called directly, and
loadlib. 156 lines, byte for byte identical, and sensitive — dropping the
const char * truncation turns 3 lines red.
It found one real divergence and one wrong constant:
searcher_Candsearcher_Crootdid not cut the module name at the first NUL. In C the name isconst char *name = luaL_checkstring(L, 1), sopackage.searchers[3]("a\0b")searches fora— this implementation searched for the whole string. Only reachable by calling a searcher directly (requiretruncates first), which is why no earlier probe caught it.loadlib's third return value has two constants, not one.ERRLIB(library will not open) givesLIB_FAIL, which is"open"on a dlopen build and"absent"on the fallback build;ERRFUNC(library opened, symbol missing) gives a hard-coded"init"— the fallback branch redefines only the former. Verified against the reference with a real.dylib:package.loadlib(real, "luaopen_nosuch")→nil, <dlsym error>, "init". This implementation'slookForFuncalways returnsERRLIB, so the branch is unreachable here, but the constant is now the one C uses.
Two things the matrix deliberately does not compare:
dlerror()text. It embeds an address;dlsym(0x…, luaopen_x): symbol not founddiffers on every run, so the C-searcher section compares only the part of the message before\n\t.LUA_PATHhandling. The;;→ "insert the default path here" expansion is real and easy to get wrong (only the first;;is replaced, and no extra separators are added around the default). It cannot be driven from this harness — the reference binary's environment is not controllable from JUnit — so it was measured separately withenv LUA_PATH=…over nine shapes (a;;b,;;,a;;,;;b,a;;b;;c,;;;,noMark, empty, andLUA_PATH_5_4taking priority): identical every time.
The matrix did not move coverage at all — PackageLib is still 0.891 with
the same 117 missed instructions afterwards. Those lines were already being
executed; they were never compared. Coverage cannot see the difference, and
"coverage went up" is not the test for whether a gap has been closed.
IoLib.LuaFile was the thinnest class left (0.80), and every line in it is a
stdio rule rather than Lua semantics. This matrix covers read(0), writes that
bypass the 8192-byte write buffer, seek on a pipe, reading a directory, and
io.lines() with no argument. 43 lines, byte for byte identical, and sensitive —
changing the directory errno from EISDIR (21) to EIO (5) turns 3 lines red.
What each group pins down:
f:read(0)returns""before EOF andnilat EOF.freadasked for zero bytes returns zero, which counts as "read something" — so the empty string is a real result, not an error. The mirror case (writing""to a read-only file succeeds) was already pinned earlier; these are the two halves of one rule.- Writes larger than the 8192-byte buffer skip the buffer entirely, so 10000 / 8192 / 8191 bytes are each written once and read back to check the positions.
seekon a pipe isESPIPE(29), not a generic I/O error. Onlyio.popenfiles are used for this — see the warning below.- Directories:
fopensucceeds, reading fails withEISDIR(21), andseekfollows yet another rule — the position is computed normally and only a negative target position isEINVAL(22). io.lines()with no argument reads the registry's default input. Close it and the error isattempt to use a closed file, notno input file: the registry entry is still there, it is merely closed.
Never seek io.stdout / io.stderr in a differential probe. When stdout is
redirected to a file, that seek("set", 0) succeeds, the write pointer goes
back to 0, and every later line overwrites the output from the start — the
reference's output becomes a shuffled fragment of itself and looks exactly like a
bug in the implementation. It only reports ESPIPE when stdout is a pipe. That
depends on how the harness captures output, not on Lua.
File operations do not fail along one axis ("exists or not"); the errno is
decided by the shape of the target. io.open("file/y") is ENOTDIR (20),
io.open("/etc/hosts", "w") is EACCES (13), os.remove(".") is EINVAL (22),
os.remove("/") is EISDIR (21), os.remove("/etc") is EPERM (1) here. This
implementation classified failures by Java exception type and fell back to
ENOENT / EIO for everything else — so five of those were wrong.
The fix is to invert rather than to enumerate a second table: errno and
strerror are inverse functions, and Java's I/O exceptions carry the libc
strerror text verbatim — FileNotFoundException as
"<path> (Permission denied)", FileSystemException.getReason() directly as
"Not a directory". So LuaAux.errnoOf now looks the text up in
LuaAux.strError's table and gets the number back. 44 lines, byte for byte
identical, and sensitive — disabling the lookup turns 9 lines red.
This is also what makes the matrix portable. os.remove("/etc") is EPERM
on macOS (SIP) and ENOTEMPTY on Linux — but both sides now derive their
answer from the same libc, so they agree on each platform instead of one side
carrying a platform-specific constant. That is the whole argument against
"this value varies per platform, so skip it": skipping it is what let a plain
wrong constant survive, and deriving it is what makes it testable.
Two traps while writing the probe, both recorded in the test's javadoc:
- Fixtures go in a subdirectory and are deleted at the end — not scattered over the repository root.
- Never exercise
os.removeagainst a real system file.os.remove("/etc/hosts")would succeed and delete it if the suite ever ran as root. TheEACCEScase uses achmod 555directory the test creates itself.
Utf8Lib was at 0.930, and unlike IoLib/PackageLib its holes are pure logic —
no platform behaviour, no IOException handlers. Reading the uncovered lines found
a literal written twice: open built the utf8.charpattern byte array inline
while charPattern() sat next to it with no callers at all. Two copies of the same
constant is a drift that nothing can detect — the coverage report just shows both
blocks nc. Merging them (and having the matrix actually read
utf8.charpattern) puts the pattern under test.
The 72-line matrix covers utf8.char at every boundary (0, 0x7F, 0x80,
0x7FF, 0x800, 0xFFFF, 0x10000, 0x10FFFF, 0xD800, 0x110000,
MAXUTF, -1, math.mininteger / math.maxinteger, non-integer, string), the
separate value out of range branch that only arguments 2..n take, every
utf8.codepoint bounds shape (i > j returning nothing at all, 0, past the end,
negative, empty string, lax true/false, float / string / boolean arguments), and
charpattern used rather than inspected — gmatch over "a中b\u{1F600}"
yields 4 characters, over "\xff\xfe" yields 0.
Byte-identical on the first run. Utf8Lib went 0.930 → 0.995 with one line left:
throw L.libError("string slice too long"), which needs a string of at least two
gigabytes. It is recorded as unreachable rather than chased.
Parser was the second-biggest hole (169 uncovered instructions) and, unlike
IoLib/PackageLib, its gaps are pure logic. Reading the uncovered lines showed
they were almost entirely error messages — and every single one of them was
nc, meaning no test had ever produced that message at all.
So the matrix is not a hand-picked list; it is an enumeration of error sites in
the C source: error_expected and both shapes of check_match, errorlimit
for locals/upvalues/constructor items, attempt to assign to const variable,
jumpscopeerror / undefgoto / duplicate labels, <name> or '...' expected,
function arguments expected, unexpected symbol, unknown attribute,
multiple to-be-closed variables, cannot use '...' outside a vararg function,
'=' or 'in' expected, control structure too long, plus the whole lexical set
(unfinished string / long string / long comment, malformed number, hexadecimal
digit, missing '{', UTF-8 value too large, missing '}', decimal escape too
large, invalid escape sequence, invalid long string delimiter). 172 lines.
Three things it pins down that would otherwise rot:
<name>and<eof>are printed without quotes, while'end'and','are printed with them —luaX_token2strreturns the bare string fortoken >= TK_EOS. And the token afterneargoes throughtxtToken, which uses the raw source text for strings and numbers, so the message can contain nested quotes:hexadecimal digit expected near ''\x''.- C stack overflow is returned, not raised. Deep nesting trips
luaE_incCstack, butluaD_protectedparsercatches it, soloadhands backnil + "C stack overflow". Onlyload's own argument check (bad argument #2 to 'load') actually raises. The probe wraps everything inpcallso the two are distinguishable. control structure too longneeds a loop body of more thanMAXARG_Bx(262143) instructions, so that case is a generated 1 MB source — 0.02 s on the reference, 0.6 s here, cheap enough to keep.
Byte-identical on the first run, and sensitive: quoting every token name turns 11
lines red, dropping check_match's same-line shortcut turns 13 red, and
errorlimit's "main function" → "main chunk" turns 1 red.
After installing it, Parser's coverage barely moved — 169 → 167, while the
uncovered-line list was character for character identical. That is not
possible if 172 lines of new error paths really ran, and they did: instrumenting
one of them with a println fires, and the byte-identical output can only have
come from those throw sites.
What gives it away is the line above:
<span class="fc bfc" id="L1314" title="All 2 branches covered.">if (!lh.v.isVar()) {</span>
<span class="nc" id="L1315"> throw ls.syntaxError("syntax error");</span>JaCoCo knows the guarding branch was taken in both directions, and the line
number table proves 1315 has instructions of its own — yet the line reads
nc. Parser has 19 throw lines and not one of them is fc (17 nc,
2 pc), while the rest of the project is 136 fc / 41 nc.
The consequence for target selection: an uncovered throw line is not
evidence of an uncovered path. Half of Parser's apparent hole is this
artifact, and chasing it would have burned an afternoon proving correct code
correct. What actually tells you the path runs is the comparison, not the
coverage number — which is the same lesson as "coverage going up ≠ the gap
closed", from the other side.
OsLib was at 0.952, and the existing FORMAT group only exercised the !
(UTC) half. The new 337-line matrix adds the other half: every conversion
specifier in local time, the same set at five timestamps (0, the 2038
boundary, 1900, -1, and year 1), the error shapes for invalid specifiers, and
the corners of os.time / os.difftime / os.setlocale / os.getenv.
It found one real bug: *t was a prefix test, but C has
strcmp(s, "*t") == 0. So os.date("*t x") should fall through to strftime,
where *, t, the space and the x are all literals, and return the string
"*t x" — this implementation returned a table instead.
Fixing it surfaced a second detail, the same trap as everywhere else in this
codebase: strcmp stops at the first NUL, so os.date("*t\0x") is still
the table form in C. Comparing f.length against 2 is not enough; the string
has to be truncated at \0 first.
Sensitivity: deleting the E / O two-character group from
LUA_STRFTIMEOPTIONS turns 38 lines red, reverting *t to a prefix test turns
1 red, and using f.length instead of the NUL-truncated length turns 2 red.
Two things stay out of the matrix on purpose: !%z (macOS's strftime
reports the local offset even under gmtime, glibc reports +0000; this
implementation follows POSIX and can only agree with one of them) and
os.date / os.time with no argument (wall clock).
A function's name is not a property of the function — it is reconstructed from
the call site (funcnamefromcall → funcnamefromcode → getobjname), by
looking at whether the caller's instruction was OP_GETFIELD, OP_GETTABUP,
OP_GETUPVAL, OP_MOVE or OP_SELF and then digging the name out of the
constant table or a register. So the same function has a different name and
namewhat at every syntactic position, which is exactly what a grid is for: 77
lines over nine namewhat values — global, local, field, method,
upvalue, metamethod, for iterator, "", and nil.
Shapes that would not survive being guessed:
t[1]()is named"integer index", not"?". Integer keys are their own branch ingetobjname; only float and boolean keys fall through to"?"."?"still reportsnamewhat = "field"— the name may be unrecoverable while the calling convention is not. The two are independent.metamethodnames are the event name minus the__(index,newindex,add,len,close, …), while__callgoes through a normal call and reportsname="a1", namewhat="local".- When nothing is recoverable,
nameisnilbutnamewhatis the empty string, notnil.
First run byte-identical (bar the script path), so sensitivity was measured:
"integer index" → "?" turns 5 lines red, and changing the case of
namewhat's "field" turns 1 red.
There is no lgc.c here; __gc is reconstructed with Cleaner, so when it
runs depends on the JVM's collection timing, while the reference's timing depends
on its own incremental collector. Neither is guaranteed, so byte-for-byte
comparison is not stable in principle — even the "add __gc after setting the
metatable" rule measured differently for tables and for userdata. The handful of
reproducible shapes already in GcTest (runs once, errors are swallowed, adding
__gc late does not count) is the whole target; no more matrices are built for
it, and an apparent divergence there is not evidence of a bug.
Nothing in the language or the ten standard libraries, and every upstream file
whose output is deterministic now matches byte for byte. The remaining work is
verification breadth: more differential scripts and a wider fuzz corpus. The only
upstream files still off the list are all.lua / main.lua (the driver: it needs
the T module built from ltests.c and writes time.txt), attrib.lua /
files.lua / big.lua (they need the C modules in libs/ and the globals
all.lua sets up — the reference binary fails on them standalone too),
heavy.lua (minutes), and bwcoercion.lua (no output standalone).
bad argument #1 to 'string.rep' (string expected, got boolean) is the message
real projects hit constantly, and every branch that produces it lives in
lauxlib.c. The grid enumerates the functions at runtime (_G plus the nine
library tables) rather than from a hand-written list — a hand-written list cannot
miss a function silently, and "silently" is exactly what happened when warn was
unimplemented for a long stretch with 828 green tests. 108 functions × {0, 1, 2}
arguments × 10 values = 2396 lines, all byte-identical.
Three judgments it pins down:
- A missing argument reports
no value, notnil.luaL_typenameon an out-of-range index givesno value. Breaking it turns 212 lines red. lua_typeon an out-of-range index isLUA_TNONE, notLUA_TNIL, so a parameter declared "nil or table" does not silently accept absence:setmetatable({})must reportnil or table expected, got no value. Only one line in the grid covers it — which is one line more than before.- Method calls subtract one from the argument index (
arg--inluaL_argerror), and if that lands on 0 the message changes tocalling 'x' on bad self (...). A direct call never reaches either branch, so the grid adds a second, literal block of(""):f(v)/(1):f(v)calls (377 lines) —OP_SELFonly comes from a syntactic method call, it cannot be assembled at runtime. Sensitivity 75 and 22.
Two normalisations are deliberate: math.atan and math.atan2 are the same C
function and findfield picks one based on the string-hash seed, which changes
every run; addresses become PTR. The functions on the blacklist are excluded
for side effects, not for difficulty — print and io.write write to stdout,
os.date / os.time / os.clock read the clock, math.random is random,
io.read / io.lines read stdin, debug.sethook leaves a hook installed,
debug.setmetatable mutates a global type metatable, and collectgarbage's
step / count are the documented deviations.
There is a third layer on top of the same grid: what a valid call returns. The
first version printed OK on success and threw the return values away, which
hides an entire class of bug — return-value type. math.abs(-1) returns the
integer 1, not the float 1.0, and tostring prints those as 1 and 1.0.
Routing math.abs's integer branch through the float path leaves the
argument-error grid completely green and turns 6 lines red here; conversely,
breaking math.type's return value turns 2 lines red here and none there. Of the
2729 output lines, 959 are successful calls covering 139 distinct return shapes.
Two functions are excluded because of their return values: os.tmpname is
random, and debug.traceback's return value contains stack traceback:, which
the harness would cut at — silently skipping everything after it.
ldblib.c: db_debug is a loop, and all three of its rules were wrong here before
this grid existed:
- it writes
"lua_debug> "to stderr before every read (no newline). This was missing entirely, so the reference's output had eleven bytes we never produced; - it loops until EOF or a line that is exactly
"cont\n", executing each line as a chunk named=(debug command). The Java version read one line and returned; - a command that fails prints
luaL_tolstring(L, -1)to stderr and the loop continues.
Testing any of this needs stdin, which the differential harness did not have — it
redirects the reference's stdin to /dev/null and leaves LuaState#in as an
empty stream, so only the "print one prompt, then EOF" shape was reachable.
Diff.assertMatchesWithStdin feeds the same bytes to both sides, which brings the
loop body, the error branch, cont, a line with no trailing newline, and the
250-byte fgets buffer split (a long line is chopped into several chunks, each
executed separately) into CI. All six cases now match byte for byte.
One thing worth recording about ordering: C's print goes through
lua_writeline, which calls fflush(stdout), so stdout and stderr interleave in
program order even when both are captured into one pipe — not "all stdout at
exit".
attempt to index a nil value (local 'x') — the parenthesised tail comes from
ldebug.c: varinfo, and none of it is stored anywhere. varinfo first asks "is
this value an upvalue of the running closure?", then "which register holds it?",
then hands that register to getobjname, which guesses from the last
instruction that wrote it. The grid (112 lines) pins every kind: local,
upvalue, constant, global, field, method, (for iterator 'for iterator'), plus the two failure shapes field 'integer index' (OP_GETI, so
the kind is field and the name is the literal string integer index) and
field '?' (a key it cannot recover). The empty tail is part of the answer too:
attempt to call a number value must not grow a (constant '42'), because
kname names string constants only — two rows pin that.
The valuable half of this matrix is what it found by not being sensitive. The
first 21 cases were written on the intuition "a conditional assignment makes the
name disappear": local x = nil; if c then x = nil end; return x.y. All 21 answer
(local 'x'), and rewriting filterpc as an unconditional return pc left all
21 green. The reason is in basicgetobjname: it calls luaF_getlocalname
first, and any register that is a live local at the failing pc returns
"local" on the spot — findsetreg, its jump-target filter and all, is never
entered. Symbolic execution only ever speaks about temporaries.
What does reach it is and / or: in (c and t.a or t.b).z both branches write
the same temporary and both writes sit before the jump target, so the last write
is judged "may not have executed" and discarded, and the tail collapses to the
empty string. Adding those turned the very same sabotage from 0 red lines into
234.
Sensitivity, measured by breaking one line and reverting: filterpc 234,
instack 435, upvalname's "?" 211, kname string-only 131, OP_MOVE's
b < a 22, OP_LOADNIL's register range 10, "integer index" 3, isenv's
global 3, method 2.
One branch is recorded as unreachable rather than quietly assumed covered.
Instrumenting findsetreg over the grid and the 33 upstream files shows which
opcode it lands on: LOADNIL 1080, GETTABUP 494, "not found" 326, GETUPVAL
199, MOVE 52, … and TFORCALL zero. That branch (change = reg >= a + 2)
cannot be observed from an error message: TFORCALL sits at the end of the loop
body, the registers it touches are live locals inside the loop, and outside the
loop they are always rewritten before anything reads them. It is transcribed
verbatim from C; no case in 1032 lines reaches it, and that is a measured fact.
IoLib.LuaFile was the least-covered class in the tree (0.758). The grid that
targets it has nine blocks and prints 1262 lines: the six file methods
(flush lines read seek setvbuf write) over the six open modes over 21 argument
values, then the same again on a closed handle; a two-argument seek
matrix; write-then-seek; read's multi-format forms and lines iteration;
write's multi-argument forms; a directory handle; a second handle opened on
the same path; a failed read("n") followed by read(1); and popen's close.
Two of those blocks exist because the obvious one is blind:
f:seek(-1)never reaches the offset code at all — argument 2 is the option, so a number just meansinvalid option. Onlyf:seek(w, off)exercises negative offsets, and it is worth 46 red lines.seekflushes the write buffer before reporting the position, so nothing distinguishes aflush()that flushes from one that does not. Opening a second handle and reading the file with it does: 4 red lines.
Two real divergences came out of it:
f:read(-1)raisedNegativeArraySizeException. C writessize_t l = (size_t)luaL_checkinteger(L, n)— a negative count becomes an astronomical unsigned value,luaL_prepbuffsizefails to allocate, andresizeboxraises the ordinary Lua error"not enough memory"(false not enough memoryunderpcall).new byte[(int) n]is not the translation of that.read_charsnow reads in chunks until the stream is dry or the count is met — which is closer tofreadanyway, sincefreadkeeps going on a short read — and only refuses when the size is one no system could allocate. The exact refusal boundary is platform-dependent (the reference binary cuts off around 2^46 on macOS, elsewhere with overcommit), so it is deliberately not in the differential table; the deterministic input is the negative one.- A successful
fseekdiscards theungetcpushback.f:read("n")failing on"three"pushes't'back; the followingf:seek("set", 0)thenf:read(3)gives"1 2"in the reference and"t1 "in ours untilseekcleared it.
Measured sensitivity: seek's negative-position EINVAL 46, read's directory
check crashes the whole grid, append not returning to EOF 5, flush not
flushing the write buffer 4, seek not clearing the pushback 3, seek's
directory base 2, write's zero-byte special case 1.
And one honest zero: tell's + wlen is 0 — f:seek calls flushW before
tell, so the buffer is always empty there. Coverage went up; the branch is
still unobserved. Blocks 8 and 9 are likewise coverage-only: removing the
done >= len short-circuit, or not closing popen's in, leaves the grid
green. They exist so JaCoCo has less to lie about, not because they pin a
behaviour — most of what is left uncovered in LuaFile is IOException catch
blocks, which need fault injection, not more scripts.
os.time and os.date were blacklisted from the library-argument grid because
with no arguments they read the clock. With explicit arguments they are perfectly
deterministic, so an entire area went unwatched. This grid is 1152 lines:
seven os.time fields × 23 values, the missing-field defaults (hour defaults to
12, year/month/day have no default), setallfields writing the
normalised fields back into the caller's table, 60 os.date formats × 15
timestamps, os.difftime, os.getenv, os.setlocale.
It found one real divergence: os.time{year=2147483647} returned garbage.
The reference gives 67767976201968000; we gave -6019000092870206. The cause
is that GregorianCalendar holds milliseconds, so at that magnitude
getTimeInMillis() overflows and wraps to a negative number. It now goes through
OsLib.civilToEpoch (proleptic-Gregorian arithmetic in plain long), and the two
agree — and 67767976201968000 is exactly 86400 * days_from_civil minus the
zone offset, i.e. pure arithmetic, so both libcs produce it.
Two things are measured as out of scope rather than guessed at:
- Anything before 1900. Apple libc's
mktimerefuses it (measured:year = 1900works,1899does not); glibc does not. Neither implementation is "right", so the probe printsSKIPfor that band. os.time(nil)/os.time()—lua_isnoneornilmeans the clock.
Measured sensitivity: the conversion-specifier error text 54, civilToEpoch's
month base (tmMon is 0-based) 29, isdst always 0 / isdst = 1 not shifting
an hour 24, setAllFields's field names 22, falling back to Calendar (overflow)
3. And one honest zero: os.date's MAX_SEC guard is 0 — 2^60 falls outside
it either way, so this grid cannot see that line.
This is a breaking rewrite. The old fakescript syntax is gone entirely:
package, include, struct, const, ->-style for, array(), map(),
fake coroutines, :=, the Java binding annotations, and the profiler.
Existing fakescript scripts must be rewritten as Lua — which is the point: after this change, what runs is Lua 5.4, and scripts are portable to and from any other Lua 5.4 implementation. Users who still need the old language should pin to the last commit before the rewrite.
./mvnw clean test # requires JDK 21+With no lua5.4 binary on the machine, the differential tests skip themselves
(a single assumeTrue guard in Diff) and the coverage gate is skipped with
them — the differential suites are most of the line coverage, so a gate that
runs while they are all skipped would only ever be red for the wrong reason.
Set LUA54_BIN, or build the reference into target/lua54/src/lua, to get the
real gate:
LUA54_BIN=/path/to/lua5.4 ./mvnw clean testCI builds the reference from source and exports LUA54_BIN and FKDUMP_BIN, so
this only affects local builds without a reference binary.
MIT