Published September 22, 2026
| Version v1.0.2
Software
Open
When a Judgment Layer's Self-Reported Fields Lie
Description
A measurement and characterisation study of three judgment layers -- a local non-autoregressive typed-decision model (Laya), a remote typed-decision service (TypeSafe Jev), and a frontier autoregressive model (DeepSeek-V4.1-Flash) -- on a common item set. Four claims are tested; three hold independently (cost is not the binding constraint; access-layer self-reported fields are not trustworthy; capability collapses on tasks requiring the detection of an absence) and the fourth -- that a heterogeneous judge provides incremental coverage over the generator -- is NOT supported in any of three task regimes.
Notes
Files
PerryLink/llm-jev-laya-bench-v1.0.2.zip
Additional details
Related works
- Is supplement to
- Software: https://github.com/PerryLink/llm-jev-laya-bench/tree/v1.0.2 (URL)
Software
- Repository URL
- https://github.com/PerryLink/llm-jev-laya-bench