Skip to content

Commit 67ae582

Browse files
ryzhykryzhyk
committed
[adapters] Journal replay respects transaction boundaries.
With the introduction of transactions, the model of computation changed so that the circuit now produces one output per transaction rather than per step. As a result the existing journal replay mechanism that runs each input step in a separate transaction is only correct assuming that the original inputs were also ingested with a step per transaction. But even that is no longer true after the introduction of streaming exchange, which can cause a transaction processing the exact same input take more or fewer steps than the original run of the program. This broke the validation mechanism that assumed that step numbers during recording and replay should match precisely. The solution that this commit implements has repa - The journal records transaction boundaries in addition to individual steps. - The replay mechanisms breaks up the sequence of replayed input steps into transactions and commits each transaction at exactly the recorded boundary. This restores the invariant the the outputs of the circuit during replay are identical to the outputs produced while processing the same inputs before the failure. As a side effect of this fix, we also save the last transaction number as part of the checkpoint, so after restart the circuit will resume contigous transaction count. Signed-off-by: Leonid Ryzhyk <ryzhyk@gmail.com>
1 parent 4c722f3 commit 67ae582

7 files changed

Lines changed: 557 additions & 120 deletions

File tree

0 commit comments

Comments
 (0)