Ethereum Simulations ==================== Ethereum is using the EVM to drive updates over the world state. Actual execution of the EVM is defined in [the EVM file](../evm). ```k requires "evm.k" requires "asm.k" requires "state-loader.k" module ETHEREUM-SIMULATION imports EVM imports EVM-ASSEMBLY imports STATE-LOADER ``` An Ethereum simulation is a list of Ethereum commands. Some Ethereum commands take an Ethereum specification (eg. for an account or transaction). ```k syntax EthereumSimulation ::= ".EthereumSimulation" | EthereumCommand EthereumSimulation // ---------------------------------------------------------------- rule .EthereumSimulation => . ... rule ETC ETS:EthereumSimulation => ETC ~> ETS ... rule ETC1:EthereumCommand ~> ETC2 ETS:EthereumSimulation => ETC1 ~> ETC2 ~> ETS ... rule #halt ~> ETC ETS:EthereumSimulation => #halt ~> ETC ~> ETS ... syntax EthereumSimulation ::= JSON // ---------------------------------- rule JSONINPUT:JSON => run JSONINPUT success .EthereumSimulation ``` For verification purposes, it's much easier to specify a program in terms of its op-codes and not the hex-encoding that the tests use. To do so, we'll extend sort `JSON` with some EVM specific syntax, and provide a "pretti-fication" to the nicer input form. ```k syntax JSON ::= ByteArray | OpCodes | Map | Call | SubstateLogEntry | Account // ----------------------------------------------------------------------------- syntax JSONs ::= #sortJSONs ( JSONs ) [function] | #sortJSONs ( JSONs , JSONs ) [function, klabel(#sortJSONsAux)] // ------------------------------------------------------------------------------- rule #sortJSONs(JS) => #sortJSONs(JS, .JSONs) rule #sortJSONs(.JSONs, LS) => LS rule #sortJSONs(((KEY : VAL) , REST), LS) => #insertJSONKey((KEY : VAL), #sortJSONs(REST, LS)) syntax JSONs ::= #insertJSONKey ( JSON , JSONs ) [function] // ----------------------------------------------------------- rule #insertJSONKey( JS , .JSONs ) => JS , .JSONs rule #insertJSONKey( (KEY : VAL) , ((KEY' : VAL') , REST) ) => (KEY : VAL) , (KEY' : VAL') , REST requires KEY (KEY' : VAL') , #insertJSONKey((KEY : VAL) , REST) requires KEY >=String KEY' syntax Bool ::= #isSorted ( JSONs ) [function] // ---------------------------------------------- rule #isSorted( .JSONs ) => true rule #isSorted( KEY : _ ) => true rule #isSorted( (KEY : _) , (KEY' : VAL) , REST ) => KEY <=String KEY' andThenBool #isSorted((KEY' : VAL) , REST) ``` ### Driving Execution - `start` places `#next` on the `` cell so that execution of the loaded state begin. - `flush` places `#finalize` on the `` cell. ```k syntax EthereumCommand ::= "start" // ---------------------------------- rule NORMAL start => #execute ... rule VMTESTS start => #execute ... syntax EthereumCommand ::= "flush" // ---------------------------------- rule EXECMODE EVMC_SUCCESS #halt ~> flush => #finalizeTx(EXECMODE ==K VMTESTS) ... rule EXECMODE _:ExceptionalStatusCode #halt ~> flush => #finalizeTx(EXECMODE ==K VMTESTS) ~> #halt ... ``` - `startTx` computes the sender of the transaction, and places loadTx on the `k` cell. - `loadTx(_)` loads the next transaction to be executed into the current state. - `finishTx` is a place-holder for performing necessary cleanup after a transaction. **TODO**: `loadTx(_) => loadTx_` ```k syntax EthereumCommand ::= "startTx" // ------------------------------------ rule startTx => #finalizeBlock ... .List rule startTx => loadTx(#sender(TN, TP, TG, TT, TV, #unparseByteStack(DATA), TW, TR, TS)) ... ListItem(TXID:Int) ... TXID TN TP TG TT TV TW TR TS DATA syntax EthereumCommand ::= loadTx ( Account ) // --------------------------------------------- rule loadTx(ACCTFROM) => #loadAccount #newAddr(ACCTFROM, NONCE) ~> #create ACCTFROM #newAddr(ACCTFROM, NONCE) VALUE CODE ~> #finishTx ~> #finalizeTx(false) ~> startTx ... SCHED _ => GPRICE _ => GLIMIT -Int G0(SCHED, CODE, true) _ => ACCTFROM _ => -1 ListItem(TXID:Int) ... MINER TXID GPRICE GLIMIT .Account VALUE CODE ... ACCTFROM BAL => BAL -Int (GLIMIT *Int GPRICE) NONCE ... _ => SetItem(MINER) rule loadTx(ACCTFROM) => #loadAccount ACCTTO ~> #lookupCode ACCTTO ~> #call ACCTFROM ACCTTO ACCTTO VALUE VALUE DATA false ~> #finishTx ~> #finalizeTx(false) ~> startTx ... SCHED _ => GPRICE _ => GLIMIT -Int G0(SCHED, DATA, false) _ => ACCTFROM _ => -1 ListItem(TXID:Int) ... MINER TXID GPRICE GLIMIT ACCTTO VALUE DATA ... ACCTFROM BAL => BAL -Int (GLIMIT *Int GPRICE) NONCE => NONCE +Int 1 ... _ => SetItem(MINER) requires ACCTTO =/=K .Account syntax EthereumCommand ::= "#finishTx" // -------------------------------------- rule _:ExceptionalStatusCode #halt ~> #finishTx => #popCallStack ~> #popWorldState ... rule EVMC_REVERT #halt ~> #finishTx => #popCallStack ~> #popWorldState ~> #refund GAVAIL ... GAVAIL rule EVMC_SUCCESS #halt ~> #finishTx => #mkCodeDeposit ACCT ... ACCT ListItem(TXID:Int) ... TXID .Account ... rule EVMC_SUCCESS #halt ~> #finishTx => #popCallStack ~> #dropWorldState ~> #refund GAVAIL ... ACCT GAVAIL ListItem(TXID:Int) ... TXID TT ... requires TT =/=K .Account ``` - `exception` only clears from the `` cell if there is an exception preceding it. - `failure_` holds the name of a test that failed if a test does fail. - `success` sets the `` to `0` and the `` to `SUCCESS`. ```k syntax Mode ::= "SUCCESS" // ------------------------- syntax EthereumCommand ::= "exception" | "status" StatusCode // ------------------------------------------------------------ rule _:ExceptionalStatusCode #halt ~> exception => . ... rule status SC => . ... SC syntax EthereumCommand ::= "failure" String | "success" // ------------------------------------------------------- rule success => . ... _ => 0 _ => SUCCESS rule failure _ => . ... rule #halt ~> failure _ => . ... ``` ### Running Tests - `run` runs a given set of Ethereum tests (from the test-set). Note that `TEST` is sorted here so that key `"network"` comes before key `"pre"`. ```k syntax EthereumCommand ::= "run" JSON // ------------------------------------- rule run { .JSONs } => . ... rule run { TESTID : { TEST:JSONs } , TESTS } => run ( TESTID : { #sortJSONs(TEST) } ) ~> #if #hasPost?( { TEST } ) #then .K #else exception #fi ~> clear ~> run { TESTS } ... syntax Bool ::= "#hasPost?" "(" JSON ")" [function] // --------------------------------------------------- rule #hasPost? ({ .JSONs }) => false rule #hasPost? ({ (KEY:String) : _ , REST }) => (KEY in #postKeys) orBool #hasPost? ({ REST }) ``` - `#loadKeys` are all the JSON nodes which should be considered as loads before execution. ```k syntax Set ::= "#loadKeys" [function] // ------------------------------------- rule #loadKeys => ( SetItem("env") SetItem("pre") SetItem("rlp") SetItem("network") SetItem("genesisRLP") ) rule run TESTID : { KEY : (VAL:JSON) , REST } => load KEY : VAL ~> run TESTID : { REST } ... requires KEY in #loadKeys rule run TESTID : { "blocks" : [ { KEY : VAL , REST1 => REST1 }, .JSONs ] , ( REST2 => KEY : VAL , REST2 ) } ... rule run TESTID : { "blocks" : [ { .JSONs }, .JSONs ] , REST } => run TESTID : { REST } ... ``` - `#execKeys` are all the JSON nodes which should be considered for execution (between loading and checking). ```k syntax Set ::= "#execKeys" [function] // ------------------------------------- rule #execKeys => ( SetItem("exec") SetItem("lastblockhash") ) rule run TESTID : { KEY : (VAL:JSON) , NEXT , REST } => run TESTID : { NEXT , KEY : VAL , REST } ... requires KEY in #execKeys rule run TESTID : { "exec" : (EXEC:JSON) } => loadCallState EXEC ~> start ~> flush ... rule run TESTID : { "lastblockhash" : (HASH:String) } => #startBlock ~> startTx ... rule load "exec" : J => loadCallState J ... rule loadCallState { "caller" : (ACCTFROM:Int), REST => REST } ... _ => ACCTFROM rule loadCallState { "origin" : (ORIG:Int), REST => REST } ... _ => ORIG rule loadCallState { "address" : (ACCTTO:Int), REST => REST } ... _ => ACCTTO rule loadCallState { "code" : (CODE:OpCodes), REST => REST} ... _ => #asmOpCodes(CODE) _ => #computeValidJumpDests(#asmOpCodes(CODE)) rule loadCallState { KEY : ((VAL:String) => #parseWord(VAL)), _ } ... requires KEY in (SetItem("gas") SetItem("gasPrice") SetItem("value")) rule loadCallState { KEY : ((VAL:String) => #parseHexWord(VAL)), _ } ... requires KEY in (SetItem("address") SetItem("caller") SetItem("origin")) rule loadCallState { "code" : ((CODE:String) => #parseByteStack(CODE)), _ } ... ``` - `#postKeys` are a subset of `#checkKeys` which correspond to post-state account checks. - `#checkKeys` are all the JSON nodes which should be considered as checks after execution. ```k syntax Set ::= "#postKeys" [function] | "#allPostKeys" [function] | "#checkKeys" [function] // ------------------------------------------------------------------------------------------- rule #postKeys => ( SetItem("post") SetItem("postState") ) rule #allPostKeys => ( #postKeys SetItem("expect") SetItem("export") SetItem("expet") ) rule #checkKeys => ( #allPostKeys SetItem("logs") SetItem("out") SetItem("gas") SetItem("blockHeader") SetItem("transactions") SetItem("uncleHeaders") SetItem("genesisBlockHeader") ) rule run TESTID : { KEY : (VAL:JSON) , REST } => run TESTID : { REST } ~> check TESTID : { "post" : VAL } ... requires KEY in #allPostKeys rule run TESTID : { KEY : (VAL:JSON) , REST } => run TESTID : { REST } ~> check TESTID : { KEY : VAL } ... requires KEY in #checkKeys andBool notBool KEY in #allPostKeys ``` - `#discardKeys` are all the JSON nodes in the tests which should just be ignored. ```k syntax Set ::= "#discardKeys" [function] // ---------------------------------------- rule #discardKeys => ( SetItem("//") SetItem("_info") SetItem("callcreates") SetItem("sealEngine") ) rule run TESTID : { KEY : _ , REST } => run TESTID : { REST } ... requires KEY in #discardKeys ``` - `driver.md` specific handling of state-loader commands ```{.k .standalone} rule load "account" : { ACCTID : ACCT } => loadAccount ACCTID ACCT ... rule loadAccount _ { "balance" : ((VAL:String) => #parseWord(VAL)), _ } ... rule loadAccount _ { "nonce" : ((VAL:String) => #parseWord(VAL)), _ } ... rule loadAccount _ { "code" : ((CODE:String) => #parseByteStack(CODE)), _ } ... rule loadAccount _ { "storage" : ({ STORAGE:JSONs } => #parseMap({ STORAGE })), _ } ... rule loadTransaction _ { "gasLimit" : (TG:String => #asWord(#parseByteStackRaw(TG))), _ } ... rule loadTransaction _ { "gasPrice" : (TP:String => #asWord(#parseByteStackRaw(TP))), _ } ... rule loadTransaction _ { "nonce" : (TN:String => #asWord(#parseByteStackRaw(TN))), _ } ... rule loadTransaction _ { "v" : (TW:String => #asWord(#parseByteStackRaw(TW))), _ } ... rule loadTransaction _ { "value" : (TV:String => #asWord(#parseByteStackRaw(TV))), _ } ... rule loadTransaction _ { "to" : (TT:String => #asAccount(#parseByteStackRaw(TT))), _ } ... rule loadTransaction _ { "data" : (TI:String => #parseByteStackRaw(TI)), _ } ... rule loadTransaction _ { "r" : (TR:String => #padToWidth(32, #parseByteStackRaw(TR))), _ } ... rule loadTransaction _ { "s" : (TS:String => #padToWidth(32, #parseByteStackRaw(TS))), _ } ... ``` ### Checking State - `check_` checks if an account/transaction appears in the world-state as stated. ```k syntax EthereumCommand ::= "check" JSON // --------------------------------------- rule #halt ~> check J:JSON => check J ~> #halt ... rule check DATA : { .JSONs } => . ... requires DATA =/=String "transactions" rule check DATA : [ .JSONs ] => . ... requires DATA =/=String "ommerHeaders" rule check DATA : { (KEY:String) : VALUE , REST } => check DATA : { KEY : VALUE } ~> check DATA : { REST } ... requires REST =/=K .JSONs andBool notBool DATA in (SetItem("callcreates") SetItem("transactions")) rule check DATA : [ { TEST } , REST ] => check DATA : { TEST } ~> check DATA : [ REST ] ... requires DATA =/=String "transactions" rule check (KEY:String) : { JS:JSONs => #sortJSONs(JS) } ... requires KEY in (SetItem("callcreates")) andBool notBool #isSorted(JS) rule check TESTID : { "post" : POST } => check "account" : POST ~> failure TESTID ... rule check "account" : { ACCTID:Int : { KEY : VALUE , REST } } => check "account" : { ACCTID : { KEY : VALUE } } ~> check "account" : { ACCTID : { REST } } ... requires REST =/=K .JSONs rule check "account" : { ((ACCTID:String) => #parseAddr(ACCTID)) : ACCT } ... rule check "account" : { (ACCT:Int) : { "balance" : ((VAL:String) => #parseWord(VAL)) } } ... rule check "account" : { (ACCT:Int) : { "nonce" : ((VAL:String) => #parseWord(VAL)) } } ... rule check "account" : { (ACCT:Int) : { "code" : ((CODE:String) => #parseByteStack(CODE)) } } ... rule check "account" : { (ACCT:Int) : { "storage" : ({ STORAGE:JSONs } => #parseMap({ STORAGE })) } } ... rule EXECMODE check "account" : { ACCT : { "balance" : (BAL:Int) } } => . ... ACCT BAL ... requires EXECMODE =/=K VMTESTS rule VMTESTS check "account" : { ACCT : { "balance" : (BAL:Int) } } => . ... rule check "account" : { ACCT : { "nonce" : (NONCE:Int) } } => . ... ACCT NONCE ... rule check "account" : { ACCT : { "storage" : (STORAGE:Map) } } => . ... ACCT ACCTSTORAGE ... requires #removeZeros(ACCTSTORAGE) ==K STORAGE rule check "account" : { ACCT : { "code" : (CODE:ByteArray) } } => . ... ACCT CODE ... ``` Here we check the other post-conditions associated with an EVM test. ```k rule check TESTID : { "out" : OUT } => check "out" : OUT ~> failure TESTID ... // --------------------------------------------------------------------------------------- rule check "out" : ((OUT:String) => #parseByteStack(OUT)) ... rule check "out" : OUT => . ... OUT rule check TESTID : { "logs" : LOGS } => check "logs" : LOGS ~> failure TESTID ... // ------------------------------------------------------------------------------------------- rule check "logs" : HASH:String => . ... SL requires #parseHexBytes(Keccak256(#rlpEncodeLogs(SL))) ==K #parseByteStack(HASH) syntax String ::= #rlpEncodeLogs(List) [function] | #rlpEncodeLogsAux(List) [function] | #rlpEncodeTopics(List) [function] // -------------------------------------------------------- rule #rlpEncodeLogs(SL) => #rlpEncodeLength(#rlpEncodeLogsAux(SL), 192) rule #rlpEncodeLogsAux(ListItem({ ACCT | TOPICS | DATA }) SL) => #rlpEncodeLength(#rlpEncodeBytes(ACCT, 20) +String #rlpEncodeLength(#rlpEncodeTopics(TOPICS), 192) +String #rlpEncodeString(#unparseByteStack(DATA)), 192) +String #rlpEncodeLogsAux(SL) rule #rlpEncodeLogsAux(.List) => "" rule #rlpEncodeTopics(ListItem(TOPIC) TOPICS) => #rlpEncodeBytes(TOPIC, 32) +String #rlpEncodeTopics(TOPICS) rule #rlpEncodeTopics(.List) => "" rule check TESTID : { "gas" : GLEFT } => check "gas" : GLEFT ~> failure TESTID ... // ------------------------------------------------------------------------------------------- rule check "gas" : ((GLEFT:String) => #parseWord(GLEFT)) ... rule check "gas" : GLEFT => . ... GLEFT rule check TESTID : { "blockHeader" : BLOCKHEADER } => check "blockHeader" : BLOCKHEADER ~> failure TESTID // ---------------------------------------------------------------------------------------------------------- rule check "blockHeader" : { KEY : VALUE , REST } => check "blockHeader" : { KEY : VALUE } ~> check "blockHeader" : { REST } ... requires REST =/=K .JSONs rule check "blockHeader" : { KEY : (VALUE:String => #parseByteStack(VALUE)) } ... rule check "blockHeader" : { KEY : (VALUE:ByteArray => #asWord(VALUE)) } ... requires KEY in ( SetItem("coinbase") SetItem("difficulty") SetItem("gasLimit") SetItem("gasUsed") SetItem("mixHash") SetItem("nonce") SetItem("number") SetItem("parentHash") SetItem("receiptTrie") SetItem("stateRoot") SetItem("timestamp") SetItem("transactionsTrie") SetItem("uncleHash") ) rule check "blockHeader" : { "bloom" : VALUE } => . ... VALUE rule check "blockHeader" : { "coinbase" : VALUE } => . ... VALUE rule check "blockHeader" : { "difficulty" : VALUE } => . ... VALUE rule check "blockHeader" : { "extraData" : VALUE } => . ... VALUE rule check "blockHeader" : { "gasLimit" : VALUE } => . ... VALUE rule check "blockHeader" : { "gasUsed" : VALUE } => . ... VALUE rule check "blockHeader" : { "mixHash" : VALUE } => . ... VALUE rule check "blockHeader" : { "nonce" : VALUE } => . ... VALUE rule check "blockHeader" : { "number" : VALUE } => . ... VALUE rule check "blockHeader" : { "parentHash" : VALUE } => . ... VALUE rule check "blockHeader" : { "receiptTrie" : VALUE } => . ... VALUE rule check "blockHeader" : { "stateRoot" : VALUE } => . ... VALUE rule check "blockHeader" : { "timestamp" : VALUE } => . ... VALUE rule check "blockHeader" : { "transactionsTrie" : VALUE } => . ... VALUE rule check "blockHeader" : { "uncleHash" : VALUE } => . ... VALUE rule check "blockHeader" : { "hash": HASH:ByteArray } => . ... HP HO HC HR HT HE HB HD HI HL HG HS HX HM HN requires #blockHeaderHash(HP, HO, HC, HR, HT, HE, HB, HD, HI, HL, HG, HS, HX, HM, HN) ==Int #asWord(HASH) rule check TESTID : { "genesisBlockHeader" : BLOCKHEADER } => check "genesisBlockHeader" : BLOCKHEADER ~> failure TESTID // ------------------------------------------------------------------------------------------------------------------------ rule check "genesisBlockHeader" : { KEY : VALUE , REST } => check "genesisBlockHeader" : { KEY : VALUE } ~> check "genesisBlockHeader" : { REST } ... requires REST =/=K .JSONs rule check "genesisBlockHeader" : { KEY : VALUE } => .K ... requires KEY =/=String "hash" rule check "genesisBlockHeader" : { "hash": (HASH:String => #asWord(#parseByteStack(HASH))) } ... rule check "genesisBlockHeader" : { "hash": HASH } => . ... ... ListItem(HASH) ListItem(_) rule check TESTID : { "transactions" : TRANSACTIONS } => check "transactions" : TRANSACTIONS ~> failure TESTID ... // --------------------------------------------------------------------------------------------------------------------------- rule check "transactions" : [ .JSONs ] => . ... .List rule check "transactions" : { .JSONs } => . ... ListItem(_) => .List ... rule check "transactions" : [ TRANSACTION , REST ] => check "transactions" : TRANSACTION ~> check "transactions" : [ REST ] ... rule check "transactions" : { KEY : VALUE , REST } => check "transactions" : (KEY : VALUE) ~> check "transactions" : { REST } ... rule check "transactions" : (KEY : (VALUE:String => #parseByteStack(VALUE))) ... rule check "transactions" : ("to" : (VALUE:ByteArray => #asAccount(VALUE))) ... rule check "transactions" : (KEY : (VALUE:ByteArray => #padToWidth(32, VALUE))) ... requires KEY in (SetItem("r") SetItem("s")) andBool #sizeByteArray(VALUE) check "transactions" : (KEY : (VALUE:ByteArray => #asWord(VALUE))) ... requires KEY in (SetItem("gasLimit") SetItem("gasPrice") SetItem("nonce") SetItem("v") SetItem("value")) rule check "transactions" : ("data" : VALUE) => . ... ListItem(TXID) ... TXID VALUE ... rule check "transactions" : ("gasLimit" : VALUE) => . ... ListItem(TXID) ... TXID VALUE ... rule check "transactions" : ("gasPrice" : VALUE) => . ... ListItem(TXID) ... TXID VALUE ... rule check "transactions" : ("nonce" : VALUE) => . ... ListItem(TXID) ... TXID VALUE ... rule check "transactions" : ("r" : VALUE) => . ... ListItem(TXID) ... TXID VALUE ... rule check "transactions" : ("s" : VALUE) => . ... ListItem(TXID) ... TXID VALUE ... rule check "transactions" : ("to" : VALUE) => . ... ListItem(TXID) ... TXID VALUE ... rule check "transactions" : ("v" : VALUE) => . ... ListItem(TXID) ... TXID VALUE ... rule check "transactions" : ("value" : VALUE) => . ... ListItem(TXID) ... TXID VALUE ... ``` TODO: case with nonzero ommers. ```k rule check TESTID : { "uncleHeaders" : OMMERS } => check "ommerHeaders" : OMMERS ~> failure TESTID ... // --------------------------------------------------------------------------------------------------------------- rule check "ommerHeaders" : [ .JSONs ] => . ... [ .JSONs ] ``` ```k endmodule ```