Ethereum Virtual Machine: Architecture & State Transitions

[00:00:00]i'm going to explain how the ethereum virtual machine works what it does where it runs and how it's implemented the evm is sort of the brain of the ethereum system so understanding the evm is crucial to understanding how ethereum itself works so ethereum has this world state and the world state is sort of the state of all of the important things in the system all the nfts and tokens and d5 all the account balances that's all captured in the world state at any given time ethereum only has one world state it's just the current world state there is a history of world states these are the historical states but generally you kind of only care about the current up-to-date world state and this world state changes whenever a new block is mined a block has a bunch of bunch of transactions in it each transaction represents a change to the world state so when a block is applied all of those state changes in each transaction are applied all at once and you get a shift from a transition from one world state to the new world state and this ability for the system to have this state of all these accounts and all that stuff and transition to a new state as a result of transactions that is actually the purpose of the entire system you know the blockchain proof of stake proof of work all of these things are in a sense just there to support this ability for the system to be this state machine where you can have a state and transition to a new state and so on and the way these state transitions are computed is largely with the evm the evm will take these transactions the starting state and compute what the new state looks like which i'll explain more in a minute so that world state that i've described consists of you know all of the stuff in ethereum so it's like all the d5 all the dows all that stuff right but in a sense it's simple because the world state just consists of accounts all of that stuff the d5 etc etc is just a whole bunch of accounts and their state and etc etc so there are two types of accounts there's the simple externally owned accounts these are like metamask ledger etc they're externally owned and controlled by usually a person or maybe some code they have a private key and you know they have an address the address points to some some state some data for that account but in the case of these externally owned accounts it's very simple it's just a nonce and a balance they're just two numbers right there's no extra storage there's no tokens or anything like that in that account where all that stuff is implemented is in contracts so these contract accounts they have an address that points to some storage as well they have a nonsense as well and then they have the storage and code so these accounts actually have storage their own storage and they have some code and the code is immutable the storage can change and so the way this looks right the entire state consists of these two kinds of objects right now if you actually look at the way the ethereum clients implement this and keep track of all this data and sort it and you know allow for relatively quick accessing of this data is they keep it in this merkle patricia tree and this is a data structure it's this huge tree with all of the accounts in it so it has all of the regular externally owned accounts and all the contract accounts and the regular externally owned accounts they like these accounts are you can imagine they're the leaves here they're the bottom set of those little triangles if you can see that and the externally owned accounts they just have a little bit of data they just have a nonsenate balance there's not a whole lot to keep track of but the contract accounts they have non-balance code hash and storage root and their storage is implemented using the same data structure another merkel patricia tree so if you could imagine the entire ethereum state what it would look like is a huge tree thing with like a million accounts at the as the leaves at the bottom and then maybe like half of these accounts however many contracts there are versus regular ethereum account maybe let's say it's half half of those accounts are going to have their own little trees right keeping track of their storage that contract storage so what you would have if you could visualize the entire ethereum state would be this huge tree with all these leaves and then like 500 000 more little trees underneath it and that's the ethereum state and what the evm again what the evm does is it's as a result of processing transactions it's making little changes to this this giant data structure and the the state root so this entire data structure has this root and the root if you change any single thing in the entire in any of the accounts in that entire data structure then that change will result in the state root being different right because all of the storage and all the accounts all that stuff gets hashed together and it trickles up to the root and so which is why anytime a new block is mined this state route changes because even if only one account changes or if anything at all changes in in the entire ethereum state it'll change the state route okay so that's what the evm does and so this diagram here you have you have the starting world state and in this little situation we're at where the evm is acting on just one account you have address n account state n so this is a contract account because it has code and storage some transaction or message call came in with some input data so this could be someone kicking off a transaction to like uni swap to interact with a contract so that transaction will have some data some input data and it'll be calling a function right on like the say uni swap contract you know maybe like swap right they'll be calling that function they'll have their input data saying what they want to swap and all that the evm will take this input data and this world state including the state of the uni swap contract and it will take that information and compute the new state which i'll talk more about in a minute so the evm does this like i said it it computes these state changes especially for the more complex things that happen in ethereum the smart contract interactions for regular ethereum heath transfers the evm technically the evm doesn't do that it it doesn't actually process those transactions so to speak but i'll show i'll explain that in a second looking at the go ethereum code but the evm computes these state changes okay fine um so how does how does the evm work the evm is a virtual machine and so every node in the ethereum system like every node has to get the exact same result when it processes a transaction giving some given some starting state and some transaction everyone has to be able to take that starting state transaction and get the exact same result the exact same so virtual machines are good for this kind of application where you you want to have a wide range of different computing platforms different hardware different operating systems all that right but you want the operations happening on that on your your your code your your thing you want it to be the same regardless to um to turn out the same to operate the same regardless of what the underlying computing platform is so virtual machines are really good for this because they abstract away the underlying hardware and virtual machines have been used for a long time this is not specific at all to ethereum they've been around for decades and what they do is like you have a regular computer you have your physical hardware you have your cpu and memory and blah blah you have an operating system on top of that right you have then a bunch of your regular applications that run on that operating system your regular programs then alongside that you can have this hypervisor which is just a some software some code that manages your virtual machines and you can have your virtual machines and your virtual machines are like little computers running on your computer so you can have you know you can have a windows computer windows operating system and then you can have this uh hypervisor and you can have linux you can have like you could have three instances of linux running on your machine and so your the hypervisor will be managing your your resources so it'll allocate some some of the ram it'll allocate a bit of ram to each machine it'll allocate maybe a bit of storage um and the machine will run the little virtual machine will run just as if it was running on a full computer you know so in the case of ethereum it's similar to that you have you know on an ethereum node you have your physical computer and it can be running whatever linux windows mac you know you have your physical computer and whatever um platform you want then you have your ethereum node like geth or a parity and then within the ethereum node software you have the evm that's where the evm is instantiated that's where it runs it's not like a full uh like with other virtual machines it's not like you have a full operating system with like the linux and you have to allocate a bunch of ram the evm is much simpler but it still is a virtual machine and so the evm i'll talk more about solidity and where solidity fits in and how that solidity code compiles down to something the evm can understand but i'll talk now about a little bit about the architecture so you know at kind of a high level so we have this virtual machine which is it's a virtual machine so that you can get the same result every time you compute um every time you process a transaction given the same starting state and the purpose of the evm is to compute those those state transitions and this is the architecture of the evm so this is not the exact implementation of the evm isn't too too important i'll talk briefly about it but the fact that it uses a has a stack here and the memory here and and whatever it's kind of just an implementation detail you know it could have been designed to be different perhaps it doesn't matter too much what matters though is that all of the implementations of the evm on whatever client it is whether it's geth or parity or whatever they have to implement this virtual machine the same or kind of the core architecture of the virtual machine has to has to match a certain spec such that you get the same again you get the same result when you process a transaction you know some lower level behind the scenes aspects of the implementation can vary but much of it has to be matching a spec so you get that same result so the way the evm is architected is you have you have this virtual rom here you know it's like a read-only memory and this is just when the evm is working on some transaction processing some transaction it will pull the code for the contract that's being interacted with it pulls that into this virtual rom it just grabs that code and just pulls it into memory right but it doesn't change it it's immutable then in the actual the machine state like this is the volatile part of the evm this is the stuff that the will change as the evm runs uh and will also once the evm is done processing this transaction a transaction it'll wipe all this machine state so that's purely ephemeral it lasts for a bit and then it's wiped every time and it has a program counter i'll explain more of that in a minute it keeps track of the gas how much gas has been used so you can if the caller didn't provide enough high enough gas limit it will execute execute and then stop it has this stack and memory these just help it to compute those state changes you don't need some memory to work with and it has this account this uh access to account storage so if you make a transaction to some contract it's going to pull up that contract code and it's going to pull up that the storage for that contract because if you're interacting with the contract you may be making changes to the storage for that contract right so it pulls that up as well and uh that contract storage is part of the world state right so that is persistent that will if you make a change there as a result of your transaction then that change will persist it will become assuming the transaction completes it will become part of the canonical world state so that is not ephemeral and so you know the stack has this certain width 256 bits wide 1024 you know deep the you have the memory it's byte addressable linear memory uh you have account storage this is the storage in the actual contracts and this is key value mapping 256 to bit to 256 bit okay there's no registers so if you're familiar with with cpus and like assembly and all that like in a regular computer the cpu will have rather than being a stack machine you know the evm is kind of a cpu it's kind of acting like a cpu of sorts actually quite a bit like a cpu and but it has this architecture of having a stack so it's a stack machine as opposed to a register based cpu architecture so most as far as i know most cpus like intel cpus they have registers so rather than having this stack where you just you you know you put an item there you put another another item you pull it off you pull it off they'll have registers so they can put you know temporarily store data in register r0 r1 r2 and the evm is a stack machine so it's not like that so when the evm is processing a transaction so here this diagram is showing okay here's a transaction that came in it's two the transaction is to a certain address which happens to be the address for this contract here it has some data included it has a gas price gas limit so when this transaction comes in the evm will pull up the contract that it's calling right the contract has its address there it has some code i'll talk more about that in a second but it has this code associated with it and its storage et cetera the evm will you know use its its program counter to to step through this code so it can interpret and understand what the code you know what this contract code does um and it'll take this data that came in with a transaction and you know use that to know how to run this code so if you've called a specific function on some contract with some inputs that'll all be in your data the evm will read this you know grab the contract like i said start stepping through the code and it will do whatever you've told it to do it will it'll take that function that you're calling with the inputs execute that use the its stack and its memory to keep track of things and do math and blah blah blah and at the end of that it may have changed the storage here or it may call another contract which i'll talk about in a minute but that's what it does so this is a sort of interesting point you know the evm is in a sense single threaded or you could think of the world computer the ethereum world computer has it is single threaded if you if you think of it kind of analogously to a to a cpu within a regular computer you know all of the transactions that happen in the entire system are processed in serial like in order you know and the entire ethereum system that entire set of all those nodes right say there's whatever there's like a hundred thousand nodes you know one of those miners wins the block and so first they compute the state changes for that block right and then they they win the block they uh you know the nonce is sufficient that the hash is low enough that it beats the difficulty and they broadcast it out and now all of the other nodes in the network are execute those transactions compute the state the new state they compute that transition and then they all are synced up to the latest block and if you imagine this entire like huge set of all these nodes they really are executing like almost in lock step these all these transactions just in order it's like the whole set of them is like they go like this transaction that transaction that transaction and all of them are like right and they they're computed in blocks so it's more like every like 14 seconds it's that they kind of like jump forward to a new state but it's single threaded in the in that there's no you don't have like part of the network computing some state changes and then another half not computing those right if you had a sharded uh sharded execution then you could have that but we don't have that yet anyway so yeah like i said when each transaction happens when each transaction is being processed by the evm the right the contract code is pulled in it's loaded up the program counter is set to zero so it can start reading the code for the contract the contract storage is loaded up the memory is all set to zero and it loads up some block and environment variables which i'll mention in a minute i think and also like i mentioned if the gas limit is hit during execution right like a transaction was initiated but the gas limit wasn't set high enough or the eth runs out then all of the state changes that have that the evm has calculated up to that point for that transaction it just throws them all away and it still charges the sender uh it still takes their eth for executing that stuff but no changes actually happen except the sender loses some eath and uh they're nonce increments so like i said the evm is similar to a cpu in a sense and cpus have these very basic operations they do so if you write code in any regular language you know like uh like c or c plus plus you compile it down you eventually compile it down to assembly code in a regular cpu and the assembly will consist of a bunch of these basic operations like you know add and multiply and like store this value in a register if it's a register based cpu and so everything that happens on computers is compiled down to this low level you know these basic operations basic math operations basic storing of things in registers and moving them around and blah blah blah you know so the evm is the same in that way the solidity or viper or whatever it is you write when you write it you know on your computer it's written you know it looks like this or whatever and you'll compile it down to uh evm bytecode so the what the compiler does is it takes your code that's written in a way you can understand it and it compiles it down to something like this this is actually the the evm byte code for this tiny contract here and so what this is is this huge uh chunk of you know these hexadecimal numbers this is the evm byte code for this contract and this maps to these op codes here so these are the ethereum op codes similar again to the cpu architecture you have these low level operations and all contracts are compiled down to opcodes and so like this one starts with six zero that should be one of these op codes it's this one so six zero is push so this contract is starting by pushing some data onto the stack right and that's how it works you know that's how the contracts actually work they're a bunch of op codes and uh yeah so the op codes they have a gas cost associated with them and the way these are determined like gas is a unitless number so when you have for example the add operation it costs three gas right that's not three gray or anything you probably know that it's three gas it's just three gas and the way these are determined is these gas costs are essentially a reflection of the actual real world cost of running these on a regular computer running these on the evm which is running on a regular computer right so multiply is slightly more gas than add because multiply is more intense of operation i guess and you know the operations like writing to the world world state memory that costs a lot more uh gas so s store that writes to the uh to the um the actual contract storage and this costs a hundred gas right compared to like two or like these math operations are there computers can run these math operations very quickly so the gas is low on them but when you're storing something to the actual world state now all of the computers in the ethereum network have to store that as well on their hard drives this is just for one key value pair so that's 512 bits of total storage which is super small but still it's 100 gas you know because it's been stored forever and uh create a new account that's 32 000 gas because you're you're creating a bunch of new data that everyone has to keep track of and so like i mentioned the the evm runs this code you know op codes and that gets compiled down from from solidity right like i showed here this gets compiled down to a bunch of op codes these op codes are like this chunk of uh of data here this is actually what gets put onto the chain when you deploy a contract when you deploy contract it does not store solidity it does not store viper on chain because the evm can't interpret it anyway so what it does is you'll compile it down you'll get your byte code and that will be deployed on chain the evm can understand that if you go to ether scan you'll see a bunch of solidity here but this solidity is not uploaded to chain all they've done here is they you know they they send somehow they send their this the source code like they'll deploy their contract and it's deployed like this a whole bunch of bytecode and then they'll upload the source solidity to etherscan the service not to the blockchain and etherscan the service will check on their end that what they've been given does in fact compile down exactly to this right so then they can say yep that is uh that's clearly the right code because it compiles to the bytecode that's on chain right so yeah this solidity combine compiles down to this bytecode like i said and that's how it works so yeah like i said the got the gas cost per op code is a reflection of the actual cost of running those operations uh the gas price is different right like the add op code costs three gas and that doesn't change but when you're doing a transact so you create a transaction and it's going to do a bunch of stuff like a uni swap trade and let's say it's going to cost you a hundred thousand gas so it's essentially like there's a hundred thousand it's kind of like a hundred thousand cpu steps are required for your transaction to be processed but they're not exactly cpu steps because different operations are priced quite differently like the s-store opcode is 100 gas it's not really that it takes an ethereum node a lot of time to execute that operation uh it's partly that but it's also that it's been going to be stored for a long time maybe forever so the gas cost is trying to capture the entirety of the resource cost needed to execute that operation but the price you know the those gas costs are like they're just they're not in units of eth right so when you're actually doing a transaction if the network is under heavy demand at the moment or at that time then you'll need to set you'll need to pay you know even if your transaction is just the same 100 000 gas it's a uni swap trade if the network is under heavy load you're gonna have to pay more to get that transaction executed so no matter what time you do that transaction it's always a hundred thousand gas right let's say but at different times you'll have to pay a different amount of eth per gas right so you might pay 20 gray per gas and then your entire transaction will be you know like 2 million gway total or if the network is in demand and the you know you'd have to pay a hundred gwai and then you're paying you know 10 million gas or 10 10 million uh gway for your transaction i don't know if that's clear but so the it's another fact is because ethereum has a maximum gas per block that sets the maximum gas per transaction right so that kind of makes the evm like sort of turing complete uh to be truly turing complete a thing has to have you know like infinite tape infinite memory which nothing does but ethereum has this kind of you know regular computers they have tons of memory tons of storage and they can run you know you can run a computer kind of forever like for a long time whereas with the evm there's this hard limit uh no transaction gas use can exceed the maximum gas for a block right it's just a rule of the system so it sets just kind of a tighter limit on what you can compute on ethereum which is fine because it's not a general computer you're not going to run you know quake on ethereum and so this is one of the most interesting things i think about how learning about the evm and all this is that the evm is just code running on ethereum nodes the entire ethereum system is just a bunch of computers running you know software and the evm is just running within that within that software within that client software so we can actually look at the some functions in the evm now like i mentioned earlier the evm does not actually compute the transfer of eth from one account to another like all of this stuff is handled in the client software this is a go ethereum by the way geth this is the source code and this is the transfer function for transferring eth so i have another thing here with a bunch of another file with a bunch of evm opcodes but this is separate from those so when you do a simple transfer geth nodes process your your transfer with this code and they do some checks before this function's called but then it takes in the it takes in the world state it takes in the sender information the recipient and the amount and it you know subtract balance from sender amount add balance to recipient amount and that's how it works but so for simple transfers the evm isn't technically executing that operation although the client is executing it so it's kind of like it's happening in the same code base but if you're doing a more complex operation like calling a contract you know like i like we looked at the contract will have this big string of byte code it'll have these a huge series of op codes that it that it executes and here they are i found this really really kind of mind-blowing that this go code is going to run on an ethereum node uh could be any you know linux whatever and when it's processing a transaction that interacts with some contract it'll invoke the evm but the evm is just it's implemented here in go in a few different files i think but this go this geth client this go code will spin up an evm it'll load up a stack it'll load up all this stuff and that code that's implementing the stack and implementing all these things is just go it's just regular code and those op codes that the evm executes they're just regular functions like this is the this is the add op code that costs three gas or two three i think it's three you know it takes um it takes stuff off the stack because that's what the evm uses to keep track of its values and it adds them together which i think is pretty interesting so another little point here the ethereum you know just like the evm it runs on regular computers it runs in the go in the case of go ethereum it's just go code they have functions for the op codes and you know in the same vein of that the ethereum world state on a on a geth note or on any node is just it lives on the ssd of that machine so you know it's like where is ethereum well it's kind of everywhere and nowhere really because the actual state of ethereum it lives on the hard drives the ssds of all of the ethereum nodes and so it's kind of everywhere but it's also it's nowhere in the sense that there's no single node where it's like that's the state no it's you could lose any single node and the system would be fine so it's it's spread out over all the nodes so there's a few other op codes that are interesting so you know most of the or a lot of the op codes we looked at they're like they're doing basic math operations so you know a lot of them are like add multiply subtract blah blah blah blah but then you have some that are a little more interesting uh you know you can get like the gas price uh for that transaction so you can get some stuff that's more this stuff is more specific to ethereum right like add and subtract any low level assembly are going to have add and subtract but this stuff is kind of more specific to ethereum and these you know these ones you're getting you can see here that the evm has access to not actually that much right the evm has access to the 256 most recent hashes block hashes which i didn't know actually um that's interesting but it doesn't have access to like the transactions in those blocks the evm is that actually has a very limited view of ethereum it can see the entire world state but it can't see you know old block transactions it can't see like how many nodes are in the network it can't see i don't know you can't see the eth price right it can see it just a few things like gas limit block number so ethereum is right the evm has access to the world state like i said but very little else it can't access really anything outside of that it cannot call external apis and websites so you can't check like what is the eth price on coinbase from the evm the evm is sandboxed so it only has access to a super limited set of things and that's important because you want the evm to to perform exactly the same every single time it computes a given transaction with a certain starting state right so if you allow the evm to call an external api you know like check the price of eth then your contract could say oh if eath price is less than ten thousand dollars do this if it's above do that and then if two different nodes are executing that transaction at slightly different times right when the eth price crosses 10k they're gonna get different answers and then they'll have a different world state as a result of that transaction and then you have a fork you don't have consensus anymore so okay last thing that's that's good to know is smart contracts can call other smart contracts so if you do a transaction to some contract you call some function that contract may call another contract so the contract could actually call like a chain link oracle contract so chain link oracle contracts they they will have they can have an actual price feed they can have the eth price in a contract and the way that works bit of a tangent but the way that works is the these chain link nodes that are completely outside of ethereum will do transactions to a certain contract which is like the eth price feed contract they'll send in their update for the price youth price is now you know 2500 dollars each price is now 25.01 and so that contract will have in its memory in its storage the eth price for any given the current eth price maybe a history of eath prices then other contracts while they can't call to the outside world and check on coinbase.com what the east price is they can check this contract because at a given state snapshot whatever that price feed contract has for its eth price they're all going to see that same price right so there's no ambiguity so when you've called this initial contract it can call down to this eth price oracle contract so when the when that happens the the initial contract call will create an evm instance with the memory in the stack and all that it'll run through start executing and then come to the point where you call this other contract right then what it'll do is it will create another evm instance on the machine on the computer and new memory new stack but it will pass the the world state from the from the initial evm because the initial evm might have might have altered something in the world state right so it'll pass that potentially altered world state down but it fresh stack fresh memory and it will also pass down whatever input like if it's calling some function on this child evm child contract it'll pass that down and this this like sub evm can i mean it can call another contract and that can keep going but but that will only continue as long as there's enough gas from the original caller to pay for all of that computation and at some point if the transaction finishes properly then that world state that's been potentially mutated by like five different evms if that transaction completes that world state becomes the new world state and if it fails at any point in that stack the world state is just discarded the the initial transaction initiator loses their the eat they spent and all of the evms evaporate so all right so that's how it works uh i hope that's helpful main thing to know is that the evm is computing these state transitions that's really all there is to it and the op codes as well the contracts are compiled down to a bunch of op opcodes so that the evm can understand them and that's how it works okay so thanks for watching next video will be either on layer 2 things or eth2 let me know if you have a preference and thanks for watching see ya you