Modular Rollup Architecture: OP Stack Deep Dive Theory

[00:00:00]foreign [Music] all right all right all right you just heard Carl talk um about a lot of high level things my talk is unfortunately that slides don't look like Carl's talk so it's a little bit different I want to I want to ground this what Carl just talked about in the actual Tech behind the op stack right I want to present to you what the op stack looks like under the hood I want to present to you the theory of how you build modular blockchains right not just oh you know let's make some charts and let's figure it out and and you know somebody will piece these things together I want to give concrete apis and functions that describe the different layers of a system like this so you can really start to wrap your head around how something like the op stack works I'm going to be continuously out of breath because we're like 8 000 feet up in the air it's really hard to talk so here you go modular roll-up Theory through the lens of the op stack AKA you just watched Carl's talk and now you want to understand how this whole modular roll up thing actually works okay all right let's go um we're gonna do drawings we always do silly drawings uh in these talks so here we go uh first I want to give some context so I'm going to be talking about the theory behind modular roll-up architecture stack is a specific software stack that turns this Theory into practice right this is about taking that theory and turning into specific software components that you can then use to build modular chains um and I just don't like talking about Theory alone I think Theory alone is not very useful unless you ground it and so I'm going to be using this stack to keep the talk grounded right I'm going to be using the in the abstractions that we came up with in the op stack to keep things grounded I'm also going to use typescript types to represent things typescript is god tier don't at me here is typescript ascending into heaven because truly greatest language of all time I hope you enjoy it let's go all right so modular rollups 101 which lost the money oh there we go we get it back um give some brief history back in 2020 everyone was building monolithic rollups this is what they looked like Beefcake Roll-Ups really absolutely ripped six-pack abs roll ups it was one giant code base and it was really hard to understand where the separations were and what what a data availability mean um basically all the everything was defined and limited by our proof systems right so we would we would we were like okay what is a roll-up well uh we an optimistic roll up we have to build this this this fault proof so we're going to build default proof um and then we're going to figure out how to make the system work like the evm inside of a functioning fall proof and and we did that because we had no clue what we were actually doing we had no clue what Roll-Ups really were this was us how do we roll up who knows let's just build something oh yeah that's what we did mental models are really really important if you don't know what you're building you're gonna build it wrong as an aside I think this is funny you know like I've been working on this stuff probably for like five years now and every six months we sit there and we're like ah it actually works completely differently from how we thought it works like the other day Carl and I are sitting there and we're like oh withdrawals are just totally not what we thought withdrawals are so it's crazy it's just interesting I think you know we really don't understand the problems that we're working on it takes a long time for us to build those mental models and so if you don't understand something that's okay because not everybody does nobody really understands what they're working on even when you're working on it and building a whole protocol out of it um then we finally got it so between 2021 2022 we started to really understand Roll-Ups in more detail the first key realization was that proofs should be execute or should be separated from execution we should not allow the structure of the proof to control the behavior of our execution layer and this was you know so here we go you know with the power of mental models I Vanquish the monolithic rollups oh oh Jesus oh they got ripped in half oh no um and so then we got stuff like optimisms evm equivalence upgrade arbitrims Nitro upgrade all the same idea let's separate proofs let's just run the evm as built with minimal modifications to make it work like a roll-up and have the proofs be so General that it doesn't matter what you can what you do on the execution layer you can do whatever you want all right then we also came to realize that we could break out the data availability layer um bra right so uh you know we had got a fork and it added a data availability committee arbitrim releases Nova with the data availability committee this is all a realization that we can also separate data availability right we can we can include a different thing that's not ethereum and these systems can still work with different security properties and so Roll-Ups were coming modular right we start to see this we generally start to see Roll-Ups beginning to break down into three primary layers if you've looked at modular Roll-Ups before because people have been screaming about this for the last year you kind of understand where it's going consensus execution settlement that's the high level idea but that's the high level right isn't that just modular blockchains yeah in a way but really it's about taking the theory of modular blockchains and putting it into practice it's about saying okay let's not just have charts in a blog post that says well if you separate it if you put this it could be this thing and if you put this it could be a validium and whatever but actually saying let's define a software stack makes it possible to put the things together for the average person to not have to build the whole thing from scratch and it was time to make things official so this was modular blockchain design and being put into practice but the beginning was very messy and very haphazard right people were kind of just ripping things out they were kind of just tweaking the execution layer and hoping it worked there wasn't a formality around it so you know what time it is it's formalization time uh you know like Loosely formalized because I never graduated college so I don't know how to do that but we're gonna it's good enough it's good enough all right so here section two now um the section numbers are completely messed up I just never even got around to fixing it but whatever beep boop bias warning I'm going to use the abstractions that we Define in the op stack I think these abstractions are good there are other abstractions out there but I think the OB Stacks abstractions are pretty good so deal with it I'm going to use those um like I said there are three primary layers we got consensus inside of consensus we got two things what we call data availability you've heard of that before derivation you may not have heard of that before then execution then settlement so let's look at consensus let's specifically look at these two sub-components we've got data the data availability layer and we've got the derivation layer right so what is the data availability layer even all right let's try to formalize this a little bit well you know it's like the place where you post data right like how does that actually work so let's form let's get slightly more formal I like to say and I think a good way to think of the date availability layer is it's an ordered list of blobs it's just an array of byte strings right that's the most abstract way that you can think of the data availability layer there are certain properties that you might want out of a data availability layer in practice right you really want this to be a mostly immutable append only list you want this the data to actually be available but that's a that's an implementation detail of the data availability layer itself at the highest level it really just looks like this it's an array of byte streams in the context of ethereum right if we look at this what is that array of byte strings it's pretty straightforward it's just every new block every block is our blob right and every it's this array of blocks that just grows and grows and grows and has pretty good data availability properties and has pretty good immutable prop mutability properties um so you can do that you know here's a whatever some code of we're pulling the data from from L1 blocks those are our blobs um same idea in 4844 except you're looking at the 4844 you know data sidecar instead of looking at the primary block got stuff like Celestia and you got like a stack of Post-its like you could do it with a stack of Post-its if it's like a sufficiently good stack of Post-its um all right so array of blobs pretty straightforward generally blobs blocks now let's get into derivation because derivation is what I think to be one of the more interesting layers in this whole system at a high level derivation is taking the data availability layer and it's understanding what that data means and it's parsing it and using that data plus the current state of your layer 2 system it's producing the inputs to your chain right it's producing for example I'll get to it but it can produce transactions you know that's sort of the classic example our case we use the engine API the op stack uses the engine API why the engine API yeah it's like one of these things we're opinionated on basically we think you know where as as optimism we're building this evm equivalent optimistic rollup the ethereum already standardized the separation between the consensus client and the ex execution client it didn't really make much sense to come up with a totally new way of driving execution you might as well just reuse the same thing because the apis are there um all right let's go so let's formalize this what does this actually mean I think this has a relatively simple function signature and it kind of looks like this you have this derivation function and it takes the previous state of your roll up or your system it takes the data availability layer and it's either going to produce a payload for you to execute on the on the execution layer or it's going to produce nothing and it might produce nothing if let's say you're a roll-up and there just haven't been any new transactions published to layer one it might just say well I looked at this new block that came in from the data availability layer and there's nothing new to put into the roll up so I'm just going to produce nothing or it could look at a block in the date availability layer or a blob whatever and it could say okay actually here there's a transaction that we need to execute on our layer 2 system so let's make this a little more concrete let's look at derivation in Bedrock we look at data from Three locations the number one piece like the most common place we look for data is sequencer data posted to a specific address on layer one this kind of makes sense we the sequencer collects a bunch of user transactions compresses it shoves it onto layer one and it posts it on to a specific address not a contract interestingly enough um you know and it's here's the many it's actually very complicated if you want to derive this and and make it extraordinarily efficient so it's in multiple files here but you could make this much simpler if you really wanted to at the expense of gas costs then we derive data from a second location which is deposits sent to the portal contract which is basically our deposit contract so when there's a deposit event emitted by the portal contract in a block we're going to derive a layer to deposit transaction here's how we do that right whatever then the layer 1 block data itself every time there's a new layer one block we're going to derive a special transaction on layer 2 that carries the information about that layer one block into Layer Two into the layer 2 State here's how we do that whatever the point is that we can transform all of these different sort of data sources into payloads that get executed on Layer Two um and this abstraction is actually really really powerful and I want to kind of hammer this in because if you want to build a roll-up you did what we do what we just talked about right you read sequence transactions read deposit data read block data whatever but that's not the only thing you can do with this you can derive data from almost any piece of information that lives on ethereum or lives on your layer one uh or your data availability layer so I want to just show you a toy example of something that I think is really cool that you can do with this let's say that you had a layer two without user transactions without anything right and the only way that you derive these engine payloads was every time there was a uniswap swap event on layer one you derive a transaction on Layer Two that includes the assets and the amounts right and you feed that transaction into a smart contract on Layer Two that keeps a running tally of the volumes that kind of looks like something like all of a sudden you have this thing this provable thing that is deriving data from layer one and is generating some sort of you know State on Layer Two and it kind of looks like an indexer you can essentially build an entire verifiable indexer as a roll-up are they just Roll-Ups who knows I think secret indexes are kind of just roll ups people just didn't realize it um there's a lot you can do with this you can derive Layer Two payloads from anything on layer one and you can do so much with that all right section 4C it skipped section three this is uh this is whatever section three four um what is the execution layer let's get on to it it is what you think it is the execution is what you think of it is it's the interesting part of the state transition function it's the fun stuff it's where you get to do your crazy crazy applications of course in this system it's also represented as a function in a very abstract way it looks like this you take the previous state of your system you take a payload that was derived by the derivation layer and you're going to transition your state into the next state this is very abstract this is not you know in in this statement it says nothing about the evm it says nothing about the actual State transition function it just says you need a state transition function derivation and execution in this system work together they work together to form something that I call the state transition function loop the loop is very simple when there's a new element that comes in from the date availability layer so let's say a new block was produced on ethereum we're going to pass that into the derivation function and the derivation function is going to do one of two things either the derivation function looks at that block and it says Ah there's nothing in here there's nothing to put into the layer 2 system there's no payloads to derive you know no transactions were published to the special sequencer address whatever and in that case it's just going to wait for the next element to come from the data availability layer to look for the next thing that it might derive then the other option it has is to return a payload if it returns a payload it's going to pass that payload into the execution function where the state is going to get updated then we're going to go back to the derivation function see if there's anything new to derive and this thing just repeats and repeats and repeats and this is your entire transition function if you want to see that as little drawn out image essentially we have the data availability layer a new blob comes in we derive something is it null then let's just wait for the next thing to come in if it's not null then we're going to derive a payload and then we have to execute that payload see if we derive anything new it goes in this you know beautiful figure eight for all of time right this is how derivation and execution work together to form the state transition function Loop all right execution in Bedrock let's make this a little more concrete how does this apply to what optimism is doing in optimism it's just the ebm we just changed the evm we you know mostly it's just dbm um we added a few things that you need to do to make a system roll up compatible the primary diff in our div is the addition of a new transaction type called a deposit transaction type which allows contracts on layer 1 to interact with Layer Two right it's what allows you to perform deposits it's essentially the message passing layer you need this in the evm right now I don't know that we know of a way to do it without adding this diff but it's a very very very small diff um and essentially we're now at less than a thousand lines of code in a segment uh here's the diff like Carl showed and actually 71 of those lines are this uh Circle CI config file so it's 750. uh this gives you a really important thing because it gives you support for multiple clients it means that we can very easily Port our diff to multiple clients and if you don't have multiple clients running a bug in a single client can be catastrophic for your roll up no no matter how good the proofs are because the proofs are just confirming the execution of your clients if the proofs are just confirming the execution of your clients and there's a bug in your client then the proof is going to say go ahead the bug executed correctly and nothing matters so you need client diversity and having this small diff is a critical part of that if you want I don't know if people can scan this ship scan this is the diff or you can go on our GitHub repo op Geth and you can find it's a single commit which makes it extremely easy to rebase onto the latest head state of ethereum of go ethereum makes it super easy to deal with so all right let's continue because I don't have a lot of time left just because we're doing the evm doesn't mean you have to do the evm this approach is really really flexible there's nothing in this that says you have to do the evm you can do whatever you want you can annoy the Maxis and do a Bitcoin you can put a Game Boy inside of a roll-up you can do a python interpreter inside of a roll up as long as you have a state transition function you can put it behind the engine API and the rest of the system will just work publishing transactions to ethereum will just work the pro the fault proof system will just work everything just works you just switch it out as long as it compiles to mips you're good you're fine so like Carl said the sky's the limit you have so much flexibility with this design and I highly recommend that you mess around with it okay on to the really final part of this talk settlement settlement is you know like kind of the fakest part of all this it's a little hard to understand is it even a real thing and I think yeah you know in a way um this is how I'll Define it you know for the sake of the old peace time settlement is a view that another chain has your chain and it's about making claims about the state of your chain to another chain and being able to back those claims up and so if you think about that for a second that means that you can settle to multiple chains at the same time there's nothing about this definition that says you can't sell to ethereum and Bitcoin at the same time there's nothing about this claim that says you can't have multiple settlement layers settlement mechanisms to the same chain right because you could be making claims you could be backing those claims up in different ways essentially it's just about making claims and backing them up um and actually let me get to that in a second well we'll see whatever fine all right so you can make all sorts of claims about a system but the one that you'll find most commonly is making a claim about what we call the state route of the system the state route in ethereum you kind of if you've played around with ethereum before you're probably familiar with it basically it's just a commitment to everything right the full state of ethereum so similarly generally the easiest thing to do is just to make a a claim about a commitment that commits to everything and then when you make that claim about the commitment that commits to everything else you can just prove against that commitment and now all of a sudden you can do you know pretty much whatever you want um but that doesn't mean that's the only way to do commitments you could do commitments about specific claims on Layer Two like this very specific thing happened on Layer Two um it's just that the most General and generally the easiest thing to do is to make a claim about the state group in order to make a claim about the state route we see we need a function that looks like this it's this validity function this looks very straightforward we have a previous state we have the next state that we're trying to figure out if that next state is valid given the previous state we have the date availability layer we have the derivation function we have the execution function all stuff that we've already talked about and we spit out a Boolean right we spit out something that says is this state transition valid and so the key question of how do you actually make this work in practice right this is this sounds great but how does this work in practice so look at this carefully let's try to figure out where each one of these inputs comes from on chain let's say we're settling to ethereum where does this come from so state is a given the previous state we agree on the previous state generally that's how we want to design these systems we start with the previous state that we agree on and then we have a next state and even if we don't agree on the validity of the next state we generally agree on the fact that this is the thing that we're putting into the function so the first two are Givens the derivation function in the execution function are literally functions right they are pieces of code that run and you could Implement those things on chain the old ovm optimism's old architecture Implement those implemented those things literally on chain so you execute the entire function on chain um generally speaking that's not feasible depending on how complex your system is so we tend to bypass this and instead of implementing the functions on chain we basically Implement something that acts like a proxy of the function and that's usually either fault proofs or validity proofs right these are our proxies for the execution because it's too expensive to actually carry out the execution on chain but you can think about how this fault proof of this validity proof is representing the derivation in their execution functions it is the code of those functions it's just that we execute it in a different way we execute it in a way that is actually feasible to do on chain but the interesting question is how do we access the data availability layer right how do we get access to that data availability layer on the chain that we're settling on of course it's another function if you remember our data availability layer takes the form of this array of blobs so we want a function to access this data availability layer and kind of an easy way to do this is you have this thing that says get Blob by Index right if it's an array and you have like get Blob by index you can access any element right but there's something really important being formalized in this idea and the important thing is that there's two properties that this function kind of encodes under the hood the first property is that the ability to resolve this function the ability to actually return a blob by a given index is fully dependent on actual availability of data on the data availability layer if the data is not available how are you going to return something out of this function you just can't the second thing is that you have to depend on a mechanism to prove that the blobs are correct because otherwise I can just give you anything right and I can just say oh the database is you know ABC but it was really something completely different so you need a proof mechanism let's see if we okay let me go to this really quickly go back for a second okay so in ethereum uh you know you know the proof mechanism we'll get to that in a second but for something like let's say Celestia this Celestia's data lives on a chain other than ethereum and so what we need if we want to prove that a given correct piece of data is correct we need to First carry over commitments to Celestia on ethereum and then we need to prove against those commitments so now we're essentially relying on the not only are we relying on the availability of the data availability layer we're also relying on the correctness of the function essentially the light client that is carrying commitments to Celestia onto ethereum there's two things that you need to think about when you want to use something like Celestia as your data availability layer all right so let's just look quickly about how Bedrock sort of does this validation in Bedrock we start with the latest block this is the magic right ethereum has this beautiful thing that ethereum from the latest block you can access everything it's beautiful the block hash hidden inside of the latest block hash is all of the information about ethereum for all of time because I can start with the latest block hash and then I can reveal the block and I can reveal the previous block hash and I can reveal the previous block hash for that and I can just go back in time to infinity and I can access every single block in the entire history of ethereum and not just every single block I can pull out every single piece of data in every single piece every single block in ethereum and so what I'm going to do in my validation function is I'm going to start with the latest block and in my is all of this is kind of hidden inside of the the fault proof but my program is going to walk backwards and walk backwards and walk backwards and figure out which blob in the data availability layer in this case ethereum blocks it's going to start the state transition function loop from that's the loop that we just saw and it's going to apply that Loop over and over and over again in the program until you get a final State and you compare that final state to the final state that you pre you know you proposed and all of this happens inside of the fault proof so none of this computation really happens on chain you just sort of have to you know if you want to challenge it you just have to do a single step but this is the magic of ethereum which is that you can the data is guaranteed to be available and the data is guaranteed to be correct because you're settling and putting data onto the same place all right so bringing a bell back together really quickly um with the multi-headed angel thing going on here there's a lot of content but there's not that many components to something like this remembering the components we got the data availability layer which this is array of bytes right array of blobs we've got the derivation function which takes the previous state and the date availability layer and it's either going to spit out a payload or it's going to spit out nothing you've got the execution function which takes the previous state and it's going to take in a payload generated by the derivation function it's going to produce a new state and you've got this validity function and then inside of this validity function you have this get Blob by Index right so this is all of this composes each one of these different layers is an API that you can plug into so you can take this or you can build your your dream chain right this isn't just Theory this is in optimism's code base in the Bedrock code base you can build whatever you want do you want to build a Bitcoin plasma yeah you can build a Bitcoin plasma you can do it really easily go build a Bitcoin plasma you want to build a bridge that's actually a roll-up and it has multiple data availability layers and multiple settlement layers why not um you want to build another paralyzed VM and raise like 200 million dollars from VCS why not you can do that you can build whatever you want you just have to fit the apis right that's what we've we've boiled this all down to a set of apis if you fill the apis you can build it so that's the whole talk uh thank you try to remember life my name is Kelvin I'm building the optimism Collective uh Bedrock specs get in touch thank you build something cool