Flashbots: Finding & Capturing MEV 101 | HackMoney 2021 Workshop

[00:00:00]flashbutts with us um with robert miller and steph who will be going over finding and capturing mev101 so without further ado i'll let them take it away if you have any other questions please post them in the chat and they will get back to you either during the talk or towards the end and let's get started thank you so much super excited to be here talking to you all and i hope to see you all on chain shortly after this presentation i'm robert miller i'm a product manager at flashbots and i'm an med searcher myself uh steph want to introduce yourself hey everyone i'm staff i'm stefan i am working on product stuff at flashbots and uh yeah super excited to be doing a bit of a follow-up actually robert and i were on another hackathon presentation panel was a month ago or so and the title of that one was called how to make a million dollars a month a facility developer so maybe this is the follow up with a bit more alpha leaking a bit more discussion highly recommend you guys go check out that video on youtube as well maybe there's a way to link it somewhere i don't know um cool so maybe i'll um i'll go through a bit of an agenda so let's start by covering the tldr on mev then talk a bit about flashbots from a very high level what are the core things that enables and then quickly talk about the the use cases that people have previously built on flashbots then i think robert wants to do a walkthrough of how to build a bot and then we can dive into some of the hackathon ideas for uh for hack money um actually is there a way i can share my screen is that a feature that we have the bottom green share screen um if you want to share your screen on my behalf yeah you can just go to explore flashbots.net on it there you go okay so this is the beautiful little number go up chart that we like to put when we talk about uh mev it's a topic that um has garnered a lot more attention over the last few months and and you can see the attention is probably correlated very highly with the amount of money that people are making uh by extracting muv so what does mev stand for minor extractable value or we're trying to rename it to maximal extractable value it represents all the value that the party who's in charge of putting transactions into a block is able to extract from the system by reordering inserting or censoring transactions so practically we can probably scroll a bit lower down what happens is a bunch of d5 protocols have some implementations that release more or less value by users sending transaction to their system that value can then be captured by miners or bot operators the majority of the value so far has been captured by bot operators but you know over time it's expected to move over to miners so here we can see a bit the type of activity and the uh the distribution between uh miners and operators the really interesting thing about this is uh if we scroll back up is that this ecosystem of extraction was already happening for quite a while so we only started tracking this in january 2020 and throughout 2020 it sort of slowly started picking up and then right about halfway through during d5 summer we really saw a lift-off in amount of extracted value this liftoff came with some negative externalities so what we also observed was uh gas fees going way higher as well as for some a lot of chain congestion so a lot of reverted transactions landing on chain so flash plots came together at that time and we've we asked the question is there a better way to do this um that doesn't impact the network as much um and uh and is incentive compatible so every single party in this ecosystem comes off comes better off so i'll jump into our documentation if you want to switch over to docs.flashbots.net so this is the flashbacks documentation i encourage you guys to go read through this it highlights the organization and the system how it works maybe you can click into flashbot's core and then um look at the overview so um this highlights a bit the way the system works i think we'll scroll down a bit to where we we have some pretty charts to show at the end of the day what flashbots does is it provides a way to communicate uh transaction directly to the miners by sending them an array of transactions that are included atomically those transactions include any arbitrary condition that say whether or not those transactions are valid and the the miners will include them so long as it's more profitable for them to do so than not what does this gain you well as a bot operator what you get is essentially guaranteed execution so your transactions only land on chain if they are profitable for you and you never pay for reverted transactions so that's a really really interesting property for anyone that's trying to compete for opportunities that are only there for maybe one block and only one person can grab so these ball operators go from sending these transactions in the regular transaction pool to now sending them on on flash plots and they save money the other thing that it does is that it's completely private so no other member in the network is able to snoop on the transactions as they go through only the miners can see them and include them in the uh in the blocks and then yeah maybe that's those are the main things to touch on there's three main use cases for who would be interested in using flashbots we actually list them yeah right here so the ball operators if they want to have risk-free execution the uh just regular ethereum users were looking for some level of front-running protection and keeping their transactions uh uh private so unislab traders or or liquidators that kind of stuff and then finally the apps that are looking for some more advanced user experiences so the ability to send multiple transactions at once and and have them be included atomically the ability to send transactions from accounts that have no ease in them so so gas abstraction and then finally the what's the last one yeah i think those are the main ones and and uh yeah so we've been seeing a sort of more and more application use these in novel ways so for example tornado cache um i think implemented a relayer that processes withdrawals and is able to essentially keep repeating until it it's able to get included at a price that's necessary so it provides better ux and then mistakes and and some other uh abstractions on top of dexes that uh allow for limit orders that you don't pay for reverts and are are front-running protected um so yeah there's a bunch of use cases uh and uh maybe robert wants to dive right into some of the interesting work that he's been doing building open source searchers sure happy to do so so i want to both show you an actual searcher that i wrote and was going to operate as well as talk about the process that i used uh for finding this opportunity developing it testing it and then eventually executing on it so first like to set up the foundation what i think you need in order to run a searcher is an understanding of ethereum so you're in the right place you need to understand transactions and smart contracts at a high level you need to understand defy at a high level um and then you need to know some sort of programming language to be able to get data off of the ethereum blockchain and then send transactions to it lastly i think you also should know about foughts and this is just the starting point from which i think you'll begin to be able to reason about and start executing on med opportunities but keep in mind that it will be a learning opportunity for you and a journey to capturing sorry about that got a call so a couple weeks ago on the discovery part of this mev capturing opportunity i was given a tip in a public discord the flashbacks discord by a person named caleb that synthetics had a a collateral program where you could use ether instead of the native snx asset to issue synthetic assets and this was always supposed to be a trial for them and eventually they would close the trial down and [Music] liquidate the loans that had been issued using ether instead and in fact caleb was letting us know that this trial was going to end and the governance of synthetics had voted for that and so there was about a month before this trial was going to end and they needed people to liquidate these loans provide the service to synthetics and in return for doing so you got a percentage of the loans and so immediately i knew there was med here it was like this member that is very close in the community telling us that there was an mev opportunity you provided some service and you could capture some med from there and so that's what the discovery process was like for me in this instance it was literally looking at the flashbacks discord watching carefully and a person that was very integrated into the platform knew that something was there and they pointed it out to me and more broadly i think that you should uh you should do similar things in the d5 communities that you're looking at stay up to date on the different updates that are happening the changes that are being made and try to think about where would i be able to capture some value provide some service and then make some money if i had very granular control of transaction ordering and if you think with that frame i think you'll find any of the opportunities so that's the first stage in writing a searcher is discovery and finding an opportunity and when you think you have it you need to move one step deeper into understanding that opportunity on like a deep level so here you are looking at the actual contract of synthetics where this opportunity was um you can go you can look at this on chain on your own time it's pretty public it has quite a bit of money in it um and i went to the synthetics documentation and i like had sort of had an idea about what it did but in order to capture any any enemy ned from the system you need to get pretty substantially deep into it and understand it on a deep level so i started by just looking at this i read the high level code i tried to understand what the features were you don't need to read line by line but i identified the things that i thought would be important to me so i was liquidating alone so that seems like closed loan would be important and i started there so i'm just like searching through in real time here um finding the close loan function this looks like an internal one so that is what we want so i'm looking through this i'm trying to understand it what's going on is the system active yes i know it is okay here i know that the system is pulling some data about a loan i'm like making a mental note to myself that there's some information uh that this contract needs to pull and i probably will have to get that information myself in order to understand what i can liquidate and how much mev can be made and what i need to provide so looking for places where data is collected within the contract and making a note of those either mentally or somewhere checking to see if the loan is open so that tells me some loans may not be open that's like another important piece of information [Music] now you can see that there is actually interest on these loans and it looks like it's dynamically created and so again this is another piece of data that like i'm mentally cataloging that i need to go back through later and find as well as the amount that is repaid that looks like it's just returned from this line up here and these things get added up to this variable total accrued interest i think that's the interest you have to pay on the loan and i'll have to do that the loan is then closed and the synthetics that you provide is burned and then this looks all like the collateral that i would get in return for doing so so i go through again this step is really about understanding the contract and how it works and then what kind of information i will need to interact with it and so just really like in that last minute all of a sudden i have all these different pieces of information um that i know i need to get from this so i probably need to look into whatever this get loan from storage function is i need to know what loans are open i need to know the interest amount because that gets looped up into this total accrued interest and that informs how much synthetic usd i need to provide to the system in order to get collateral back and once i provide that then it will give me i will be able to liquidate this position and i can get this total collateral liquidated position back okay so all that information that i'm going to need to get on my own and some monitoring capabilities that i need to set up and all of this stuff it is on chain you're looking at the smart contract right now you could you could do all this within a smart contract but the best searchers will do as much as possible off chain do all the calculations they can such that they are consuming less gas in their smart contract and their transactions are more competitive because they have higher gas prices due to the lower gas that they consume so that that like first i discovered the opportunity then i'm like trying to understand it and reading the code and then the next uh step in this process for me is building some monitoring tools so i will walk you through some of that i think you're now looking at my vs code if i'm not wrong this is a library that i intend to clean up and then i'll share with everybody so you can like look and see how i did this myself you just were pointed out a couple different pieces of information that you needed in order to liquidate loans and i know i need to collect so i started by [Music] making this very messy kind of function here that is an api key but that's okay this messy function here to get some addresses and loan ids to understand what loans were open and once i had the addresses and loan ids i could then pull some data out of them and then i could structure them in a way that was like easy to interact with so i went through several iterations of this of like creating scripts that would just only pull data out of the chain i wasn't sending transactions or anything like that i didn't even have a test environment on up it was just finding data and pulling it out and understanding what it looked like to get a feel for it once i had some basic monitoring capabilities open i went to what i like to think of is like execution or some form of of planning where um i'm not running anything in production yet but i am designing a smart contract and i have a test environment that's up and running so after reading the contract i knew that i needed to provide susd to the system in return i would get ether and there was several million dollars that was in this contract so i knew i would have to flashlight on this as well so i started to design a contract um which you can see here in in short this contract just took a flash loan from dydx because that's the most efficient thing to do and there's a lot of money on the line because you don't have any fees to it and i will pull you to the the place yeah so it takes a flashline from dydx and right about here i have a couple million dollars from dydx and then i'm taking in all of this data that i had monitored and pulled out of the scripts that you had just seen and decomposing it getting those variables so a bunch of stuff about loans and how much money i needed at different stages of this i swapped this for usdc since i start in ethereum but or ether rather and then i need to end up in susd so i'm starting here in uh and i swap for usdc this is like some console logging uh just won't happen in production but it's useful for testing so i have usdc i swapped that for susd and more console logging and then i can actually get to the part of this process where i'm liquidating loans so for all the different loans that i passed in in this like large looking structure here i can loop through them and it is actually on this line liquidate unclosed loan that i close positions and i get collateral back and then just for my purposes again i'm like doing all this logging to make sure that the numbers that i pulled out in my monitoring scripts actually match up to what happened on chain and i have my console output for this here this is actually what my contract ran whenever i whenever i run it uh it takes a couple minutes when you're in testing to do this so i won't run it live but you can see here here's like the different outputs of these different stages of the smart contract i'm liquidating alone here the first loan uh it has this id it's from this address and there's these amounts um that are coming to it and i only have three minutes so i'll try to try to speed this up but there's this loop for me where i am trying to understand the contract i'm trying to build monitoring capabilities to get the data that i need and then i'm trying to like plan my execution in a testing environment and by um by doing that i learn more at the very end of this process after i've written this contract about the information that i need that would make this easier and so i'll go back uh and i'll like build more monitoring capability or i will notice that my numbers in this console log are just like slightly off what i expect them to be and so i i need to go back to the contract and understand it a little bit better and so there's that loop between understanding monitoring building testing capability uh i also here have like um when i open source this repo you'll be able to see like how i actually would execute on these opportunities as well and how the transactions are end up getting sent in to the flashbots network and landed on chain but that just uses his flashbots and our bundle provider so those are my steps i start with some process to discovery finding things i go on to understanding the contract monitoring it i plan on executing and i do testing and i have this loop back to like understanding monitoring planning testing loop back several times until i feel good about it uh and then i do some dry runs of what it would look like actually passing something into right here what you're looking at which is the actual execution part of this without actually sending any bundles in so that's my high level process and like a walkthrough of how i'm thinking about contracts when i'm reading them uh and and like a little bit of annotating of my code i have two minutes left i think if there are any maybe robert do you wanna do you wanna share um if you weren't rug pulled by the synthetics team how much was the opportunity that you were uh trying to target here uh yeah you can actually see it in my console log so um let's see so the the i would liquidate these in a couple different bundles a couple different batches actually so this was the first one that would have been 26 eth profit and 2600 but i would have in in addition liquidated these next six loans for 13 more eth and 3 000 more dollars so 39 eth and 5 000 and then two more loans on top of that for one more ether and 3 000 more dollars so you can do the math it's like 60 something eth as well as 13 000 dollars was like my expected profit from this net of what i would pay to the miner uh and unfortunately this opportunity is no longer available so i can't take it and that's why you're seeing it now cool yeah so a couple hundred thousand dollars and you have the whole research process hopefully this helps people think about how to approach thank you thank you so much i think you muted yourself as your speaking step uh but um thank you for the presentation we're kind of running out of time here but it was a great presentation super interesting topic um you guys will be on discord for the event so if you have any questions for the flashback team join their discord channel i see that there are questions popping in but we're really out of time there's another workshop right after this one so please bring your question over to the actual sponsor channel thank you very much for your time thank you very much i appreciate it