WebAssembly in Production: A Compiler in a Web Page

[00:00:12]so welcome to web assembly in production compiler and a web page I'm Brian Carol okay so let's talk about web assembly and first of all I want to get just a sense of who's here so how many people have worked in web development in some way shape or form okay pretty much every everybody and how many people have heard of web assembly before getting close to everybody as well how many people have web assembly live in production with a real app that's about four people okay so that's kind of what I expected so the reason uh so what I were're uh the for anyone who hasn't heard very much about it the basics are it's a binary code format runs in the browser and so JavaScript isn't the only option anymore for running code in a web page um but uh let's in this talk uh I want to get beyond the basics and uh give you some idea of what it's like to go into production with a web assembly app um so we're going to talk a live talk about a live production web app uh that was that comes from a large codebase and talk about some of the advantages that we got from uh using web assembly and some of the challenges uh that we ran into um so the context for this uh is an open source project called Rock which is a new uh programming language um and it's fast friendly and functional is the tagline we have so it's uh uh it aims to be faster than the garbage collected languages like go and Java and c and so on but not quite as fast as C and C++ and rust um so uh a language project is a huge thing and there's lots of sub projects within that and we're going to talk about one of those and in particular we're going to focus on the rock reppel um so a repple is a tool like this you've probably seen them before even if you didn't use that same terminology uh so it's just a a little tool you type into the command line and uh it'll run some code for you as you type it uh so it reads the input evaluates it prints it and then Loops back around again um and the question that we asked ourselves was can we Port this to web assembly uh now why would we want to do that well one of the reasons is it's really good for newcomers so somebody who's coming to check out our our new language this is a really good way to do it right you don't have to install anything and it's just it's right there and you can start playing around with it um another Advantage for for us is as we'll see later web assembly allowed us to make a static site out of this uh no no back end um and how is that possible well it's because Rock the language uh one of the things it can compile to is web assembly and the compiler itself is written in cost and that also compiles to web assembly so like I said JavaScript isn't the only option anymore and lots of different languages are compiling to web assembly and let's see what we can use this for okay so to give you just a better feel for the kind of tools developer tools we're talking about um I'm going to just actually switch over to to a couple of them here um so this is a website called repet and they have various different reels on their site for different languages so I can do things like x = 1 2 3 and Y = 111 and then I can go x + y and it's 2 three4 amazing um and so that's evaluating my code as as I as I type it in and python is an interpreted language so it's taking one line at a time and executing it one line at a time um there's another class of languages like compiled languages which are they're not interpreted they they they're compiled and usually in a language like like that you you have a different type of uh of tool on the web so um this is a website called compiler Explorer and uh I'm using the example of C++ language C++ is a much more static language and that means that you've got to actually give it an entire program before it will uh run anything you can't just do one line at a time um and what this website is doing is there's a lot more going on on this page but uh the main bit is you've got some code code in the in the middle in the upper panel here and in the bottom panel it's uh printing out um the results of that program so if I change this code here to uh to a different number you can see the bottom changes uh at the same time there was no meaning behind that number uh and uh so it changes live as I as I change the code here the answer uh updates and it's worth having a little look um at the developer tools to just see what's actually happening there so um if I change that by change this number again I can see HTTP request happening this is going off to some endpoint called compile uh which is doing what you would expect it to be doing so if I uh if I change this again and put in another digit there somewhere it goes off and compiles again so it's basically grabbing this code AS text sending it off to a server somewhere and uh printing the result out again um this is the rock uh Rebel uh which is what this talk is mainly about so uh I'll do the same kind of thing in here uh and use my same example um and it is doing uh doing the same sort of stuff so but there's a difference which is where's the thing developer tools so if I keep the network tab open again uh Z is 74 x + z there's no requests so this is actually all in the front end and uh that's something we did with web assembly uh and I've never actually seen another uh another tool like this uh use that so I think this is something we've done that's uh a little bit new um so let's have a closer look at how these tools work so the typical architecture you find and I'm going to use uh compiler Explorer as the example um is that you uh like I was saying you you have an input panel here you type in some some code sends that off to a web server which runs a compiler um and display sends that the result back through the web server and displays it out here um some websites so compiler Explorer uses HTTP for that um the other one that I demonstrated with python uses uh websocket um so but in our case one of the reason we wanted to explore web assembly was all of this involves man managing servers and we're just an open sour source project and nobody really wants to have a beeper that's going off because an open source project is something you do as a hobby in your spare time and mostly that's what we have so uh we really didn't want to be doing this uh and also for a language project you tend to get spikes of of traffic as as a lot of people do I guess um but if there's an article on Hacker News about you or uh you know somebody shares some article um or shares a go-to talk or something uh you can get a spike in your traffic and then you know that's bad news now you got to think about how do I scale my back end and all that stuff um so what we got out of using web assembly was we got this architecture where we just serve the website from a static file server so just like a a CDN these are pretty cheap and they have replication all over the world and everything you just load the the um the entire front end and it contains um HTML user interface that we were looking at um it's got an input at the bottom you type in your code it sends it to a compiler which is in web assembly in the same page so it doesn't have to go out to a server um and the execution output then gets uh displayed on the web page and so to build a reppel out of a compiler so I kind of mentioned briefly earlier the difference between an interpreted and a compiled language so this type of reppel setup is a little bit easier with interpreters because line by line is what an interpreter does anyway um for a compiled language the the other system like like compiler Explorer where you do the full program is easier to do uh but we really wanted this reppel experience because it's a bit more interactive and so how you build a reppel out of a compiler um is you take all of the input lines of text that have ever been entered in by the user in the current session you jam them all together you wrap them up in this main function so this is this syntax is probably not too familiar but it's just indentation means that it's inside the main function um and so we just grab all of those lines wrap it in a main function and return this X Plus Y which was the thing I uh last line I entered in so the last line becomes the return value of that function so we're basically grabbing some strings sticking them together and pretending it's a file because the compiler knows how to take a file and make some code out of it so so we just do that every time you type in a new line we make a brand new file that is all of that that is one line longer than the last time and uh compile it and run it okay so some of the challenges with this uh one thing is the download size uh so the it's a 300,000 line of code project it's quite a lot um the the binary size this binary size is just from my Mac that I'm doing this on uh which isn't web assembly but it's just to give you an idea you you don't want a page a web page that's 66 megabytes it's it's not going to be a good time so we're going to need to figure out how we shrink that that's one of the challenges and the other challenges are mainly around uh various web assembly limitations that exist so web assembly is not actually able to listen for user input so that's going to be a problem because we need to type user input in um it can only understand JavaScript numbers so if we want to listen for an user input with JavaScript that's going to be difficult um it can't modify the web page and you can't run web assembly inside web assembly so I I did say that both the language we're compiling and the compiler itself are in web assembly but that's not much good if we can't run web assembly inside web assembly so these are the challenges we're going to have to figure out all right so the first one download size it's way too big um so let's have a look at how what is inside this uh this project what how does it break down so in the we'll take the terminal reppel as the as the example we saw that at the beginning and so what it needs to do is receive some input parse it uh so that it understands what the text means all the different syntax type check it um specialize it which is just to do with uh working with uh generic types um then generate a binary and evaluate it turn it then turn the result into a string and display it um and so over on the on the right hand side of this I'm showing we've got a big part of the compiler which is for generating production binaries and it works with web assembly and also Intel and arm style um machine code uh that uses a library called llvm which some some people might have heard of but it's basically a big chunk of code that is there for optimizing and making sure that your final code code runs fast so that's a big piece of the compiler we've also got these other um pieces on the on on the bottom right here for generating development binaries um and we'll talk a bit about why that's different so these are um separate things that we built ourselves uh which just basically generate something as fast as it possibly can without worrying too much about optimizations and so the reason why we structure things that way we we've actually got separate code paths entirely for generating production V binaries versus development binaries uh and what I'm talking about here is um you know when you're using a compiler you've got two different scenarios that you You' be commonly be in so the production scenario is where you you want to compile your program once and then you want to run it lots and lots of times or maybe millions of your customers or something want to run it um so in that sort of situation uh fast runtime of the final binary is really what you care about and if your compile time is really long because you're doing lots of optimizations to try and get that final binary to be fast that's okay you're kind of willing to pay uh a long compile time because it's worth it you're going to compile it once but you're going to run it many times in the development scenario that's sort of the opposite because you want to compile it lots and lots of times because every time you do you find a bug and you need to fix it and need to go back and change something and then compile it again and run that and then you find another bug and you got to do that again so what you care about there is the total time the compile plus run uh time and in that scenario usually people like to run without optimizations um because that gets you the fastest compile times because all this optimizing takes some time for the compiler to do and so what we noticed was uh that this development scenario uh we we already had the compiler structured with entirely separate code paths for these and what we eventually what we noticed was the the development side of this is much better for the reppel use casee because you're only going to run the code uh one time because the next input line we're going to have a new program we're going to compile that again so the first thing we can do then is use the development uh code path for uh for the reppel um so we don't really need the production side and we don't need the Intel um output and we don't need the arm output so the way we went about slimming down the um the page size was to create an alternative entry point into the into the compiler when I say entry point I just mean what you would usually call a main function um so for the standard build of the compiler it's a command line tool so you're you know typing in the name of the file you want to compile and stuff like that with some options maybe um that's made up of a lot of internal packages so we have some structure inside of the compiler where it's a it it has a bunch of internal packages which are rust called crates and rust um and the entry point is in the CLI package so that's the one that's interpreting your command line options um and it's the one that Imports everything else and sort of glues them together for the web reppel what we did was just create a brand new entry point we just forget about the CLI because there's no command line interface on the web anyway so there's no way there's no need to be interpreting you know what minus minus ver Bose or whatever um so we just created a new entry uh completely and import only the packages or internal packages and external packages that we actually need for this web repple uh scenario so there's no command line interface as I said and none of this optimization stuff um no Intel output or arm output just only web assembly and that um enabled us to make a kind of size optimized build for this for for the web repple by removing all the code we don't need and we got down all the way to 6 megabytes uh of uncompress web assembly and then we can shrink that down again using uh compression uh we use brle compression which is kind of like gzip but more aggressive and uh yeah we got it down to 2 megabytes which is a chunky enough web page but it's acceptable uh so uh yeah all right so um the next challenges uh are coming from web assembly limitations when I say wasm I mean web assembly just an abbreviation um yeah so I mentioned these a little bit earlier just to remind ourselves can't listen for user input uh only only understands numbers and can't modify the HTML page and so on um to understand uh this a bit uh to get a better picture for for what this restricts you from doing uh it's good to understand what or to remind ourselves what web apis are so web apis are the JavaScript functions that are built into the browser they're part of theow are not really part of the JavaScript language as such but they're actually what make JavaScript useful in the first place there's about 120 of these apis and web assembly can't access any of them and that's important um so just what do I mean by apis well it's all of the stuff you you you've used these uh you all put up your hands as as web developers so you've definitely use this stuff uh one of the the most famous API uh web API is the document object model the D uh which lets you insert text into the HTML and listen for user input and stuff like that so you'll recognize get element by ID probably um there's no equivalent to this in web assembly and similarly for the fetch API which lets you make HTTP requests and uh interact with the browser to do that and again there's no equivalent of that in web assembly you just can't do it and this goes on right I just I won't go through them all but this is me scrolling through the m Mozilla documentation for all of the apis there's tons of them and um you know same thing for all of them there's no web assembly equivalent so the implications of that is that although you can kind of run any language you like in the browser now it does have to be restricted to basically just data Transformations you take data in one format and you turn it into another format and you spit it out the other side um and if you need to do things like IO or uh using anything that's built into the the browser you need JavaScript um so you can have any language you want in the browser but you also need to have JavaScript working with it in order to access all of this useful stuff so let's look at how we use those um and before we do we'll see well where where do we need them for this particular project that we're uh using as our example where do we need web apis well we need them for this is the same diagram I showed earlier on we need them for receiving the input we don't really need need it for parsing or type checking or specializing they're all just data Transformations or generating web assembly um we do need it to evaluate the web assembly because all we get out of here is just some btes and we can't we can't run those um turning that into a string we don't need it for that but we do need it for displaying the output on the screen okay so let's just take that I'm going to take that diagram and just rearrange it a little bit and show it kind of flowing uh flowing downwards um I've squished a few of those boxes into one here compile because lots of them are just data Transformations and we're not going to focus on those anymore um so we need our Dom API at both ends and we need a web web web assembly API in the middle so yeah let's start from the beginning and talk about receiving input um which web assembly can't do so if web assembly can't do it then we need to get JavaScript to do it so we'll receive our input string from the user remember I was typing some code into this uh web page we'll receive their input in JavaScript and then call some web assembly functions and then we can do our compile evaluate etc etc um I'm going to explain how all of this works using these kind of box flow diagrams uh the ones with the yellow background are JavaScript and the ones with the darker blue background are web assembly and uh yeah things on the left are calling things on the right so these kind of represent functions or chunks of code but let's look at some actual code for how that would work so this is the uh the rock reppel that I was using earlier on um and some simple JavaScript code to listen for changes on that would be grab the element by its ID and add an event listener and here's our event listener function on input change and it's able to grab the text from that input and call this thing called entrypoint from JS uh which is as the name kind of suggests is a web assembly function uh or actually a wrapper around an a web assembly function that we're calling from JavaScript uh there's a to-do here because we need to finish this code off I'm hiding some some bits for later um so one of the limitations that I mentioned earlier was that that web assembly can only understand JavaScript numbers it doesn't actually understand uh JavaScript strings or objects or arrays or anything but on the last slide I just said I was passing a string from JavaScript into web assembly so how does that work um well first of all just to get some background the way that JavaScript and web assembly interoperate together um is that uh they use the fact that they build on top of the fact that JavaScript engines use just in time compilation for JavaScript and that means that hot functions which are just functions that you call a lot maybe thousands of times um get compiled down inside the browser engine into this into an optimized form it's just down to machine code um it takes thousands of calls for a function to be considered warmed up enough to for this to happen but it does happen and web assembly essentially uses that and and just bypasses all of that warm-up stuff and compiles straight into this optimized format so it looks just like a JavaScript function that you've called thousands and thousands of times so the run time kind of sees them as the same and that's how the two can work together um and so to do interrupt you can do function calls both ways um but the arguments have to be numbers and uh the return value has to be a number if you have a return value um so this is again because it's comp it's it's this low-level um interface between the two there is some shared memory though between the two so the JavaScript uh sorry the web assembly uh instance has a memory and JavaScript can see that and it can write into it sees it just as an array of numbers so you can read it and write into that array of numbers and it's web assembly's memory so that's how you pass data structures by the way that only works one way the the the web assembly stuff can't see any JavaScript objects or can't look inside them it's uh but JavaScript can look inside web assemblies memory so let's look at an example of what this means for something relatively straightforward like passing a string um which is what we need to do now uh passing a string from uh JavaScript into web assembly so I mentioned earlier this entry point from JS and that it was a wrapper around uh a web assembly um function and this this is uh what it looks like so we're receiving a uh JavaScript string and we need to encode that into something that web assembly can understand so we need to uh JavaScript has this one of the many web apis uh is text encoder you can grab a text encoder and pass a string into it and it'll give you back the bytes that that encodes into um in utf8 then we're going to allocate some memory space inside of the web assembly module um because it needs to make sure that we don't overwrite anything else that's going on inside that memory um so we tell it how many bytes we want it'll tell us where to put that what index to put it at in the array um and then we can work out um we work out the end index start index and the end index and we set those equal to oops the bytes that we had and then we're ready to call the real uh web assembly function so that's wm. entrypoint from JS um we can call that with two numbers now because we've written all the bytes into memory and now we just need to say where they are here's the start and here's how long it is okay cool so we got some input um and let's look at how we do the other way it's displaying output uh now we said earlier uh web assembly can't modify the HTML page so again we're going to need to take this display output piece and move it from the web assembly side to the JavaScript site and this is something like how that might look so here's the on input change function that we looked at earlier we are again still just uh grabbing the input string the input text um from the event calling our uh wrapped uh entry point with the text um getting some text back and just shoving that into the inner HTML of of some output element that we've uh that we've got on our uh display area here and so how do we return uh a string from web assembly to JavaScript again things can only return numbers and again we're using this shared memory so um the way that this actually works um in this project I used a a code generation Tool uh that we'll talk about again later um wasm bind gen it's called um and this is how how it implements that so uh the there is a helper function in JavaScript that gets called from web assembly with a pointer a length so I uh that's like a binary representation of a string you just got a pointer for where it is and how long it is um we can then grab those bytes uh based on that pointer in length and decode them so it's just the exact reverse of what we did earlier um make a decoder call decoder decode and get a string um and the way this works from here is so this is being called from web assembly it creates that uh JavaScript string like this and then insert inserts it into a global array that's like actually a they actually call it a heap um push it in there and get a an index from it because we now have to return back into web assembly and back out to JavaScript again and we can only pass a number so it seems super complicated and it kind of is but this is a general enough mechanism that lets you create any uh JavaScript object um uh yeah and uh just refer to it using a number as an index um and so our rust code calls this uh that helper before it returns back out to JavaScript so it's um this create JavaScript string here at the at the bottom is is what's being called so this is our entry point from JavaScript that's inside the rust module it's taking the input pointer and input length um making a string out of that um evaluate compile and evaluate and stringify the input so that gives us an output split that into a pointer and and length again create the JavaScript string and uh put it into this Global array and get the index back so it's kind of a lot um so this is what this is how it works to return a string so we've and this is it in kind of more pictorial form so we've stringified the thing we made a JavaScript in a Glo uh string in a global array returned the index back got it out again and displayed it okay so that's kind of a lot of Hassle and this is why you use code generation so uh this glue code is actually generated by this package I mentioned called wasm bingen which is a rust um specific thing other languages have similar sort of stuff um so the handwritten rust that we actually write does just deal with strings it takes a string in and returns a string back um there's a link there for ASM b b Jen so we can use this kind of more simplified model um of uh just working with strings okay um so the trickier part is going to be evaluating the user's code and again we have a web assembly limitation that we need to deal with which is that we can't actually run web assembly from inside other web assembly um but that's what we need to do we need to uh take this X Plus Y and somehow run it so we can take our X our string and the compiler will turn it into an array of bytes um now normally a compiler would get an array of bytes and just write that into a file and then that's your um that's your compiled uh binary um but we don't want to write into a file we don't have a file system we're we're in a web page so we need to evalu we just evaluate those bytes but they're not in a format that's runable right now and so that evaluate step actually splits into two smaller steps um one of which is called instantiate and one is run the user code and so what instantiate means um is you take an array of bytes that's just numbers and you turn it into an executable web assembly instance and that does require um one of these web apis that I mentioned and then once we have an executable instance we can actually run the user code and stringify it and display it so instantiate um is yeah just creating an instance so we there's a this is one of the um one of the uh functions that are built into the browser that you can call you give it some bytes which are the web assembly binary format you give it an object with some call back functions in it so if you want your web assembly to call back out to JavaScript you put them into the into an object which lets you give them all names they they have like the keys are are strings and the and the values are functions so it can call them so what happens when you do this is it takes those bytes and loads them into the virtual machine in the browser engine um and uh yeah that makes it runnable so it then turns it into that uh shared low-level format that we discussed and you what you get back is a web assembly instance which uh contains web assembly functions that you can call from JavaScript so we're now going to go a little deeper right so this this is this is as far as it goes but we have created a JavaScript application that calls into some web assembly but that's a compiler and it's going to need to make more web assembly so we're now going to have to call some JavaScript that makes more web assembly and this is a more clear picture of what that looks like so uh there's our top level JavaScript receiving the input and displaying the output calling into the compiler web assembly which does the compiling and stringifying and within that it's going to call down again into some Java script and that's going to instantiate and run our user code then we have to create a new web assembly module that is the user's code run it and pass the answer all the way back okay so if you thought that was bad we've got to go one more step this is as complicated as it gets so just to give you an idea of the kind of things you can run into doing a project like this for real that is what we're here to talk about so that call back I showed that as one call back on the previous slide but actually it ended up being three callbacks and I'm going to explain why um so first of all uh we ran into a minor issue with async so the that instantiate function that I showed earlier on web assembly instantiate is actually async and if you've ever worked in any language that has async and a weight uh you've probably found that one of the things that happens is once you make something async then all its colors become async too all the way up the call stack and that's true here too so this instantiate is async which means that the rust function that calls it to compile it is also has to be async um and the uh receiver um what you call event listener uh at the top level has to be async 2 now there was some code in shared code in here that we are actually sharing between the terminal CLI reppel and the web reel and we just didn't really want to make that async all the way up because which is what would happen if this run user code and instantiate were stuck together then some of this would have to be async 2 um now we could have done that I it just seemed easier to spit the function though so so that's what I did um the and so async interaction between um rust and JavaScript uh again a lot of that is helped by uh this uh code generation Library wasm bind gen which can tie together rust Futures and JavaScript promises which are you know conceptually the same sort of thing um and then the last complication was the um when we get the result of the US running the user's code we've no way of knowing in advance what size that's going to be we don't know how many bytes it's going to be so that means it's it's It's tricky to send it back to the compiler without doing an extra step we need this extra call back to do that so we had to return back um run the user code return back with a a result that says here's how big the result is um in you know a number of bytes or kilobytes or whatever um then Reserve that much space copy the memory from the user code's web assembly module to the compiler's web assembly module so that it can analyze it figure out how to turn it into a string and then display it and we're done okay so uh so just to summarize what we've uh what we've looked at um we got a major architecture Advantage out of using web assembly we've got no servers to manage and that means no open source contributors getting beat in the middle of the night uh we reduced the download size to make it fit into a web page in a reasonable sort of size um and to do that we made a dedicated entry point that's separate from our command line one and removed the subsystems that we didn't need um and I I think we we talked in detail about how that worked for our compiler project but I think the general point of make a dedicated entry point and remove the subsystems you don't need I think that applies to other projects too um we also talked about how you always need JavaScript uh you can't get away without it even if you're mainly using another language um so for things like HTML input and display um in our case for creating the the user codes web assembly module um but again if in your project that might be something else because there's 118 other apis that you you might need to use uh in the browser and we also talked about JavaScript and web assembly interrup um and how the engine treats web assembly a little bit like optimized JavaScript functions um there's a very low-level API between JavaScript and web assembly it's all just numbers and shared memory and so that means that code generation is your friend for things like strings and async and lots of other stuff and yeah I think it's fairly common in a project like this to have to reorganize um how your code is structured in order to work around the relatively limited um interface between the two worlds and that's uh that's the talk so uh there's some links here um which will the slides will be sent out afterwards um these are some links to the project and uh and some of the source code and um yeah don't forget to write the session thank [Applause] you