Web Audio Performance Optimization Techniques for Real-Time DSP

[00:00:00]is working yep hello everybody so i'm paul uh from mozilla i've been working on the web api for nine years give or take um and spoken at the mini web audio api conference um so might yeah so i i i rushed to prepare a talk so i don't think i have very philosophical or inspirational uh keynote type thing to say however i i am pretty good at talking about performance and reasoning about performance so that's what you're gonna hear today uh talk is called we need to go faster because we always need to go faster because going faster means you can do um you can do a lot more with uh same computer for example or cheap devices etc so i think that's important and uh now we are the stage where we can start continuing those problems and as we saw yesterday during a few talks there is um information missing in the greater web jp community about all of this so i don't do my yearly this year in webaudio today i'm gonna talk about exactly one api uh that is new um but all the rest is gonna be higher level topics and some demos of tools that we have um it was rushed so i maybe i i wanted to go with a very like philosophical title doing more with the zen of hyperphones real-time programming didn't feel right or uh 27 weird tricks to improve your audio code the 19s would surprise you i didn't like the tone um so i ended up going with something yeah shorter let's see um so let's get started real-time audio programming is demanding to the programmer and to the machine right it's if you've done it you know it's hard you have to care about lots of things lots of details and the machine is literally sweating when it's rendering your your audio it needs to do things extremely fast exactly when it's needed no delays no nothing and if there is any error then a human that hears the the music will will it will sound jarring it will a click pop a pause uh yeah you don't want that right as low level uh there's lots of maps uh often latencies are increasingly small in devices that we have so it's even harder um so yeah now on the web you know nothing uh you don't know what uh computer the the the user has or even if it's a computer at all maybe it's a ten-year-old android device you don't know that um i listed a bunch of reasons uh a bunch of things to consider when when when attempting to reason about what your code could do it's a lot all those items in a big list um will have consequences on how you uh can write your code and how will how well it will perform in the field like in real life in the hands of the user so now now we have the problem statements we have lots of unknowns uh we don't know anything but real-time audio programming by essence is about determinism right so essentially we're going to have a hard time so the thing is a web browser or a digital audio workstation etc also had the same problems digital your workstation can all can often ask the user to configure things in certain way whereas in web browsers we we don't really if it doesn't work the person will switch to another one or just say it's broken and the web is not ready for audio or whatever um but we have we have a few a few tricks we have a few tricks that we've amassed over the years um and uh and i'm going to try in this presentation to uh to kind of bring what i bring some things i did of least to make the code in firefox and other software very fast and very safe for real time uh to the web so that people that want to deploy their their audio programs to the web it will be as fast and reliable resilient as possible on any machine whether it is a phone super powerful computer old computer windows linux mac os whatever right so this is what we have at the beginning dog slow code so it performs very slowly it's maybe it's written in javascript maybe it's not even using type arrays maybe the maths are not optimal maybe it's not using a bunch of short tricks and then this is what we'll have in hand it's very fast you can see there is motion blur uh because it's moving very it's already arrived it's already arrived to you because it's finished did the deadline already that's fine so that's that's the end right that's the end result um and now i've got 27 slides to teach a few things right so we have four items today we need to care about the cpu we need we need to we need to understand what the cpu likes and then we need to mangle our code or our software so that um the cpu really like what we're giving it and then the cp when you do that it's just going to be immediately a lot better a lot faster then we don't want to forget the memory so the memory subsystem is the is the cause of most of your problems if you don't treat it with respect it will make your life horrible and then our good friends the scheduler is essentially the the the central uh backbone of a well-performing uh real-time application right you really want to to to to work with the scheduler to make all your real-time deadlines so that it sounds perfect low latency no pops no clicks no nothing in the end we like all those parts of a computer but we will always verify that things are the the way we think they are because it's one thing to make optimization but it's useless if you haven't verified that it actually makes things faster or better so in the end here we will i will show how to verify all of this but let's start with cpu so cpus today are extremely extremely extremely complex right so there is only one thing that can tell you that always work if you don't do something it's going to be faster like if if if you want if you thought you you had to do something to to for your algorithm but it turns out you don't well that's the best optimization you can do right that's like extremely fast prediction in in cpu usage i've got a bunch of keywords at the bottom of the slides that's what uh modern cpu do so it some people say for example when you write in c it's it's like um what the cpu sees well not at all that's a complete lie the cpu will see your assembly code and completely rigid behind the scene and execute it and yeah it's not it's gonna have widely different performance characteristics depending on various things so we can't reason only with assembly we need to measure right except if you are one of the cpu guru that exactly understand how microcode works inside the cpu etc but i don't think anybody here including me knows about that sorry now if there is one slide that you remember in this talk is if you do dsp usually personally and i know that some people yesterday found conflicting results right but the rest of us will explain why um now there is two parts implement using web assembly and then minimize js to assemble round trips js towards them round trips so i'm going to publish the slides there's a bunch of links but we won't visit them all today so js to webassembly calls are extremely fast they're a lot faster on firefox and chrome but they're catching up um the problem also with js to wasm round trips is the that you have to copy memory if you copy memory that it means you have less cpu cache utilization you just spend some time coping memory in the first place a bunch of bunch of issues so at the design stage when you haven't written a single line of code think about how you can organize your dsp code to minimize js to wasm round trips why not javascript um for starters and that might be the only personal thing here i think c plus plus or restore c it's a lot more ergonomic to write dsp right [Music] typing the code is simpler it's typed i like it better now there is lots and lots of free software or libraries you can copy paste right with a good license uh not so much in javascript from my experience it exists but lots of it is in c plus plus no risk of garbage collection at all which is the bane of our existence as real-time audio programmers you have a very tight control and memory layout and we will see that later it really is faster i know we had also conflicting results yesterday but we can we can try to measure and see what's going on and we can chat with the people that had talks yesterday we can we can chat to try to optimize um now wasm you will compile it using gcc or llvm or that kind of thing the optimizing compilers are magic makes everything faster the only way to do cmd programming which is to compute multiple values at once is in wasn't it used to be that you you you you would be able to do same cmd.js but that's not available anymore it was only in firefox and we unshipped it when we shipped um wasn't cmd for audio in particular it's ahead of time compilation and not just in time which means that before the person uh your user press play or start or whatever on your demo you can compile everything in one go and you will not have a single performance problem or recompilation during the lifetime of your app which means it's extremely predictable in terms of performance and also not only it's ahead of time compilation but the difference between firefox scroll and safari when it comes to wasm pure wasm execution is is very very small um i think we're still leading a bit on most benchmark but really it is not a big difference whereas in javascript depending on the direction of the wind and the season and which version of the browser you have and if it's happy or sad you will have very different um benchmark results uh depending on the web browser and regression do happen so if you have an app that works great in chrome 90 maybe it's going to work slower in chrome 100 evotech i mean it could be any version because regression do happen now if we take for granted that we're writing wasn't uh the nice things that will happen in the future but not so quite relaxed cmd um you can do one cycled multiply and add for to compute your filters for example for dsp people that's quite good that's available in firefox i think but not enabled by default and it's the only browser that implement it shared was a memory meaning you can do zero copy communication between wasm and javascript for example or between multiple between the main thread and the worklet etc so that's also very nice and then for the people writing big programs such as doors and that kind of thing where you cannot really assume what's going on people can load lots and lots of plugins [Applause] you will have to have wasm-64 to have very big hips more than four gigabytes of rams of ram so that's recent that's coming i think uh wasn't 64 is available now shared was a memory it might be only in firefox it's coming to chrome or it maybe got released react cmd really good for us uh math people is the equivalent of uh dash f fast maps on on on gcc and and and clang so yeah it's nice it's um it's quite cool now there is a common um a common thing in audio that comes from a long time ago where we used we we used to use to cache lots of things in tables like for example sign or logarithm or whatever that's not always faster these days right and we will touch in that in a few slides so you need to measure if that's the thing of this talk you need to measure measure measure measure don't take anything for granted cpus are extremely fast but memory is a lot less fast so sometimes it's better to recompute even something that looks complex um instead of storing it in an array but it always depends it it also can depend on the actual computer which on the web you can't detect right part of the problem cheating is okay that's fine we are not in high school anymore uh so as long as in the end it sounds the same you can cheat like you can cheat you call that an optimization um how much resolution or precision do you need in your algorithm probably not a lot it needs to sound alright that's about it can you do hybrid hybrid lookup table and interpolation can you have a very small table and interpolate um between instead of recomputing very complex function what's the input domain of your trigger function it's probably minus pi to pi or minus 2 pi to 2 pi and show you this for example this is sinus x in black and this weird formula in green is almost the same right by ear i don't think you could notice it um oops what broke it but the second one is 3.2 times faster that's the thing right but nobody normally notices and there is even better approximation right but the thing is this function only works in this domain right which which is the only thing you need for an oscillator because you've wrapped your face once it goes more than one pie then you don't care and yeah dirty trick but you know three times faster if you're doing additive synthesis by oscillator well that's gonna pile up um so yeah just cheat it's fine nobody will nobody will look people will look at how much cpu you use all right now the memory i think this is the most important and underrated thing and this makes all the difference in the world especially with dsp so memory access dominates they always dominate they start to not dominate when you've paid a lot of attention to it so now some some we're going to talk about uh scale fetching a single byte of memory is approximately 400 editions on a modern chip right so let's say like core i7 or apple m1 whatever so 400 can you imagine what you can do with 400 additional multiplication or subtraction than any of those you cannot fetch one byte if you fetch one byte it's going to fetch either 64 bytes or 128 bytes but it's going to be a line in memory right and it's going to be it's going to go in the cache right so if you fetch one byte of memory for free you have the 63 bytes that are after this byte right it's already been paid for and paid for so you need that's the critical absolutely critical point so if you touch an array you know you have for free the 63 more items or 127 on more recent apple silicon based things than iphones and ipads etc because it's all same architecture now to put things in perspective this is a big moment one addition 0.3 nanosecond give or take right this is a scale um if we scale it to one second then an l1 cache access we have 64 kilobytes of it on most chips it's two seconds that's fine right that's common case very common case l2 cache access when your data doesn't fit into l1 cache is nine seconds so you see in life it means like going to a kitchen and coming back to your office give or take right l3 cash access 16 megabytes shared across all cores of your machine 33 seconds probably something like taking out the bin or something main memory access fetching a single byte of memory six minutes to put things in perspective right so take care about how you access memory that's gonna be the most important thing you can do now we won't watch it now but there is an absolutely remarkable talk about by admiral grace hopper which is the person that invented the first compiler right so like pretty legit that explains what a nanosecond means a nanosecond is the speed of light in about this like this this big right so from here to here the speed of lights about one nanosecond and then uh you will check the check the talk because she she speaks about what it means a microsecond and that's pretty funny but yeah it's it's really big so main memory access six minutes scaled so it's the super super super super long now as i said things that should be you that are that are used uh closing time to each other should be closed in in memory together so so that you don't waste cash space so i mean i put a few examples filter coefficients any algorithm state really like the the the the pointer in um the pointer on the buffer that you are playing out um are you organizing your audio data in planar meaning each buffer of a ch each channel has its own buffer or our channels interleaved together so left right left right left right left right if your algorithm is does any kind of specialization where it needs to access the the left and right channel together to make some good computation it may be preferable to do interleaved right it really depends on the app no good answer it really depends um in the end everything is memory on a computer you look at your program and if everything you see is going to be memory accesses everywhere so you really want to reason about all of this now the one thing about audio programming is that you really want to work with the the computer so you want to be v to do very simple thing um arrays arrays is enough just arrays just a bunch of arrays indexes arrays small variables that's it like nothing complicated no crazy hash table like splay tree or whatever those are full of pointers don't not good with caches very often not always but very often that's why i have the second line up here um and yeah really in the end uh i think it was uh one of the originator of unix that says fancy algorithm are slow when n is small and n is very often small um so yeah there you go in the web audio api yeah we we are working on small buffers because our latencies are very small and so you you go super fast you take your for loop and you blast through the array computing your thing it's over you don't need to go fancy um reduce working set size so the working set size for a particular algorithm is going to be the number of bytes of memory it takes to be computed as i said the caches are very small the l1 cache the fastest is 64 kilobytes usually um you've noticed that the faster the cache is the the the smaller they are right so if you want to be really fast you can you can try to make it fit into the 64 kilobyte and when you start measuring you see there are performance clips each time you you go through the cache size and you can measure that it's absolutely trivial to do with a simple c program you run it and you see immediately when you graph you can determine the size of the caches um in wasm because you're writing c or rust or c plus plus you have extremely tight control on what type you can use so it can matter to use uint8 or uin16 instead of you in 64 because you can pack everything super tightly and and and maximize cash utilization so again a link that can teach you everything about it um that i that i like um now there is a thing because we in audio we often have very big audio assets so we make copies so good rule of thumb you can keep that in your mind ddr4 which is soon to be replaced but not quite by ddr5 is about one millisecond per megabyte of memory of copy right so if you're streaming audio it takes one millisecond to copy memory if you if you're dealing with big sample banks like an orchestra piano whatever you're going to copy a lot of memory and this becomes significant on those modern apple machines that are not everywhere uh it's a lot lot faster but you can't really count on it because again you don't know what your code is running on right so you can play with it it's really impressive but um when you copy memory it's gonna go through the cache right so it's gonna trash everything all your nicely laid out cache um and packed structure are gonna be evicted from the cache and you're gonna have to refresh the memory again so you really want to do memory copies before everything or after everything certainly not in the middle of computing your algorithm right and very often you don't have to copy large arrays in one go in audio programming because we are really temporal based we care about a moment in time that we need to render we don't care about what's coming on in one minute very often so can you figure out a scheme to stream your memory to the cpu or maybe to to stagger the the loads so do it little by little um instead of making a big copy and one thing that people don't care about enough is that if you do fancy graphics and we like fancy graphic honestly very often the graphics processor uses the same memory as the cpu and so if you're doing very intensive graphics your cpu memory fetches will be super slow and that's reasonably hard to diagnose and if you want fancy graphics then there is no other way right you need the gpu will need to use some memory and they will crash your computation and that's annoying and again it's better with a dedicated gpu but you can't you can't really count on it except maybe if you do an installation or something but that's special cases really um i want to stress this if you do heavy memory operation everything else is slowed down um you might be on a real-time thread perfectly without any locks or anything slow but if another thread is doing a big copy in the same machine and you're trying to fetch your little audio buffers in your real-time thread it's going to be slow because the memory bus is shared and yeah real time no real time high priority low priority doesn't matter it slows you down um so yeah pay attention to that try to reason about the whole system and if you for example if you disable the graphics and it's super fast you know the rendering you re-enable the graphics it's super slow you know why maybe you can you can change things do don't do things at the same time etc now the scheduler is really really important part uh for us is probably the most important part to have a solid performing web do api application even if it's not particularly complex if you screw up the scheduler even a simple computation will start glitching and the audience will not uh like it at all um so as i was implying it doesn't matter if it's fast or slow we like fast things because it means we can do more things and we use less cpu and less battery power or less watts what we care about is that it's finished on time right that's what we care about um before the deadline the deadline is imposed by the system you have no control in it um yeah so communication we all know post message from the main thread to the worklets right it's best to not touch it it's sad but it's the way it is right um to solve all your problems for free um i've written this javascript library that some people here have contributed to which is a a nice ring buffer that doesn't allocate doesn't lock doesn't do anything but let you communicate between the audio worklet and the main thread or the worker it's an npm it's got a bunch of tasks etc it's got a it's got a nice little demo site with a bunch of examples etc the code is readable um the code is readable and uh the only thing you need to start using it npm install on your project and set a couple http headers so that shadow buffer is available if you want to know more got a big blog post uh about it um and this is the kind of speed up you can get for free right so this is for example on chrome post message versus shadow buffer higher is better it's like insane speed ups if you don't use postmessage but we will check what happens with post message in a few moments so that's communication for you that's absolutely critical absolutely critical more so than memory i think more so than maximizing cpu usage that's why it's at the end what to avoid for now we have to write javascript when we do audio worklets it will change in the future sometime uh soonish you'll be able to just write c plus and just load it into an audio worklet a global scope and it will work but we're not there yet so try to keep everything simple um no garbage collection just simple for loops array copies you can use the ring buffer if you want read the articles etc they explain to you a bunch of nasty things that javascript can do behind your back for example it's a lot better to have two variables than an object that contains two variables it looks the same but it's not the same at all in terms of what the engine does and then some interesting and very small benchmark it's computing we're gonna check we're gonna wait for that oh it's it's really slow anyway um we can check that later um don't drop references it's better to keep a reference to an object and not null it than to know it and then have the risk of a garbage collection coming in and interrupting your audio right it's fine most of the time you can reuse objects that's also fine um modern garbage trucks are extremely good really really good i try i try to stress it on the on the audio um [Music] on the on with audio workload in firefox and you can go pretty far with garbage collection happening but in the end it will always glitch if especially if your performance is quite long because it's statistical right now sometimes you can give yourself some breathing room by increasing the latency of the audio context if you increase the latency it means that the web browser will render lots more audio in one go taking opportunity to keep the cache is very hot so everything becomes a lot faster it's non-linear but it's a very very nice speed up here i've got two examples that's in chrome canary on this particular machine if i request playback type latency it's going to be 72 milliseconds of output latency which is perfectly acceptable for lots of applications if i request interactive uh it's gonna be 32 right so it's a lot better and sometimes you really want the 32 if you're doing a music type stuff or a game but sometimes you really don't care you really don't care so give yourself some breathing room it's one line it's one line of code now we're gonna check that's the that's the eye candy part of the of the talk um how to measure everything in a web browser right first my motto uh you can say you optimized if you haven't measured that's what i say constantly at work so people i've optimized this function see okay show me the benchmark show me the profile you have a profile url um do you have a micro benchmark what does the micro benchmark look like is it reasonable does it look like something you would do in real life um as we hinted at earlier modern systems are way way too complex to reason about them in isolation right some simple algorithm you can check you can see that clearly it's going to be faster but that's the exception and not really the common case turns out measuring is extremely easy it's extremely easy right so stop guessing and start measuring so we're gonna we're gonna do a little demo now we're going to measure a number of things um so i'm going to use a web app written by ableton ableton developer jacques schedler whoops which one is it um so it's it's doing couple strong synthesis in in either javascript or wasm and it's simulating a large number of strings um and uh trying to load the machine enough until it starts glitching right so we can check we have a hundred strings simulated by power worklet we're going to use the message board and we're going to start by using javascript as the as the language in which the dsp is implemented you can start the audio we're going to pick like i don't know 400 strings or something see it starts glitching already can [Music] reduce i think that sounds all right so 200 message per javascript now watch because it's not going to take very long how do you profile things in firefox new tab profile.firefox.com enter click here new thing appears go on your page go here media start start audio do something it's hard now lots of things happening in audio that's why it's taking a bit of time too the markers chart right now i've got some interesting things that i can show you get my slides again i've got this nice little firefox add-on that can help you making sense of it so to install it you go to the slide you go to the url you just click on the latest version i'm gonna do it live up you install it like this add so now i've got the extension and it shows you exactly how long each callback took here and we see every i don't know how many uh callbacks we have one that is extremely expensive and is very periodic what's going on what's going on in this particular app is that we are using post message every now and then to send back some state to the main thread and that's blowing our performance see the algorithm is pretty well optimized here but once we do weird things with communication we blow the budget and we can't render any more glitches right so we can um we can stop that um we can start again trying to profile uh another thing so if we use shadow i buffer still javascript [Applause] lots of lots of [Music] things still start audio i'm going to start profiling [Music] done we're going to look into the same graph again mark your charts oh look at that oh like super flat right we have it's always it's almost always except here i don't know what happened um almost below the line we have rendering a lot more strings uh it's super flat so this is what a good um audio workload looks like it's like almost always constant regardless of what you're doing and it's rendering a lot right so we can see when we send back the data to the main thread we still see little spikes but they are not very problematic at this point so look for this now just for kicks and giggles we can do web assembly that kind of thing um almost four times the number of strings i think this macbook goes at 3000 so with wasm it's even it's a lot better that's a very simple algorithm by the way it's a free it's open source you're just on the page you can read it look at what what tricks jack employed um but like not nothing really complicated um marker chart native audio callback zoom in a bit uh my extension see it's so low that only the communication spikes and we're 400 times the four times the number of strings it's still way under budget it's it's um it's lower than four times less strings in javascript right because it is a well-written code by an ableton person that does dsp all day so they know they know what they're doing um and yeah this is this will never glitch right it's impossible so this is into what uh for example firefox is doing you can also click into the other lanes here and zoom into it uh you can you have every little part of the rendering here so for example we had questions about how fast it takes javascript to store some call and to go back to js well you can check here if you you have uh this is what calls the javascript function and this is what uh the was i'm executing right so you can measure precisely what is your overhead um and uh this goes down to nanoseconds right nanosecond precision so you can really really be precise by uh what you're doing and and go far and then you can look into what firefox is doing so we have everything there what's really neat as well is that you have gc interrupts so anytime the gc kicked in you you can know it from the real-time thread so you can really know you nailed your architecture if you don't even see this line called gc interrupt right if it doesn't exist you have no gc that's fine never it will never block your thread so yeah that's pretty nice and then uh similarly if you want to do if you want to check what uh the main thread is doing because you want to do graphics as we always often do uh with um with audio you can really drill into your javascript code on the main thread and see what is expensive etc and you've got all the javascript stack frames see what's expensive and and make things really really fast so please you've seen how easy it is to to install and run you can also upload a profile send it to a colleague or to me if if you if you have a workload that doesn't behave properly shoot me an email upload local profile um and it's compressing it you can choose what we want you know you you all know we are big on privacy at mozilla so you can pick and choose exactly what you want to include in the profile and then you can either download as a as a zip or gzip or upload if you upload you directly get a nice short url that you can uh immediately share with a colleague and he will uh you can you can i mean that's the power of the web right url you'd send something there you go um yeah so [Music] and works on any version of uh of firefox uh you can also load in this very nice web app chrome chrome traces if you want to to be able to compare and upload et cetera use the link system back to the slides um so again yeah a long blog post about all this with details extension whatever so when you are i'm going to skip this but when you are reasoning about optimization you want to read this blog post well not as it's just a gist on github by my colleague marcus that explains to you how it what it means to reduce percentage when computers if there is some little maths it's non-intuitive but when you get it you get it it's not long uh but for example when you say oh i've optimized this by 20 you've probably optimized it by a lot more than 20 because the maths between reduction are slightly counter-intuitive but this this hopefully explains otherwise i can also explain so make sure when we when you optimize and lastly the one api i'm going to talk about which is very related to what we did today the audio render capacity api which is soon to be shipped in chrome and soon in firefox as well so you can have you can measure the performance of an audio context rendering while it's running and know how much dropouts you did and if there was dropouts you can scale down your processing immediately so that you can for example adapt to a less expensive reverb if you're on a phone for example or you notice that the computer is idle so you can use like super nice virtual analog filters or whatnot you can go both ways so as always it's in the spec so look at it it's really quite straightforward it could have been simpler but we didn't want to leak too much info about the machine of the user so there is some provision for privacy but yeah you will check it's it's really quite straightforward um to to use please use it so that uh users that have cheap phone or very expensive phones or desktop laptops etc have the best experience possible and the best experience is always to not glitch very often the super fancy convolutional reverb doesn't add that much to the experience compared to not glitching and that's it uh that's it for today i hope it was not too rushed uh for you i had a couple days repair so that's that's it and yeah i don't know if we have a few minutes or [Applause] thanks a lot paul do we have any question yes okay thank you uh yeah i had one question uh you wrote a slide about javascript glue code yep and how for now it's still sort of a necessary evil but you know soon you'll be able to load your compiled c plus plus directly into an audio worklet so i'm just curious uh you know what what is the remaining work there to be done and what would that look like does that involve like uh the wassi or so generally uh is to be able to expose a module in wazom uh emoji in the ecmascripts uh sense it also uh we need a bit of a bit of specification so that the audio buffers uh that the worklet passes to you are directly from the wasm heap so you don't do copies um and yeah that's it essentially okay and also go from a class-based interface to a function-based interface because um wasm really likes functions and doesn't really understand about classes so that's it um i assume it will be also useful for people that really prefer function style and not object style but yeah it's going to really help for wasan the wisdom story yeah i'm looking forward to that thank you uh i was wondering how much processing power and memory does the profiler takes when it's running well we can we can check because it's a profiler okay so let's see we go here an auto tracer here right so and this another one here four times so out of yeah fourteen thousand um record like a stack unwinding uh we picked up give or take how many 12 like 13 times so it's all right all right to say yes it's fine we we have grand plans to make it almost undetectable uh my colleagues work on that i help sometimes on a profiler but that's not my main thing at mozilla but yeah it's really really quite good if you really want to have even less [Music] overhead you can decide to not collect stack prices but only collect markers and the markers are really what we want as real-time audio programmers markers i made i made them completely free by using some log-free techniques inside firefox so that's that doesn't really have any overhead especially with any meaningful audio workload so there is an option so if you go to the advanced settings here you can tweak exactly what you record and you can reduce your overhead or increase it if you want to if you want if you want more info to this conference over a year over the years i have the impression you're diving more and more into the low level details which eventually certainly has interesting results in terms of performance i just worry about a little bit about you will you dive up yeah so as a certain point and we're going to touch on that tomorrow it was my originally scheduled talk for tomorrow morning but i had to rush that instead tomorrow i'm gonna remove my mozilla hat and put my w3c editor apps and talk about what's coming up in a spec and you will see that not everything is about the worklet um and i will show exactly the scheduling we have and the priorities that we have at the working group it's all well labeled uh hopefully readable i mean that's still spec lingual etc but so yeah not everything is about the worklets but but the the thing is uh we are trying to finish the worklet and to bring some relief to all the people that really need very high performance to bring products to market uh we have some dows now uh online which wasn't possible before so because we have cut out so long for non-advanced people now we are taking some time to to do it and then we will circle back again and again and again as always in software i do have a question okay so um if i'm going to uh to develop some audio software i'm not going release at such a low level because i'm going to use a higher level language and maybe compile it towards them so the only way i can have some kind of control is by checking the types and the arrays where i'm going to store my objects right or is it useless to think like that at this level of development no no it's you have to think about it from the get-go okay i think think about making everything small okay and the other thing is that if for some part of my development i'm using the standard audio nodes that have the simplest plus code in the browser then in that case wood making hybrid development sometimes with custom wasm audio worklet code and sometimes relying on predefined nodes do you recommend this or yeah i mean it's always a trade-off between using a library and writing everything yourself and maybe the person that has written the library that you're using is really good okay and my my last question is that is related to my students work you said don't release references don't remove references but in that well we you saw the demo yesterday where we had the plugin that was loading plugins and sometimes we remove the plugins of course but the sound is always being processed so i i asked my student be sure to unlock properly the plugins yeah i mean it's a trade-off it's a trade-off sometimes that's what you want to do but you want maybe to stagger them so only do per render quantum you want to learn one okay so that like you don't unload like 15 of them because you are at the risk of okay making a glitch so that means that when we change presets you know when you remove change preset what you can do is duck the sound remove everything and then ramp up okay right that's because this is a technique that's used in commercial products that's a good advice listen students and a last comment that maybe is not related but it's related to a norbert command is that i presented a proposal we've been developing with several people about a web audio plugin standard called web audio module so yesterday you heard about this distill and you could see maybe the demo by my students and some of the commercial doors are using such plugins and so on and i presented that at the web conference in last april during the developer track and it was a w3c developer track and the chairs uh told me do you need help to make this a real app application standard and not a browser low level standard and i thought that the w3c was just interested by protocols and low-level apis and such things but apparently what i understood is that since they started to work on autonomous vehicles protocols and applications and so on they had to standardize things at a higher level so dominique as hellmaster from wrc asked me if we would need some help for trying to write a real specification for ir level application level standard the only thing paul is that we don't have the manpower to do this right now and the people in the group are contributing you know on their free time and this is a really a work that needs some expertise you you know it you are the editor of the spec and you know the work it represents but yes apparently norbert the w3c is trying to develop this but so far we don't have many examples i think of such standards yeah it's not very common but really good yeah yeah so that was my closing comments i like standards it's very nice okay so thanks a lot paul for by the way will you share your slide yes but github is a problem i can't create a repo okay so the url uh is going to be yes oh okay send it to me and i will add it to the program of the the conference or you tweet it or any way okay thanks a lot so we will now start okay yes [Applause] thank you all for this wonderful keynote to my students lots of interesting information there too