JavaScript is the New Punk Rock: Web Audio API Music Creation

[00:00:15]I'm going to talk to you uh my name is First St memo I'm a developer from Glasgow in Scotland and I'm going to talk to you about an event that was just mentioned there that changed the world that changed all our lives it changed your life and and yours and and mine and that was this gig at the Lester free trade Hall in Manchester it was a Friday 4th of June 1976 this was the night the Sex Pistols played now this is one of their first gigs outside London they hadn't had a record out or anything and there were only about 40 people in the room but those 40 people were inspired they were inspired by the rawness and the the passion uh and enthusiasm that the Sex Pistols showed that night now Bernard suler from Joy Division new order was there he said this he said they were terrible I thought they were great I wanted to get up and be terrible too now uh he wasn't the only person that was there that started a band out of all those people who were there we got the following bands we got Joy Division New Order Buzzcocks magazine moris who was there started the Smiths after seeing the Sex Pistols as well as Mary Smith from the fall now Mick huckell was also there from Simply Red but we don't talk about that those bands who were they they were inspired by The Sex Pistols but those bands also went on to inspire other bands such as Aran Nirvana radio heads Pixies Bey boys and PJ Harvey all great mans and play but we don't talk about that okay so I'm going to talk about something that I'm inspired by and that is you know JavaScript I you know I love it um but I've also become recently become obsessed by something called the web audio API now I don't know if you know about the web audio API it's basically a kind of high performance way of manipulating audio in the browser via JavaScript we can take audio and process it and do anything anything we want with it now um it's only available in Chrome Safari and mobile Safari at the moment but Firefox is looking to implement it as soon as they can now the way the web audio API works it's this kind of concept of a node graph where imagine you had a guitar player with a guitar like this one they would strum a cord that signal would travel down their wire into maybe a an effects pedal like a distortion pedal that would manipulate the sound then go to another wire and then maybe to the speaker so in this situation The Source node is the guitar the Distortion node is manipulating that audio and then passing it on to our amplifier now the other thing you can do is that we can create audio completely in the browser from scratch we don't need to use an MP3 or a WB file or anything we can just make sound from thin air it's amazing so I'm going to show you that just now we just get uh the console up now the first thing we do is we create uh what's known as a context and this is like I don't know if you have used the canvas tag which kind of creates an area that that you can you can draw on in Java Script context is a kind of similar thing for audio it kind of lets you it's kind of world of sound you can kind of paint with sound that's terribly hippie-ish analogy but um that that that's what you can do so we create a context and then from a context we need a Sound Source so need to so in this case I'll make a sine wave but I'll talk a little bit later what a sine wave is so we do context.

[00:04:24]create uh oscillator okay and then we have to get a lead from from our swave do connect to our speakers and that's context.

[00:04:37]destination okay so everything's right plugged in ready to go the only thing we need to do now is start the S Wave off so oops uh dot note on and we're going to do that after 0er seconds so do we have audio out okay great so we should hear some beautiful sounds wow [Laughter] so we can just do not off so that that's us that say creating sound completely in the Rowser there was there was nothing else involved in that um which I think is amazing now so you might think Stuart that's just that's just noise that isn't music that's just a a sound do you think you are brain you know but no a similar thing was used in one of the best records of all time Good Vibrations with the beach boy they use a thing this was an electro thein which is slightly different than a normal thein if you wave your hands this one created a sine wave like we just heard but in a big wooden box now due to the wonders of its technology I've been able to shrink that big wooden box down into a web browser and we can see one here um so for got to do again [Music] it's very SP age so um so if you'll indulge me I'd like to do you a little uh a little performance with thin uh with help of my backing okay space the final frontier These Are the Voyages of the Starship Enterprise it's 5 your mission to explore strange new worlds to seek out new life and new civilizations to boldly go where no man has gone before [Music] [Applause] [Music] [Applause] [Music] [Applause] [Music] [Laughter] okay thank you okay so we've heard the S Wave and that's this nice round sound it's just you know nice and even and the thing is so so sounds H just explain waves so or what sound is sound is the again a hippie phrase the vibration of air it's this idea that the the AUD sorry the air is um oscillating at really high speed and then hitting your ear at like 40,000 times a second it's really quite something but the the way we graphically we can graphically represent sound using waves and so different sounds look different so we see the sign wave at the top here and so then we've got a square wave in a triangle wave so just kind of play one of these so we that's a square wave you can hear it sounds a little bit different than before and maybe a triangle wave is a bit softer so the way this this kind of way of of of taking waves and then manipulating and changing the shape is basically how to make a synthesizer and that's that's how we would do uh and that's what I've done here through the the magic of the web a API and Javas script we've got a little uh keyboard and so this one just basically combines two different types of waves um let's see let's have a Sawtooth and a triangle and you can hear um so this sustain we can put the sustain up and that allows us the sustain is if you play a note that's how long that note takes to kind of Decay and disappear so I kind of note like that hangs around for a while whereas one really short is just so again uh this is you know this is really easy to do this isn't any fancy this is just a different frequency a different pitch assigned to each type of wave and then they're combined um so turn the volume down that so so I don't not very good keyboard player but [Music] [Applause] so um so if if you want to make your own uh synthesizer it's really easy I've made this thing called quiry Hancock which allows you to um it's done in RAF and it basically takes allows you to take a a vector keyboard and stick it as a front to your own your own synth and it can take you know keyboard keys and stuff as well as the mouse so yeah all right the audio's gone on that anyway so what really excites me about this is if I'm not excited about all that other stuff uh is the fact that we can now write songs in JavaScript you know traditionally the way people wrote songs was you know they would you know sit a guitar or piano and they would write this lovely song and they would take it into the recording studio and get it recorded uh and you know even more recently people have been you know writing you know writing songs at computer but still this idea of you know you have a timeline it goes goes from the start all the way to the finish and there's all these multiple tracks and it's just you know what we can do is we can be we can write songs there's this this quote by the maths Wiz libnet and he says music is the pleasure the human mind experiences from Counting without being aware that it's counting and it's true music is math and um there's nothing better at counting than a computer so so does anyone have idea an idea what this does let's hear it so it's just a it's just a basic drum rock drum beats uh you can see that it's you know it's a for Loop and you know every second beat plays a kick drum every second beat after that plays a snare and every beat it plays a high hat now I mean those play functions are are are abstracting things a bit but I was using real audio that time that wasn't synthesized so they were they were samples of real drum kits that um you have to load these in if you're going to use them in the the web audio API it's different than just using an audio tag you have to kind of pull them in Via uh xhr and then kind of decode them to the context into or you have to load them into an array buffer so again I've created a silly little uh library to do this called ABY load that you can just give it a whole bunch of uh audio paths and it'll give you a call back telling you when they're all done in D coded and ready to play so with that we can do something a little more complex so we can mix those live samples with the synth I made [Music] [Laughter] [Music] earlier so and this is really easy to do this this isn't me being some sort of audio Wiz kid because I'm not this is just this web audio API that's just this Ena this um so that's fantastic okay enough of that okay you've written your JavaScript hit it's going to sell millions what do you do with it now well I guess you just put it on your website or you could upload it to GitHub and let you know people remix it for free you know you you can create remixes without this idea of throwing around massive audio files because the the file is already there this is what to go back to her Punk analogy the Buzzcocks did when they first started out they didn't have a record label they just thought they borrow money let's borrow money from friends and family they borrowed £500 and just went into a recording studio made a record themselves and put it out themselves they made a thousand copies and it sold sold out almost instantly so this idea of self-publishing you know it's it's more prominent than ever and it's so easy to do we don't even have to record things now that we just have a a file format that just makes sound it doesn't need to be recorded So this this idea that I'm proposing I only really have a name that I like for it at the moment it's a I imagine some sort of format like Json uh called J songs I don't have anything more than that um so if you do want to kind of uh helped me with this to make a kind of JavaScript spec for for songs on the web give me a shout CU all I've got as a title uh okay something even more exciting happened two weeks ago and that is that get user media for audio was enabled in the nightly build of chrome which means that we can now take audio exactly what it says we can get user media from from uh the the the the user so this allows us to take audio from a mic or a line in and manipulate it in the exact same chaining method I described before so I'd like to give you a little demonstration here if this works um so like I say this is only can I go full screen in this okay so this is the conference Crusher a little side the the more uh audacious the name of any AIO product the weedier it sounds okay so don't expect anything so the first thing when you when you use get user media you get this little option that says wants to use your microphone and we'll say oops yes please okay so we getting it is there something there yeah it's quite quiet so this is this is a guitar there there's no plugins here there's no Garage Band there's no anything this is just a browser this is just a normal web page that we're now getting sound through you know [Music] so so that's quite a clean sound right so that's just the way the way the guitar sounds through this little bit of of treble on it but what we can do is we actually take that connect it and actually change the wave forms that are going in okay so we can [Music] do so that's there's nothing more annoying than watching someone messing around a guitar so uh I'll spare you that so that was all just the same way before just the sound Going Down lead and then just the way form it's it's just a big AR Ray buffer that we can just flip about any way we want we have so much control over this so you might notice that there's a little uh Back to the Future button here just want to show you how much control we have do we have audio again okay so no oh no that that would have been good so I just do this that's a bit long no let me try again okay try it again yes so that's just showing you how much control we can have and that's that's all done with JavaScript I do we can just take it some audio in and just hold on to it for a bit and then do whatever we want okay so um I was really nervous about doing this talk because it was strange because I'm not typically someone who gets too nervous but this being JSC I thought these people are are the best at JavaScript whatever that means I thought I'm going to be ridiculed because I like semicolons you know um you know and so I'm not I'm not like I said I'm not an audio expert I'm not uh a JavaScript expert I'm you know if anyone says to me that they're an expert or anything just kind of get a little bit of sick in my mouth just you know I think it's a very arrogant thing to claim that you're an expert in anything because I think we can always keep learning now so I I I was concerned I was like why why is this bothering me cuz I used to play in various bands when I was younger and and we were rubbish but I still got up and just jumped around like an idiot and had a great time you know I couldn't sing it didn't stop me so by just chance I came across an old notebook that was that I had I used to write lyrics in which was hilarious in itself but I had this quote described at the start it's by David burn and it says the singer of his Talking Heads and he said the better the singer's voices the harder it is to believe what they're saying and this this I think is a great mantra for anyone who doesn't believe they're good enough at anything because what it means to me is that I'm going to take my weaknesses and turn them into strength that okay I might not be the greatest program in the world but there's other things other outside influences other things that I'm good at that give the the things I make character I'm passionate about what I do I believe that if you're passionate and enjoy what you're doing it doesn't matter what the end product is I didn't make all of these things because I thought I was going to change the world which I'm obviously not but I did it because I enjoyed it and I love it and I'm passionate about it I give everything I've got to do it I really really enjoy it and so that's why I believe that JavaScript is a new punk rock CU you can make anything from scratch and it doesn't matter how good it is just that you enjoyed it so I there's this video um by The Godfather of punk uh Iggy Pop that really I think Echoes the way I feel about what I do and what I think we all do and you know he mentions he mentions men but we hope he applies uh it to women as well and I would just like to finish on that thanks a lot [Music] thank [Music] you so you can follow me at these places I'm on tent if anyone knows what that he questions yeah yeah we have any questions don't be intimidated come on there one over there worst concert I don't know do you want to start a band with [Laughter] me you think of a good name we in oh yeah uh yeah I just wanted to ask really quick uh your obvious not using set time out there are you uh no so uh what's the web audio API has it basically has its own time system uh the when you create a context it has its own reference point that's in it does use seconds but it does give you kind of millisecond or even more than that control over the timing of things so what happens is you schedule things up front you say I want all these things to happen at these time these points in the future so it's yeah so no I've I've I've actually made some things or some things on uh my website that usually set time out like a drum machine when I was being shipper and I was like who makes a drum machine out Ste set time out OT hi Stuart great talk how do you how do you make the saw too and the square waves so yeah so um there are there kind of uh Native Concepts to the web audio API that when we created an oscillator at the start there that creates by default a sine wave now we just we can just change that type by giving it a different integer uh and yeah so they they're kind of native to to the API okay all right great thank you very much thank you