Programming in Go: Concurrency, Interfaces, & Design at Google I/O 2010

[00:00:08]hi everyone thanks for coming uh today we're going to talk about uh what it's like to program in the new language go um it's different from other languages it doesn't look very different but when you actually use it you'll find that it's actually quite a bit different to work in and so today Russ Cox and I are going to spend some time explaining some of the more interesting aspects of programming and go so this isn't really a tutorial um you'll understand more what we say if you know a little bit about the language but we've tried to pitch it at a level that even if you don't know anything about it you'll still get a feel for what it's all about um like all the other sessions at Google IO there's a a live wave where you can uh comment or ask questions and we'll we'll refer to it later in the talk so hold your questions until the end please and and if you have anything you want to ask please put in the wave which is at that address so uh let's start with Russ Cox to introduce some of these ideas R thanks R so today Rob and I are here to talk to you about programming and go and if you were going to write a a Python program you wouldn't start with a C++ program and translate it line by line and expect to end up with a very well-written Python program the concepts and the idioms and those languages are different enough that you just end up with a badly written non-idiomatic Python program and the same is true of go in other languages if you start with the concepts and idioms from C++ or Java or Python and you try to translate those line by line into go uh you'll just be frustrated with the experience on the other hand if you use the The Core Concepts and idioms in go to write real go programs then you we think that you'll uh get a lot out of the experience and you'll be very happy with the language and with your programs and so our goal today is to teach you the beginnings of how to think about programs in the way that makes the best sense in go so just to get us all on the same page let's look for a little bit at what go is and is not first of all go is object oriented but it's not type oriented in particular inheritance is not a primary Concept in the language and there uh there are no subclasses um in fact there are no classes at all there are basic types like int and float and bu and their are uh composite types like maps and structs and all of these can have methods defined on them second go is implicit in in two key ways uh first of all it's when you write a variable declaration in a function or or even at top level if it has an initializer you can omit the type and it will pull the type off of the initializer um and because of that you tend not to see type declarations inside uh function bodies and second and more importantly an object in go satisfies an interface which we'll see soon just by implementing the methods that the interface defines Point um it does not have to explicitly declare its intent to satisfy that interface if it has the methods it satisfies the interface and we'll see example after example in this talk of why that's really important and finally the emphasis in go is on concurrent programming rather than parallel programming and by concurrency I mean a way to structure your programs so that they can cope with having to do many things at once and having to deal with many simultaneous sources of input like network connections and and do that in a way that still lets you write a simple and well structured program now it turns out that if you have good support for concurrency then it's not too hard to turn that into good parallelism on a multi-core machine today and and go can do that and it can keep all the cores running but the focus is really on concurrency and how it makes it easier to write your programs and parallelism is just a nice benefit from that so to start we're going to look at a simple expression evaluator and just to keep it simple this expression evaluator has values uh that just have binary operations on them now if you were going to do this in a a language like C++ or Java you'd start with an abstract class called value and then you'd have concrete implementations for things like integers and strings and in go we start with value but value is an interface and an interface defines the set of methods that must be implemented to satisfy the interface and for our values we need two two properties we need to be able to uh combine a value with another value in a binary operation then that's the binary op method and the receiver is implicit and so the arguments to the binary op are the operation itself as a string and the the right hand side value Y and it Returns the result value and the second method that any value in our evaluator needs is a string method and the string method returns a string so that it can be printed and if you implement those methods then you have a type that the evaluator can pass around as a value and you don't have to explicitly say I'm trying to be a value so let's look at an implementation of a value these are just integers and and for the most part you just write the methods the first line there declares a new type Capital int that has the same memory representation as a standard int but they're different types this is not um it's not like in Java where integer um is a class and has all the boxing around it they're the same representation in memory but they're different types and in particular because we just defined Capital int we're allowed to specify the methods on it so the second line defines a string method on int and the the parameter list between the words Funk and string declar receiver of type int X and that string method returns a string by passing X to I to a from the ston package and the rest of the slide is the binary op method for INT in the binary op method the goal is to compute X op Y where op is the operator and we know that X is an INT but we don't know anything about y other than that it satisfies the value interface so the first thing we have to do is use a type switch on y to look at what type it is and pick off the types that we expect to be able to to handle and in particular if there's another if Y is also an INT then we can do the usual arithmetic operations plus minus star and if it's not an INT or if um or sorry if it's an error which we'll see on the next slide then we just return that error uh as is and on the next slide we'll see why and otherwise we create a new error that describes what failed because this is a failure the the leftand side and the right hand side and the operator that combination is not valid and so we create a new error now what was this about errors well for error handling we're going to create an error type that also satisfies value and so the result of the computation is a value but that value might itself be an error and it's just going to propagate the error up the evaluation and so again this this first line creates a new type called error which has the same representation in memory as a string and the string method on error just returns that string but it has to convert it because error and string are are two distinct types and um second we have the binary out method and here there's no computation to do there's already been an error on the left hand side of the operator so we just return that error and notice that in both of these examples we didn't have to write we're implementing value the int and the error type and the objects of that type are values just because they have the right methods so the final piece we need for the evaluator is some way to create values we need a a some way to take a string that's been typed in and create the appropriate value and so here's here's the implementation of that it's called new Val and it calls a to I on the string literal and then if that succeeds it returns an INT and otherwise it returns an error saying this is what you typed but it's not valid and the evaluator has a parser which we're not showing you that just tokenizes parses and in effect calls the new Val binary op and string Methods so let's look at a demo here's our evaluator if I type two then the evaluator has printed the answer two but it's also shown a trace of what calls it made and first it called new Val on the string to that I typed in and it returned new Val returned the in two and then it immediately turned it back into a string so that's not terribly interesting if we do four it looks the same but if we say 2 plus 4 we get the two new Val calls and then we see two called with a binary op of plus and four and it returns this new an six and then six has a when you convert it to a string gets back the the string to print on the other hand and so if you know if we do something like 2 * 3 + 1 that the right things happen on the other hand if I type X X is not an integer and I get the error back and if I say 2 plus X you can see that the two got created and then the error gets created and then the error gets passed to the binary op for plot for the int and it gets returned back and we end up with illegal literal X now it would be nice if we had Str strings in this so let's try a string and it doesn't work so let's fix that if we want to add strings we Define a new type Capital string which has the same representation in memory as a normal string and string string method Returns the quoted form and the binary op is the same form as before we look at the type of Y and we pick off the operations that we want to support and so uh this slide we have string plus string and string star in as valid operations and we have this error type um and again we return an error if it's not valid now notice we just defined a second method that has a rep or second type that has a representation string with different methods so we have error and we have Capital string and they both have a string method and a binary out method and they're both represented in memory as a string but uh I pointed out before that when you want to convert between two different things with the same representation you have to be explicit and this is one reason why so that whenever you're looking at an expression and with a method call you know what the type on the left is because there's no implicit conversion and so it's always clear which method's being called so we also have to change new Val to recognize the quoted strings that we type in so if a to I fails before we return an error we try to interpret it as a quoted string and if that works we return the string and notice that again we just added the string type by writing the code that was necessary for Strings there was no bookkeeping that was necessary so let's look at this so there's our string new Val Returns the string hello and we can say things like hello time three and it does what you would expect so if we step back for a moment if you're coming from Java or C++ or python the thing that's missing in this program is a type hierarchy and in Java and those other languages the type hierarchy is really the foundation of your program and you have to put it down before you write the rest of your program and then when you get halfway through your program a lot of times you realize oh I should have structured it differently and at that point it can be hard to change and in fact it's often easier to just struggle along with a slightly incorrect uh type hierarchy than it is to go back and change it now in go programming is not primarily about the types and the type hierarchy in fact there is no explicit type hierarchy and so the most important design decisions in your program can be delayed um and it's easy to change the types in the program later because the compiler can figure out the relations between them and you don't have to maintain that information yourself and we think that this makes go programs more flexible and more adaptable so um I pointed out that we have no type hierarchy so we have no inheritance so how do you handle a case where in Java or in C++ you would use inheritance so that's the the next example we're going to look at in Java uh if you had a a ZB compressor that has compress a compress method that takes two bite arrays but now we want to support a buffer compressing buffers and buffer is some other type and we want to do this in a way that's going to generalize to other compressors so if we have some other compressed function we can we can reuse our effort so what you would do in Java is you would define an abstract compressor class and it would have an abstract compress method with the byes bite arrays but then it would have a concrete compress method that talked about buffers and implemented it using the bite aray compress and then we would go back to our original ZB and we would add the magic words extends abstract compressor and this is very common jav Java style you inherit the abstract uh class and you get the the concrete behavior from it that you want to add now in go things look different so we have the same example again we have a ZB compressor and it has a compress method that takes two bite slices and we want to support buffer in some general way but in go we don't do it with an abstract class instead we Define an interface for the compressor and then we just write an ordinary function so we have a compressor interface that says you know something is a compressor if it has this compress method and then we have a compressed buffer function not an abstract class but a function that takes the compressor and uses it to compress from the input buffer to the output buffer and this is good go style you define what Behavior you need as an interface and then you just write a function that takes that interface this is easier and it's less typing with fewer types than in the Java and C++ way and you could use this kind of approach in Java but it's not common Java style because Java puts an emphasis on using inheritance but it's not just a matter of style even if you did it in Java um go has significant benefits over the Java interface way too so first of all in go you can use as many wrappers as you like uh because a type can satisfy many interfaces where as it can only inherit one abstract um from one abstract class in in Java it can only extend one abstract class and a bigger deal is that we had to go back to our original Java program and add the words extends abstract compressor and what if it's not what if that code is not yours to edit what if it's in some standard library that really shouldn't know about this new thing you've defined in go because things are automatically inferred by the compiler the compressor the implementor of compressor doesn't have to know that there's a compressor in and similarly um the definition of the compressor interface doesn't need to know that it's enabling this compressed buffer function and so all of these types can be in disjoint pieces of the Library without explicit dependencies between them and notice that in Java even if you used interfaces you'd still have to go back and say implements compressor and so you still have this problem that you have to go back and edit the code and introduce a dependency now that's one way that you would use interfaces to do something like you do in other languages but I also want to talk about the way you ways that you use interfaces just and go and are more unique to go and one of them is one of the reasons that there are more unique ways is that interfaces are so lightweight in fact a typical go interface has only one or two methods um and programmers who are new to go tend to see interfaces as a building block for type hierarchies they see it as a way to get back to hierarchies and big classes and they tend to create interfaces with lots of methods but we found that program and go it's not quite the right way to think about them when we think about interfaces they're often small and very precise and Nimble and and often even ad hoc because you don't have to go back and tell the other classes or the other packages that you're using these interfaces so this is a real example that we we did a few weeks ago there's an RPC package in go and it uses a package called Gob to Marshall objects on the wire and the only language that has support for Gob format is go and we thought it might be nice to talk to other languages and so we thought well what could we do to make this work with Jason and so that we could speak standard Json RPC so we abstract the codec into an interface and This Server codec um is shown on the slide and the server codec says you can be a server codec if you have a way to read a request header to read the request body and then to write a response and the server will Loop doing that for a while and eventually close the connection and there's a similar client codec and then when we went to the the RPC code we just had to change the signature of a few functions so the send response function used to take a Gob encoder and now it takes this more General interface value and similarly the function to handle requests had the same kind of change and this is basically the whole change to the RPC implementation we took the opportunity to clean up a few other things at the same time but in general that kind of change is all you need because of the implicit typing that goes on in the rest of the the function in the rest of the program and notice that we're doing this after the fact uh it's really powerful that we can make these changes after the fact and um and not have to plan ahead for them this whole change the converting RPC to use the interface and then Json took us about 20 minutes and that included writing and testing the Json implementation of the interface and a trivial wrap around Gob to implement the interface and in Java you'd probably start with you'd start by factoring RPC into this half abstract class and then you would subass it to do Json RPC and Gob RPC PC but then you might have to decide oh but I want to have the same RPC server but with per connection changes and so that would be another refactoring and go there's no need to manage this kind of decision or this kind of type hierarchy you just pass in the codec interface stub and you're done so I have one more example of interfaces we've seen examples of using it to structure a program and refactoring and also to um provide abstract wrappers but one of the most powerful things in go is is the way we use interfaces for chaining and for defining um new kinds of of um of objects that that all satisfy a common interface and so um and this kind of common re uh common interface tends to arise organically it's not something that's planned it just turns out that we have a lot of different types that have the same methods and so then it becomes useful to Define an interface that that says to have these methods and then functions can use them so you you might be familiar with the io. reader and the i. writer interfaces which are very commonly used and have just one method each read and write it's really important that these interfaces don't need retrofitting to work with the existing code it's important that when you introduce the concept of an io.

[00:18:40]reader you don't have to go back to every single thing with a read method and say implements reader so this is a more complex example than reader and writer uh when we started writing the cryptography code we noticed that we had the first code we wrote was the AES Cipher implementation and it was a struct with some methods and the methods it had were block size which Returns the the current block size the encryption unit size and decrypt and encrypt and similarly when Blowfish and XT and others came along they mimic this this uh set of methods and we noticed well if we're going to write code that takes an arbitrary Cipher we should Define an interface with these three methods and then it will apply to all of them and so the the the block Cipher package defines this interface and then defines these block Cipher functions using the interface with things like new CBC decryptor new CFB decryptor new ofb reader implement Cipher blockchaining Cipher feedback and output feedback mode all taking this this Cipher and they're not tuned to specific cry cryptographic implementations so if you want AES in CBC mode you create a new ases Cipher and you pass it to the new CBC decryptor and similarly if you want Blowfish in CBC mode you can see what to change you just create a blowfish Cipher instead and there's no need for this cross product of every possible mode and every possible uh cryptographic algorithm which you see in other libraries uh it's so simple and go to compose them that there's no point in providing all the different uh compositions beforehand and this kind of chaining is is very useful because you can chain all sorts of different things so if we go back to the the reader example you might have noticed that the the readers on the previous slide or the they took a cipher but they also took a reader and returned a reader and the the reads on The Returned reader are satisfied by reading from the underlying stream and decrypting it and so we can write this function that decrypts and decompresses uh from a source file to a destination file just by chaining a sequence of readers together so the First Line opens the source file and then remembers to close it and the third line there uh creates a new as Cipher given the key data and then it passes that Cipher and the file to new ofb reader with the key data with more key data and the result of that is a reader and that reader that first reader R when you read from it it the implementation will read from F and decrypt it and return the decrypted data and then we can pass that to a g gzip new reader and then the the r that the gzip new reader function returns when you read from that it will read the unencrypted data from the the first R decompress it and return you the uncompressed data and then finally we write we open a new file for writing we remember to close it and we copy from R into W so with that I'll turn over to Rob to talk about concurrency thanks Russ so uh Russ has been talking about thinking about programming with the types and the way the types work inside go and how you structure a program given the kinds of things you want to do when you're combining different types and stuff like that I'm going to talk about a complete different way of thinking about go programming which is how we think about concurrency if you're working in Java you you start by thinking about what the class hierarchy is how you're going to arrange the the representations between the items um and that's fine but it's not go and go if you're writing something like a server or uh some some distributed program you can use the elements of concurrent programming to construct the design of your program with a very different feel and actually a very powerful one and and as Russ said early on it's not about parallelism concurrency is not about getting all the cores humming as hot as you can by doing Vector Ops or anything like that it's a but it is a way to make programs use multicore machines well and cleanly and and Easy in a way that's really easy to understand the resulting programs are structured well and they're very uh easy to change and adapt as as requirements change so in short it's about the structure of the program rather than the performance you get but the performance tends to come out anyway so we need an example and it's hard to do a really you know full-on example in the time we have available so we're going to take a very idealized one but it illustrates a lot of the points about how this stuff works so imagine you have um a bunch of processes that have work to do and they're going to send requests out to some smaller set of of workers presumably worker machines somewhere that are going to uh perform those operations for them and then send the answers back in the middle we want to place a load balancer that uses uh that controls the load across the set of worker machines to balance the load across all of the the workers and keeps it them all sort of evenly loaded we're going to assume that the workers work best when they have a lot of work to do simultaneously for whatever reason it just makes it work out nicely um if this was a real problem of course we'd use a lot of computers and networking and so on and so on so on but we're not going to do that because that just makes the ex exle longer without really adding anything to it so this is a very simple model but it's representative of the core of something you might actually want to do so let's start with what a request is going to be you're going to have some piece of work you want to do and you're going to send it across to the uh to the worker via the load balancer and then when the workers finish the task it's going to send the answer back on a on a channel to the requesting processing here's your result and then in the middle this balancer is going to have some Metric for the load on the workers that it's going to use to even out the load so it looks a little bit like this um across the top we have a bunch of requesters probably many many more requesters than workers but the it's really hard to make slides uh that look like that so just assume there's a lot more requests on the top the bottom there's a a smaller number but still substantial number of workers and then there's this load balancer in the middle that has a single channel for all the requests that are coming in and then it immediately forwards each request to the the light most lightly loaded worker that's that's running and then as the worker completes each each request it sends the answer directly back to the thing that requested it and also although it's not in this slide it's going to Signal the load balcer I'm done now so you can adjust the load you have uh stored for me so uh the requesters look like this you have a request represented by a closure which is the operation you're going to perform that returns some type which we're just making in here for Simplicity and then a Channel of that type which is the response uh channel so you're going to send a say basically run this function and then when the answer done send it back to me on this channel that's a request and then here's a really sort of simple-minded but but functional version of of the requesting uh operation it just sits in a loop um doing something else for a while probably and then sends a request and in the form of a closure on a channel to the work Channel which delivers it to the load balancer and then uh it sits waits for the answer to come back from the worker and once the answers come back it probably does some more processing on it now the worker is going going to sit there waiting for a request to be delivered now from the load balancer and then it just sits in a loop doing the obvious thing gets a request executes the function in the request sends the answer back on the channel and then tells the load bouncer hey I'm done and by by just s sending a point to itself on the done channel so this is very very simple but it works and it's important to note that the response is going directly back to the load balancer because the channel gives you the capability to tell the worker to respond to you even though the load balance in the middle doesn't doesn't need to know doesn't need to keep track of the connections between the workers and the requesters so now we got enough to sort of think about the load balancer and we need we definitely need a pool of workers that's obvious we represent that with a a slice and then uh the balancer itself is is a simple structure that has this pool inside it and then the worker channel uh so that the workers can signal when they're done that can be a single channel for all of them because it's not going to be very busy um and now at this point this this is actually the load B this is all we have to do we sit in a loop and one of two things is going to happen we're either going to get a request in which case we dispatch it to the most likely loaded worker or a worker tells us it's done in which case we update the status of that worker so that we keep the load sorted out so that's very simple now we just have to write two functions dispatch and completed and the way to think about that is if you imagine the the load bouncers basically maintaining a priority queue and the things at the head of the queue are the most lightly loaded objects so we want to order that Q by the length of the request Q going to that worker and to do that the simplest thing to do is to just write a heap so we're basically going to make a heap of channels where the channels are representing the workers inside the worker struct and then the load is represented by the length of those cues and to do that all we have to do is attach the methods for Heap onto the pool type which is part of the implementation of the balancer and they're very they're all very simple this here's the less method there's a few others but they're all you know like most of them are like one or two lines there's one that's about four or five lines but they're all pretty easy and once we've done that the pool of workers behaves like a heap and so now we just need to define the worker structure which just has nothing in it but a request uh Channel and the count of pending requests and then an index which is part of the Heap implementation so now we can Implement right here's dispatch all dispatch has to do is given a request you you pop the element off the Heap that'll be the most lightly loaded worker you send it to request you increment its pending count and put it back in the heat that's it and for the completed set it's the inverse you decrement the the count for the the load on that worker take it out of the Heap then put it back again that's the whole thing and this is all working because the the channels are first class values and the methods can be attached to this pool type to turn this pile of workers into a Heap Sort of postao so let's let's show it running here um so this is uh not very graphical but you can get the idea uh this this job has 100 requesters and 10 workers and the workers are uh the the Heap is basically the pool on on the left there shows the count of the load on each worker and then the the two columns on the right are the average and the standard deviation of the load and you can see that the load's going up because there's a lot happening but the standard deviation stays pretty small because the Baler is doing its job that's pretty simple um and just for reference I I made a version that uh just trivially does a round robin on the on the workers rather than a balancer and if you let this run for a bit you'll see the load goes up as it does in the other case but the standard deviation starts to grow at least it does usually yeah there we go it's it's increasing it's not nearly as evenly balanced as it was in the case where we actually used a heat priority Q for the worker so it's just you know proof of concept it's not dramatic there you go you see the loads starting to go very uneven across those workers so let's so to go back to the discussion here there's a couple things about this program that are interesting one is it actually works I've shown you almost the entire implementation just a little bit of setup it's missing everything that this thing is doing is synchronous it's it's everything blocks and yet the system is highly concurrent and there's not also a single mutex in sight everything is done the structure of the concurrent operations makes it possible to write a program like this it's totally non-blocking even though it's built out of synchronous operations and there's no mutexes there's no no worry about memory barriers or any of that stuff the fundamental properties of the language make it really easy to write software like this another nice thing is that this this notion of a closure and a channel is a pair it's a really powerful idea I can send work to somebody and include a way to get the answer back to me afterwards and that's a really powerful structuring concept that you see in a lot of go code and also you see these channels being parts of first class values that are passed around between between different go routines as they're running that's a really important concept that's missing from some other all but some other concurrent languages where the connections are much more rigid than than they are when you can pass channels around now this is a very simple example we could do a much more interesting case where we had networks and and stuff like that to distribute load across many machines and it would look fundamentally the same but you'd have to do a little bit of work on it to get there but it's it's not really relevant to the idea of structuring the program principles are all in this in this uh example so uh to conclude here I think we've shown that programming in go is not like programming in Java or C++ or python even though when you read the spec you might think it's a very sort of straightforward language there's actually a lot going on that feels very different when you write real code in it these include the fact that objects are not always classes um in fact it can be anything including we've seen examples with strings and integers and slices and structures and and methods being used on all those things um and so inheritance is not only not the only way to structure a program it hardly even comes up the inheritance properties of go are not very interesting compared to some of the other things we can do with things like interfaces you also don't need to write everything down in advance you don't start by designing your the whole super structure of your type system before you write any code in fact a lot of the relationships between types tend to come up as the program evolves or even long after it does like with the RPC example and because of this a lot of things are discovered long after the program is written that can be very pleasant it was it was amazing to us that we may Implement adjacent RPC by just substituting an interface into our existing RPC implementation in in as said just a 20-minute job and that's the kind of things very hard to do when when the structure of the whole program is based around inheritance and also um concurrency a lot of people think it's really about parallelism and it's not it's about structuring software so that it can adapt and grow in a very parallel universe um and it's not about getting all of the the the it's not about numeric computation it's about system structure um and as a result of these changes uh from from existing languages we think go is a very much more productive environment um you can get a lot done very quickly because things tend to be much more Nimble when you're working on them you can give a a method to any type which means there's all these opportunities for designing your program that are very different from a world where everything is a class and you think of much larger objects as being things that have have methods attached to them a lot of the bookkeeping that you see in type driven programming simply doesn't appear in a go program the power's still there but the compiler takes care of it for you and that's just a huge productivity gain right there and then this it's hard to sort of go through big examples but I think I've shown with the this load balancer example that concurrency is a way to structure a program in a way that makes it really productive to write programs that work with you know very high load and very interesting parallel environments and as a result go programs tend to be very malleable and adaptable and and and much less brittle uh as as conditions change and if any of this interesting of course there's a lot more available at the website golang.org but the most important point is that go is a lot more fun to program in and uh of course it comes with t-shirts and tattoos and stickers so we'll do a Q&A now but as you as you go there's a pile of t-shirts down there and uh feel free to take one as you leave if that's the sort of thing you like um I said as you go we're going to do a Q&A first um and as I said at the beginning there's a there's a a Q&A thing on this uh wave live wave thing but remember to go to golang.org for lots more information so you do this so there's also microphones if you want to do live questions but we'll start with the ones on the wave um whoops can you and if you're worried about making the next session I think that everything is delayed as much as we were so the next session has not started yet right and there's enough t-shirts for everyone so you don't need to mob okay uh can you comment on the suitability of go for deploying real applications today uh if not today what time frame also my comments on server with is death toop with mobile suit ability um as far as the first thing goes we're already using go internally at Google for some production stuff so it depends on on the particular job you have in mind but the language is is pretty stable the libraries are good and getting better rapidly and uh for things that um it's ready for now it's already being used for production stuff um I don't know if I Bas a start up entirely on go now but might I don't know um so as far as time frame goes is just going to get more and more mature of course over time but I think if you have an application today it might be the right the right language to do it versus server versus desktop versus mobile suitability um it's was originally designed for Server kind of stuff but it seems to work very well for a lot of other things the desktop capabilities are limited because the graphic libraries aren't quite there yet although they're they're underway and as far as mobile goes I don't have anything to say but we'd really love to see it in a mobile environment you have anything to add uh question here so I know you have an arm compiler and uh I'm just curious what the status when I think of concurrency and arm I think of interrupt service routines for embedded programming is it applicable okay have you done it you want to go that uh I think that we're still exploring how you would do something in a very very limited arm environment on something like the Android phone which is also arm you don't have those problems and so we've we've been running our arm compiler tests on on that kind of platform uh we do have one developer who's working on very very tiny embedded things and I don't know exactly what he's going to end up with thank you um what are your plans about portability of go applications between different platforms um there are no plans per se it's a compiled language and so if you talk about portability you mostly mean Source level portability we've tried to make the libraries very portable between different applic between different environments uh it already runs on uh Linux Mac OSX FreeBSD there's a Windows Port that's pretty far along um and some other sort of arm embedded stuff um I'm not sure what to say about that except that the language is intrinsically quite a bit more parallel than languages like C or C++ but not quite at the level of java because it's one level back in terms of the the way you compile but I think it's a very portable language honestly yes um could you address uh performance as compared to C C++ and Java for server side programming with multicores and um also um what if you need to interface to existing libraries in those other languages so uh working backwards the interface to existing libraries we have uh a basic tool that lets you call C code from go and there's also work going on to hook go up to Swig so that you'll be able to wrap C and C++ libraries with Swig um as far as performance we found that things that are are highly computational tend to be about the same uh maybe 10 20% slower with C code and equivalent go code the place where ghost SRS behind is if you're littering your C code with things like inline assembly and and simd instructions and things things like that then we can't keep up anymore but but if you're doing just sort of standard portable C code and you translate that into go computationally it's not very much of a performance hit at all you did a lot of comparisons with typed languages or C C++ Java and a lot of the structures reminded me more of functional languages like hascal or Lang and I was wondering if you could do a comparison with those and where go shines relative to those languages the the functional language comparison I'm not a functional adap I I think I'm a little bit comfortable with it but not very um the the language has full closures and so that's very nice to have but um a lot of the sort of magical type inference that functional languages do are not available in go the go type inference is very simple it's basically FR initialized you can derive a type the sort of meta type stuff that the function languages have simply not Ino and I think that would probably be the biggest difference in in thinking about them feel right um uh plan allow writing and and roiding in go there are no plans I'd love to see it um but I honestly we're pretty small team and it's not where we go but but the there's it's not an accident that arm is one of the one of the things we support um come back to that one yeah um I I saw a lot of of you saying oh well this is just you know to implement this is a bunch of oneline functions and uh I think um it seems like when you go from inheritance to sort of uh implicit interfaces you find yourself writing this do you find yourself writing the same on line uh on line implementation of the same piece of the interface over and over again in a lot of different places no um you there's something we didn't talk about which is embedding which um replaces a lot of that bookkeeping like stuff that you would that I think you're alluding to it's possible to to borrow the implementation of something else and drop it into your object and and have it bring all the types along with all the methods along with it and that to avoid a lot of that sort of repetitive I've got to add this this 10 on line functions to this thing I think that's what you're referring to yeah um current situation with generics yeah that always comes up um we spent probably more time on this one issue than almost anything else um and the short answer is that we're still not happy that we've been able to find a design for generics that doesn't break something in the language or become uh too confusing for users so we're still thinking about it it's as I say still incredibly active discussion we're very sensitive that we have a language that seems to work really well and we don't want to muck it up with a generics implementation that that complicates life when it shouldn't also speaking personally this is just me Russ May disagree I don't using having used go for as my main language now for almost two years I don't really miss generics they don't they don't come up the language feels different and a lot of the things that people want generics for are already sort of covered not everything by any means there's there's a lot of places where it' be nice but it's not clear that it's important to go as it would be in some other languages which is not to say if the right design comes along we we would not go for it we definitely would I also what Rob said a lot of people or at least a few people on the mailing list who have uh written significant amounts of go code uh have said the same thing that that having written you know a thousand or 5,000 lines of code they found that you know they thought they were going to miss generics but they don't actually miss them now so yes you mentioned that there was uh uh certain intrinsic properties of the language that make it easier to manage uh consistency of your state um in concurrent programming um I'm sorry it's hard to understand thetics are terrible can you speak a little louder yeah sure um you mentioned that there was a there were intrinsic properties of the language that make it easier to do concurrency um perhaps rules uh dictating how code that is running in parallel can be interleaved and how that affect the state of variables in your closure that kind of thing is that a short list of properties in the language that you could speak to now uh well there's there's quite a few but uh you saw a control structure in there which gives you a way to control uh access from multiple channels at once the select statement there the way the stacks work in go is um they start out small and they grow on demand so you don't have to allocate a stack and say I need 10k of Stack right now or something like that the language completely handles that for you you don't have to you never think about stack size um it's and they're they're cheap to do and recovered as necessary the combination of communication and synchronization that channels give you once you've used them for a while you see it's an incredibly powerful primitive it's much more interesting to program this way than by passing memory around and and using mutexes to protect it's also a lot easier to get the code right for me one of the things that that seems to sort of say the most is uh my office mate Robert grimer who also worked on go had never worked in a concurrent language before although he has spent a fair bit of time dealing with C++ and writing server software and that kind of and he find he's written a lot of concurrent programs in go now and he just finds they work he doesn't he doesn't struggle with the problems he had writing server software in C++ and it's not that he's got a concurrent brain I've been writing concurrent programs for many years now that this is all new to him but he finds it's really natural to just Express what he wants to do with these independently executing agents exchanging information it's just a very natural way to program so the language enables that and that's that's nice I would just add that in your question you you talked about you know interleaving of individual instructions and things like that and we found that if you're thinking about that level you never you just never get it right and what go encourages you to do is especially with channels to let the go routines communicate among themselves with explicit messages and and that communication forces coordination and so you don't have to worry about can these instructions happen side by side because you know you hand an object off to a g routine and now it owns the object and it does stuff and then it hands it back when it's done and there there's no actual Sim memory shared between uh independently executing go routine there can be Memory shared but but the convention is that you know you pass an object to a go routine and it owns the object now when it hands it back then you can start reading and writing again until I have a metaphor that I use to explain this to people imagine you have a bunch of work to do and you write and you and in if you're thinking about memory barriers and concurrency what you do is you have a piece of paper and you write notes on the paper and everybody reads his paper and you you mark up dynamically what people look at everyone grabs things and they come back and they edit this piece of paper and it's very easy to imagine getting confused by what's written on the paper the way you do it and go is you have a pad of paper and you write down one task and you give that piece of paper to somebody else and he can walk around the room and give it to other people to do other parts of it and when they're done they bring it back to you and there's never any confusion because you're never looking at more than one thing at a time it's a very sort of silly childish way to describe it but it has a profound effect on how you think about writing the software so next question is is there a go compiler for Native client in the works and there is a go compiler for Native client it is done and um you can you can use it to create native client binaries uh we haven't played too much with hooking it up to the web browser and and doing interesting things with it but all the the compiler work is there and ready for playing it might have been because it was short um but the Unix programming environment was the only programming book that I ever read all the way through and a lot of the reason for that is because the assembling different uh Unix shell commands and stringing them together is conversational in nature that's important to me I'm also very event driven in my thinking this language shows me a lot of both of those um so were you specifically going for that feel like a did you have a cognitive model of programming in mind that made this just like feel really right to you um not not explicitly but I don't think it's an accident I mean i' I've worked in concurrent programming languages for quite a while and a lot of inspired by the gluing things together model from from Unix long ago and some people make an analogy between a Unix Pipeline and and and both channels sending messages but also interfaces chaining things together like in the cryptography example but it's not an explicit goal I want to take the Unix ID and make it work in a programming language it just it's a good model and it has multiple domains where it makes sense so speaking of that model the working with programs that seemed to be complete when they were written but you can string them together do you have a model for sort of external visibility of the guts of someone else's program so that you can inherit from another complete app not sure I know what you're asking like tickle seemed when it was starting out to be a really good glue language that you could take stuff and you know embed it that's what Swig is for you have libraries and then you have uh seemingly complex atomic things that are good in themselves and then you string them together in a language that's more fluid so do you have an idea for how um go would handle the management between seemingly independent apps we're not really targeting um uh that kind of glue we're really targeting you know writing go whole go programs okay and we can interact with other languages through RPC or through something like Swig for just calling a library but in general uh you know the shell works really well for that because all the the pieces are speaking the same interface they read from standard input and they write to standard output and you know in go you just you have you have interfaces like that you have reader and writer but it's all within the go world and there's not really an emphasis on interacting with the rest of the world except through things like RPC and the sort of standard mechanisms so standard input standard output things like that and and RPC Works in this world rpcs can satisfy interfaces and you know channels can be passed over networks and things there's higher level things that go on but I don't think it's EX what what you're getting at um I was just wondering is there some model for like for for sharing code like I mean Ruby has like you know gems and it's easy to like take something someone has done uh or like Java jars and sure there's a there's a a program in the distribution called go install and you can think of it as sort of a distributed cpan basically if you put your code on on GitHub or Google code or one of these code hosting sites then you can say go install and give it a path of the right form and it will install that package package and then you just import it so does it like does it have to be in source code format or I mean yeah so you can't do a binary uh at the moment no I mean you're you're compiling for many different architectures right so the binary would would limit you to just the person who you know has the same machine that you do thanks uh when I was looking at the slide there I didn't quite see if you have a bunch of libraries together um how do you make sure the name spacing doesn't Clash is there some sort of name space or scope there's there's no uniqueness in the name space so if it does Clash all you have to do is rename one of them during the import for your particular file and then you know it'll you can use the different names to refer within that file and the Linker and the compiler and all these tools can handle the fact that there might be multiple files that say I'm in package hash as long as they have different import paths they're different they're different packages so it all it all just works okay EXC there there's no such thing as a truly Global name in go okay everything every name is relative to a package and even packages have the same name they are distinct entities in the world and you can resolve it all right and my other question was I saw that you had uh something like defer io.

[00:49:08]close uh what exactly was that defer defer is a concept in the language that says right before you return run this function call and so when I said defer f. close what it meant is that no matter how you return from this function call f. close as you go and that's a dynamic call it's not scope so so you could call defer f. close for five different values of F and it would close all five FS so if you had a loop opening files and you said defer all that closing that Loop it would close all of them so it's not a scope thing it's a dynamic thing um so I suppose this language is not statically typed that's right isn't it no it's statically typed is it it is statically type but it has some Dynamic typing um depending on how you use the interfaces sometimes narrowing has to be done dynamically okay but comparing it to Java a lot of checking can be done in Java at compile time which can go is very statically typed all of the programs that we were looking at were using interfaces and interfaces even interfaces are statically typed so where there was a parameter of type value there can only be an object that has the the right value methods but you could also have a parameter of type int or bull or float just like in Java and so but the the point is that you can't check everything you could check in Java at compile time isn't that right because if you pass in an object which might be of the right uh I believe that you can check everything you can check in Java in go at compile time it's very static and the compiler if you're not being careful with the types if there's something the compiler cannot prove that it needs to know it requires you to be very explicit and say I'm promising here and I understand this could be a runtime check and a runtime fault and so it's it's it's very much statically typed and you can use interfaces to make it feel much more Dynamic but in the compiler it's all static okay thanks so I've been uh I've been using go for the past couple months um and it seems like a lot of the work is like making it more stable and fixing bugs like are there any new features of the language um you guys are planning to add like I personally like to see more Dynamic features like being able to like manipulate types more is there anything you guys are planning to add public road map has uh some things but not in that general direction I think there's there's probably coming a time when uh Dynamic code loading will be available but there's no plans for that we haven't talked about it just sort of given the way the packages work and the structure of things I think that that's got to got to develop on its own um at the moment we're more concerned with developing the libraries to the point that people can use this for production in in more ways but you know we're always open to proposals so if you've got things you want to talk about you know let us know we'll we'll consider them I also think that language changes just take longer they take a long time to just simmer and and really come up with a good design and they take a while to roll out and so you're going to see fewer of them than you see you know new libraries and so in particular the recent uh changes to panic and the introduction of recover were things that we spent a very long time thinking about on the back burner before we we finally came up with a design that we liked and then it was relatively quick to roll out and so it's mostly designed two years of thinking before it went in so and and as far as what you were talking about for types in particular I think that the reflect package is probably in in need of a a redo and I think that when we do that you might see some more dynamy things like you should be able to just construct a type at runtime I think uh but but all of that is still very much just simmering in the back and eventually it will you know bubble over and we'll have something one thing we didn't mention because I said this is in tutorial is there is full runtime reflection for those who don't don't know that which makes for some really interesting possibilities any other questions how are we on I can't see the clock back there says eight minutes eight minutes okay RC can you take can you come to a mic if you're can ask a question so Graphics processors are really powerful is there plans for can you speak up I'm sorry is there plans for something like opencl you know to uh run stuff on the graphics processors Graphics processors uh there are no plans but hopes and dreams sure all right uh you mentioned RPC for inner program language stuff uh Google uses Proto Buffs and you have internal rpcs what about for other RPC systems do you have like code generation for Thrift which does provide RPC right now or is that something you could provide we don't but I think that the changes to RPC that we talked about here with that made made it possible to plug in Json RPC as the wire encoding I think you could plug in protocol buffer wire encoding I think you could plug in Thrift wire encoding and just use this the standard go RPC server and you could speak all of those protocols at once if you wanted to all right yeah thank you it's really straightforward to to swap the encoding up um what's the policy on what goes in the standard Library package versus external I've seen a whole bunch of stuff that's in the standard one and I've seen languages like Pearl and python that have grown a really large standard library and then had to uh gut it or clean it up later we're very very sensitive to that issue because we I think that some other languages I won't name them but I think some other languages have have not been careful enough about how the library grows so we've been trying to make sure that the the basic libraries that are installed with the compiler and stuff when you get it form a really good core set that a large subset of which will be used in most programs and things that are much more special purpose or larger in scope we tend to encourage people to make separate repositories for but then in turn we have a program go install which makes it very easy to import them if you want we want to make sure that the standard package is both useful modest in scope but and and very very well written I view putting something in the standard Library as a promise that you know something like this is going to stay there and so I think you have to be careful just to you know think about you know is is that General enough that but you just mentioned redoing the reflect package for instance I'm sorry you mentioned redoing the reflect package right but I think it's a promise about the functionality more than the details of the interface so for example you know someone wrote a fuse package recently and I think fuse is a good example of something that makes sense as a separate package and you can go install it and so it's no harder to use than the the regular packages but but it you know it doesn't really make sense in the standard tree I think that once go is a little bit more mature we might see people who build go distributions that have a whole bunch of other packages pulled in you know just as part of the tree as though they'd already been go installed right uh but but I think the standard stuff is really supposed to be the kind of things you'd expect in a standard library and going works transitively so it makes it nice so we've only got a few seconds left so if there's no more questions thanks everyone for coming and grab your swag on the way out