Anatomy of Cross-Compilation Toolchains | Embedded Linux Tutorial

[00:00:00]good morning my name is Thomas and today I would like to share some of my knowledge about cross-compilation toolchains I'm kind of surprised by the number of folks in the in the audience for for such a topic but well that's interesting so I work at for electrons acting as the CTO of this consulting company I mainly work on the kennel take time and on route nighttime sober truth is an embedded Linux build system and part of the knowledge for this talk comes from this experience working on this on this tool and happen to live in in southwest of France in the sunny city of Toulouse and on the the the right hand side of the slide here you can see a drawing from a very nice artist who drew sketches from speakers and audience members at the keynote recipes conference which takes place every year in September in Paris and it took place two weeks ago I think it's a nice conference so if you've never had the chance to attend I recommend attending it where there are very very good speakers that kind of a shameless plug for for this nice conference but to give you a chance to leave the room in case you're not really interested I wanted to just make sure that we're in the same page on what this talk is gonna be about I'm not the tool change developer I'm I work on a build system with many other people so I do have some amount of knowledge about compilation cross-compilation issues but I'm definitely not a GCC or banandles or C library developer by all means it's really some some experience gained by building simple tool chains for embedded Linux build system so this talk is really an introduction type of talk and not in depth type of talk so if you already have some knowledge about tool chains feel free to just leave the room right now I won't be surprised by that so we're going to focused on on simple GCC based tool chain we're not going to cover LLVM we're not going to cover advanced use cases like link time optimization graphic optimizations or things like that we're really going to stay in kind of the very basic cases I see nobody left so hopefully all of you are going to be interested but why pose in my talk so what is the cross-compilation toolchain that's amusing the first question to ask so it's a set of tools that allows you to build source into binary code for a target platform that's different from the one you're currently running on on the one that's on which the the build process is operating most of the time people think of it as different CPU architecture but it can also be different operating system DIF library different API and things like that so for example if you're on x86 and Linux x86 and you want to build an x86 binary for Windows you're doing cross compilation even though it's the same CPU architecture if you are let's say building on arm so your machine is an arm platform that's using a given API and when do you want to target a different API it's also cross-compilation in in this build process three machines are involved and the build machine which is where the bill takes place the hosting which is where the execution takes place and the Turcan machine for which the programs might be generating code so in most Linux distributions you have pre-built native tool chain which is the one you used to build application that you run on the same machine and this tool chain has been built with build equal us equal target for example you have an x86 machine it has been built on x86 it runs on x86 and provides and generates binaries that runs on x86 that's a Native Tongues in case the cross-compilation case is where build is the same as asked but different from the targets so for example you are building a tool chain on x86 running in on x86 but it produces binaries for a different CPU architecture or different operating system or different ABI so the target is different those three machines they directly correspond to the three arguments of all attack on configure scripts that have the same name build Austin targets so by default when you run it configure script it automatically guesses that you want to do native build so when you do cross compilation you need to override these to specify what you want to do exactly so the auto account specifies a concept of the system definitions that are represented by temples so a system definition describes the system it defines its CPU architecture its C library its operating system its vendor it's a B I'm and sometimes a bunch of other information it takes different forms the full form is the one that you can see on the slide its arch Bend or OS and then some combination of information about the C library and the API there are shorter forms that for example avoid the vendor part so the different companies that you have are relatively easy so arch is the definition of the CP architecture can be our Maps x86 and a bunch of others the vendor is mostly a free form string it's just a string that it identified the company or the entity that produced the tool chain there are just a few corner cases where it actually means something but for most situation it's just a free-form string the OS is the operating system you're targeting so for the case of this talk we're only going to consider the cases none and Linux and I'm going to explain in a few slides what what they mean but there are of course other possible operating systems and then leaves the API as the name suggests gives details on which the library you're using and which API are you using and we're also going to talk about C libraries and API is in the rest of that talk so here are some examples arm Fulani api is the tool chain topo that indicates that it's the bare metal tool chain so the norman part says it's a bare metal tool chain that targets the ARM architecture coming from a given vendor and using the ABI e API arm and on linux new ABI hf it's a linux tool chain as the OS part of the table suggests it comes from some unknown vendor targets the ARM architecture uses the Jolla PCC library and the EAP IHF ABI so all those information are encoded in the temple and it goes on and on like with the arm big-endian Linux targeting useful Ipsy using EAP I'm tapo which is the third one here or another one for the MIPS architecture that you see on the slide so the main two values of operating system were interested in in this talk are known and Linux and you saw some example of these and the previous lines sewn on is used for their metal tool chains so they are used for development without an operating system so directly on the metal as the name suggests the C library in use in such cases it generally newly so it provides C library services that do not require an operating system like Street manipulation and compare and other basic services that do not require an operating system beyond it and then optionally a new loop you can enable a number of basic kind of system calls by implementing Hardware specific operations or connecting that to your small real time operating system if you have if you have one but it's it's mainly meant for development without any operating system so you can use such tool chains to in in the Linux contacts to build your boot ladder to build your Linux kernel but you cannot use that to build user space on Xcode the second type of tool chain our Linux tool chains so they are used to develop Linux user space applications and share libraries so their main difference with bare metal tool chains is that they contain full feature C library that integrates with an operating system in that case all UNIX kernel and there are different C libraries that are available and we're going to talk more about that in the next slides as well and in that case the C library supports link system calls so when you're creating a network socket when you're opening a file or anything like that the C library talks to the the keynote to get access to this service as opposed to bare metal where no such services are available so such things can be used to build Linux user space code but they can also be used because they are a kind of more or less of a superset of bare metal to change they can also be used to build your bootloader or your can also if you've already building your ear canal or your bootloader you may be using a Linux capable tool chain even though the C library is not being used it just works fine so what do we have in a cross population toshin we essentially have four main components so it's not that complicated in the end if you you know dive a little bit into it it's not that that bad so we have four main components the B metals GCC the learning cane headers and the C libraries that's very much all what we need to build a tool chain from from scratch so a few dependencies are needed to build GCC I'm going to cover that as well but they are kind of pretty easy to to build and to use libraries so let's cover those components one by one minute chills provided by the groom project and they define it as collection of binary tools you have two main tools yes which is the assembler it turns assembly code written in text format into binary code and you have the linker LD which takes a bunch of object files and combines them into another object file or a share library or an executable so that dudes are obviously the two most important tools from from from the minute chills and it also provides a number of other like debugging analysis or other kind of tools to analyze binaries so if you have never used like 3d alive Optim stuff like that it's very very useful tools for stripping binaries and so on and so forth but those are not that important for the completion process by itself LG and ASR by far the most important ones and the bending tools it needs to be configured for each CPU architecture so if you have a native x86 pintails on your on your system which you probably have as part of the linux distro it cannot by itself build or manipulate arm binaries so it can be configured to you can configure benches to support multiple CPU architectures but by default it's generally built with just targeting one CPU architecture so that's why it's not as simple as using your native billet tails and passing some arguments as a build some arm stuff or build some purposes stuff generally need to build a separate pin entails building pin details is pretty easy is just other tools base so it is configure script to run it the only thing you need to say is what you're targeting so here we find again the build host targets well parameters I've left out buildin host because they just let auto con guess what my build machine is and my host machine is because I want to build on my laptop and run on my laptop it's the only thing I want to override is for which architecture I'm going to regenerating binaries and in that case I'm going to generate for our Blahniks new api hf system another thing that you need to pass is the SIS route it's a directory that contains stuff for now I leave it to that because I'm going to talk more about this this route a little bit later the next component is obviously GCC it's by far the one that is kind of your entry point into into the tool chain and that the one you manipulate directly so it's the GNU Compiler collection project from Frank knew it has front end for many source languages C C++ Java Fortran and others I just read that they think got rid of the Java support in in recent times it has backends for mini CPU architectures so it essentially converts this sort the source front ends they converts the source code into an intermediate representation it applies a huge number of optimization passes on these intermediate representations and then the back ends they take this intermediate representation and convert it into assembly code for a specific CPU architecture so it does not generate by itself binary code it only take generates text assembly code and a s needs to do the rest of the of the work it provides different things it provides compiles themselves so if you see one for c c c1 plus for c++ and it has others for if you support other languages but those are not the ones you normally use directly because GCC is in fact the second item in the list here in fact provides what they call compiler drivers so kind of a wrapper program that will orchestrates the different steps of the build so when you run GCC or when you want g plus plus behind the doors what it's doing is that it's calling the actual compiler itself which you see one or one plus it will be calling the assembler it will be calling the linker for you and doing all the work for you so GCC and g plus paths are not it compiles themselves that just drive the compilation process and then you should of course always use them another thing that GCC provides and that some people are not necessarily aware of is that it provides runtime libraries as well so it is the compiler itself but it's also provides a bunch of libraries that you need on your targets to be able to run these programs and there are different part of the justice event I'm Lib GCC itself Lib STDs the plus class which is the standard C++ library and other runtime for other languages so like Fortran has its own runtime and other languages may have their own runtime as well and the last thing that Jesus provides is the the set of headers for the standard C++ library so Jesus is providing not only the compiler but also runtime libraries for various languages building GCC is a bit more involved than building building tales it's not like a single configure line and then off you go there's a two step process that we're going to cover a little bit later to build GCC in a cross compilation situation the next component in any compilation tool chain are the Cana headers so the one ultimately to build is a C library but our C library wants to communicate with our Linux kernel in order to communicate with our Linux kernel need to know the system call numbers the various data structures are use definitions and things like that that actually come from the keynote and instead of duplicating to the information in the C library they're kind of shared and extracted from from the Kaenel so in the kennel source code they are there's a split in the headers between user space visible headers and internal Kenya headers and this plate has been done like a few years ago I think just to be kind of a mess and nowadays it's it's much better organized and in the kernel sources so there are you API directories in include Linux and other places in the kennel where you API stands for user space API so those are the errors that are visible to user space and other key nodes are not exposed to user space there is a special target indicator build system called headers installed that will take all those you API errors from that the generic ones plus the one for your specific architectures and bunch of a bunch of other things and that will be messaging them a little bit there are a few things that need to be to be adjusted in them and then install them into a location so it's just installing a bunch of header files that describe the kennel system calls kennel data structures a number of definitions and so on as of Linux four point eight installed about 700 header files so it's quickly a bunch of interface installed here one one question that comes up pretty often when people are building or using existing tool chains is I'm running a given kennel version of my system like let's say Linux 4.7 and I have a tool chain that uses kaena headers that was built lion-o two years ago it's using chiquinho headers 3.10 is that a problem or the output is I have a system that's running 3.10 and I have a very modern tool chain that's using headers 4.4 is that a problem so the keynote to user space ABI is backward compatible so the keynote developers are normally never removing system calls or changing the semantics of existing system calls they are only extending this API with potentially more flags more system calls more features but they are not changing what already exists so the consequence of that is as long as the the headers that have been used to produce your cross compilation tool chain are at least as old as the keynote you're actually running in the target you're fine if you're using a tool chain that has adder that are more recent and McCain on you actually run then your C library will think that your canal will support system called a B or C that is very new but your kingdom will not actually provide them and that will not work so when you're choosing a tool chain or building your children just need to make sure that you have headers that are at least as all as the keynote that you intend to run on your system you don't need to have the exact same version so back to the the example my dad if you run for point seven and that's Heenan you actually build and run on your target but the ocean is built with King a header 3.10 that's just fine it will work fine right so you don't need to have an exact match there you want to know the the version of the can liners that are in use in neutral chain link schema as a file called the neck slashed version at age and it contains this next version code definition that is a bit cryptic to to read but it actually contains the three three digits of the decay note version like in that case it 3.13 zero so it's pretty cryptic to get from from one nine nine three six but you get the magic formula just just below so it's not that big to infer that the next component is the seed library so it provides the implementation of what the posix standard and very often a number of extensions that are either link specific or sometimes even see library specific they are for the one that we really care about our based on the link system calls and there are several implementations available gilepsy you sleep CNG muscle Bionic which is used for Android systems and a few other more special purpose so I already covered new leap which is many parameters this one is kind of an exception it's not based on Linux system calls and that Lipsy or Caleb see that are really like tiny very very tiny see library that cannot be really be used to build I would say full-featured user space applications but more for a very minimal Linux systems so I'm mainly going to cover a julep see you slip see and Messel as the the three most important C libraries that can be used to build a full-featured Linux user space so before we get into the just about deuce to three C libraries what they provide they provide each of them provide the dynamic linker so the nine dynamic linker is the program that actually gets triggered when you run a dynamically linked application it's not actually your application that starts up parts first it's the dynamic linker which is responsible for mapping in memory their shared libraries that your application is using of course that's only for dynamically linked application if you have a statically linked application no dynamic linker is involved that's provided by the C library it's the C library of course provided well the C library itself Lib C dot s oh and sometimes a bunch of companion libraries leap M Liberty Limpy thread and so on providing other parts of the the POSIX standard functions and obviously provides the standard c library headers stdio.h and string.h is still H and so on and so forth probably well aware of the the POSIX interface so the freaking C libraries that are I would say widely used today in on on UNIX systems are first the juleps that's the de facto standard every Linux desktop or server machine is using gilepsy provide by the GNU project started in saying in the 80s that's the full-featured C library supports many architectures many operating systems for the folks doing embedded gilepsy as no support for no mmm your architecture so if you're doing arm cortex-m or if you're doing it's a black fin or micro blaze no MMU or juice kind of no MMU architectures that Gypsy is just not a possible choice there's no official support for static linking it kind of works in some specific situations but the julep see developers explicitly say that it's not supported so dynamic linking is kind of the the only supported way it does provide a API backward compatibility so what this mean is that if you build and link a prop of course dynamic dynamically a program against gilepsy you can hear from know a great gilepsy to a newer version of gilepsy without linking your application it will continue to work so that's kind of April obviously for binary distributions because they can upgrade the C library without necessarily rebuilding the entire world that sits on top of it there's almost no configurability so it's just you build slab this gilepsy you have all of it or none of it and it used to be too big for embedded so that's why some of the alternatives that we're going to talk about exists but whether it's still the case really depends on your situation even arm system have gotten bigger and bigger and bigger and have more flash more RAM and so on so what used to be relevant alternatives ten years ago may not necessarily be very useful anymore so lots of people are using gilepsy an embedded system is that that's perfectly fine it's license and there LGPL be 2.1 or later which is kind of an important criteria there are other licensing options for other C libraries the other one and that is also usually quite well known is usually PC or it's now called use of the PC engine it's been served in and year 2000 and the goal was to provide a smaller C library at the time embedded systems add like 8 Meg's of Flash 16 Meg's of Flash so the size really matter and as I what is many embedded system have much much higher capacity so it's not nearly as useful as it used to be but we still people still work on systems that have such low amount of flash and in that case you slip C or some other possibilities or aren't still interesting to have it provides a high level high level way of configuring the decision library so compared to julep C that's just all or nothing and useful tips if you can select what features you want you want network support you want ipv6 support do you want thread support do you want locally support and so on and so forth so you can fine tune the C library to whatever you need it supports many architectures including some necessary support in gilepsy doing a YouTube support is often easier than doing a loop support so some of the people doing new CPU architectures they very often start by a UC loop support is that yeah that's easy and is your target compared to the lips I'll however it only targets linux as an operating system so it's kind of fun a narrow focus they don't provide any ABI backward compatibility so if you upgrade you slips into a newer version you may need to rebuild all the libraries and applications that rely on it which for most embedded system can be fine but necessarily for binary distributions it supports a number of no Mme architectures are no MMU Blackfin and others that I already mentioned so that's one feature that G loop C does not have at all some people think that it's related to you so Linux in some way because of the name but it's not anymore related to useful Linux and use linux itself doesn't really exist anymore so you slip C is just yet another C library that you can use on top of the Linux kernel it's not like gilepsy is for linux and new celeb c is for useful next that's completely wrong and you slip C is used on top of the regular Linux kernel on arm x86 MIPS system with MMU support so it's completely unrelated so don't be confused by the similar naming there's explicit support for static linking it's one of the one of the original goal of noose ellipses to support static linking and produce relatively small binaries even when doing static linking so the original loose ellipses project is kind of dead there has been no since May 2012 and but since I would say last year or Milly Erne hell for something like that I know if Valdemar is in the room but they're in the room in the room but is here at the conference or Valdemar broad broad corp I'm pronouncing it correctly to covered the project and forked it and they're useful Ipsy energy and since then he has been doing I think 18 or 19 releases over a year or you know half so the project is active again I think he has merge and new orchid and you CPU architectures dropped some old that were obsolete and not working so the project lindo kicking again and it's that that's very nice because it used to be a real pain to work with a dead dead upstream project nowadays it's really working again in slice ins LGPL with 2.1 pretty much like Gallucci except it doesn't have the old later statement the last one and that is interesting is muscle so it's starting much more recently only 2011 and one key difference is that it's MIT license so some people are getting interested in the project just because of its licensing it is under very active development the mailing list is very very busy there's support for a fairly large range of CPU architectures already and more is being added release after release they recently added no MMU support for super h2 specifically for the jacor open source processor there was a talk at last Yassin in the US about this process or so you want to check out the videos and it was an interesting one there is no configurability but it's still small even currently smaller than the new celeb scene and especially for statically linked scenarios it's it's pretty pretty good they try to conform to stars in a very very strict way sometimes even stricter than m'angil obscene news live see which can cause a few building shoes with with user space libraries and applications that you that you may have and they have a very very nice page comparing the three libraries in much more details than I can do in such a talk so there's a very nice table with list of features and we they're gilepsy supports it usually supports it missile supports it so it's a very good way of faking of comparing that the different options here in terms of size I did a quick tool chain build with with build route using gilepsy you slip sea and muscle so I was targeting an arm term to Architecture and I tested seduced three C libraries so you can see that the C library itself and all its companion libraries and the dynamic linker in the case of gilepsy weights about 1.7 Meg's and you'll see Lipsey and methyl or more around alpha Meg for the fool the fruit thing so when I say muscle as not available not available it's not exactly not available it's not applicable muscle has a principle that's everything is inside Li si but I so even the dynamic linker is lips it up SSO itself so there's just a single fine that you need to install and it contains all the all the functions so it's not that like thread is not available it's not at all it's directly in lip see that ISO itself you select see a migrated to this model as well in latest release one point 0.18 which wasn't used at the time I did the measurement so those are based on one point zero point 17 which is just a release before that but this ellipse is moving to the same the same model and just to give a rough idea so if you have like 256 megabytes of Flash of course size the size doesn't matter at all and you could just as well use gypsy that's fine if you have 16 Meg's of flash meg Meg point to of saving starts to be a little bit of an interesting back to other components of the tool chain that you need and there are a number of mathematic libraries that are needed to build just assume it's pretty arcane and weird stuff but it's anyway needed for to build GCC so he needs to three libraries MPF are GMP and MPC so jumpy is just the dependency of amplify FR MP frm npc are used by GCC to make calculations on floating point numbers and complex numbers at Build time so if your program or library is having as a number of constants floating point constants or complex Constance with mathematic operations Jesus is evaluating them at build time and to do that it needs arbitrary precision libraries to help doing those calculations so that's the as my parents tending the only reason why they are you so it's kind of a very specific usage of those libraries but they are nonetheless absolutely needed to build to build GCC so they are needed on the host machine they are only needed to run GCC itself so you need to have them on your host machine but they are definitely not needed on the target so the overall process to build the chun qiang is not that complicated a lot of people thinking it's it's like something completely crazy it's not that crazy you just need to build mini tails so it's just a one configure make make install kind of thing with not that many argument to the configure script then you need to build dependencies of GCC MPF are jumpy and MPC and dudes are just built for the last so it's native builds configure script configure make make yourself for each of them pretty easy then you install the Kenya headers and you've seen previously it's just one single comment to say a please install the Kanan headers for this architecture and that's that that's all you need to do that's also pretty easy and then you can start the the biggest part of the the tool chain beyond you need to build a first stage GCC and its first stage is the same we build the GCC that has no C library support that has only support for starting linking and is only support for the C language and it's kind of a limited just to see that we put kind of on the side and we use this first stage just you seem to build the C library obviously the C library needs to be built for the target so we need to have a cross compiler to build it so we use this first stage GCC and once we have the C library produced it's dynamic linker and everything else then we can build the final GCC which needs the celebrity to already be built because obviously GCC is going to generate binaries those binaries encode the path to the dynamic linker they it encodes the shared library names for the C library for the please read libraries and so on and so forth so it really needs to know which the library is being used so the C library unlike all other libraries that you might be later on for crypto or network or graphics or whatever is really an integral part of the tool chain it's not something that's separate from the tool chain so if you only change the C library you need to recreate a new tool chain from these steps the first three its these steps were kind of the second shot but there's no interdependency between the first three steps you can build the new tools gem pfr gmpm please install the k-9 others in whichever order you want you can even do that in parallel if you want there is just that belittles the GCC dependencies are are needed to start building GCC the first stages you see and the kennel headers are only needed at the time you start building the c library it's just to kind of make it clear and what the dependencies are and maybe this I don't know if it's really readable makes it a little bit even clearer so that's the dependency chain of the packages in Bill route that produce a cross compilation tool chain based on gilepsy so it's a dependency chain so the build order is the reverse of the one you can see here so the build actually start from there and goes up in this direction so to build AHS GCC initial we need ospa materials we need Austin PC and and so on and so forth to build gilepsy windows deucesy initial and the King headers and to build all just as you find all we need ellipse e to be built but that's pretty much all what you need to build a pretty simple and working tool chain and with that we built a large embedded Linux system you can build a QTE x.org whatever big big user space programs and there's nothing else that induce four main components so if we dive a little bit more and I'm not sure how far I am in the time it's fine another interesting part to discuss when talking about tool chain is the concept of this route so remember when I mentioned the build process for miniatures there as there was this - - with this route configure arguments what is this this route thing and this this route is the logical route directory for headers and libraries that's kind of the official definition from I think the GCC documentation or whatever or maybe been a chance so this is where Jesus automatically looks for headers so when you're doing sharp include stdio.h where is this is he looking into in what we call the SIS and this is where LG looks for libraries so when you do - small L foo LD is going to look for Lib who in the sea route so that's where we want to put all the headers and libraries that are going to be used by your compiler bus GCC and Bill new tools when you create a sister tool chain are built with - - this route equal some location which obviously is is the same for the from the tool and what we do is that we install the kena headers the C library and it's headers all in the sis route so that your compiler automatically finds them at the right location this is route is an absolute path on your build machine so the question is what happens if you move your tool chain around like you move it into to a separate machine or in different location in your system well GCC is an LDR a kind of smart enough that if the this route is a subdirectory of the prefix then it will automatically be capable of recalculating the relative location of the the tool chain and it's an assist route and be capable of finding the headers and libraries even if the tool chain was moved around however if the prefix and the sis Road are completely separate in different places it won't be able to find them again right so most of the people who build to change that need to be relocatable that you can move around in your system or install in other machine in other locations have this this route as except directory of the prefix you can override it at runtime using the - - this route GCC options and so for example in the route we leverage it because we create an our own sister oude where we install all the libraries that you build for your embedded Linux system so we override the destroyed that's use at using this Ren time option and you can at any time print the currency's route using the - spring - this route this is the option most tool changes have just once this route so they have one copy of the C library one copy of the K no headers one copy of the C library headers and that's just fine but those libraries so the runtime libraries for GCC and the scenery they are built for the target right the dudes are meant to be installed on your targets they are contained arm code or MIPS code or PC code or whatever Sippi architecture targeting so at the time you build your tool chain you are deciding what optimization level and what exact CPU architecture bar in your targeting are you targeting our v6 v7 r m v5 with the floating-point without putting point with this ABI of that ABI you're the designing net at the time you'll be a little jewel chain because the tool chain is not just the cross compiler but it's also a set of runtime libraries that are built for your target so if you have a tool chain that is like contains I don't know an arm v7 build c library and your target is arm v5 know like you will put the c library on your target and it's not going to work so what you can do is of course build different tool chains for the different cases you have but that was not to the taste of some tool chain providers so they came up with mechanism called mutually pull chains to the idea of mutual obtains is that you have a single compiler but you are multiple sis root and the difference these routes have been built with different CPU optimization flags and a target different platforms so let's take a very specific example Mentor Graphics is providing some number of tool chains infinitely less than they did in the past but one of them is for arm and it's a multi leap capable tuition that provides three sis routes so if you do just to see - brain - multi Lib it gives you a list of three three entries so there's the dot entry in a the default one and arm v14 and some - and then next to that you see the GCC flying so a march army 40mm and arch arm d7 a so this tool chain provides three C's routes one for MV for F which is the default one one for M before and one for mv7 Sammy - and the compiler is going to automatically use and one or the others this route depending on the GCC compiler flags that you have passed so if you just say just to see arm v5t princess route it's going to show you the main sit which is in this weird location if you pass arm before te prince this route it gives you a different district location in which there is a different variant of this library that has been built for arm before and then if you pad other GCC arguments like our b7 am/fm it's going to give you a different suit and if we look at the different libraries in the sea route we have here I just dumped with Rydia life the architecture for which the dynamic linker has been built so LTS o is the dynamic linker we can see that each of them has been built for a different CPU architecture what's that it's the same it's arm but it's different variants of the CPU architecture so that's what mutually tool chains are just multiple cysts routes built inside the same tool chain to provide a little bit more flexibility to continue this talk what I wanted to do is kind of look at what's inside the tool chain so I took a pill grid build tool chain did find slash we will not find slash flying dot and look at all the files that we have in there and try to come up with some reasonable explanation and on what all you do spines are it wasn't that easy for all of them my time I tried this exercise so at the top level of a cross compilation tool chain generated by peeled root you have something like that so you have one directory named after the torch and top row and then you have the usual like being include leap and other directories so if we look at the first directory that's named after the topo we have a first sub directory called bin in which we have the a small set of gluteals programs that have been installed in there without their cross compilation prefix or they are just named a s LD NM and read EF and so on so it's not a full site but just a subset of them and those are actually are links to the actual penetrance programs which are in the in the bin folder that we will see later in the bin folder they have a prefix like in that case our building you slip seek new ABI HF yes for the assembler but they don't have such prefixes in this bin directory and this is where justice he finds them so as I said earlier just to see is a compiler driver so it's orchestrating the build and when it needs to call LD when it needs to call a s is not calling like your arm do papa papa blah yes or probable LD it's actually calling juice be neutrals without the the prefix another thing that you find in there is the set of headers for the C++ standard library that were installed inside installed by GCC for some reason they are not part of this issue itself I'm not not sure why it puts them elsewhere but this folder is anyway part of the default header search paths for GCC so it's fine finds them naturally and then in Lib we have that's the location where GCC installs its runtime libraries so they are built from the target so we have lib atomic which provides a number of kind of software implementation for atomic operations when they cannot be implemented directly by using CPU instructions Lib GCC which is the main GCC runtime it has a bunch of optimized functions like it does also have like 64-bit division for 32 bits architecture floating point emulation and batch of other things that Jesus doesn't provide on its own but are needed on the targets to be able to run your programs transaction memory library it has the standard C++ library and a subset of libous to dc+ paths with only the language support but if you really do say pass pass application it's Lib as to this purpose that will be used and then we have the sis route itself which is really what I was referring to earlier and this is where the C library gets installed and kino had errors and C library others and so on then we have the bin folder that's the one most people look into first because that's where you find your cross compilation programs binutils and GCC and also wrappers for lgo but as i said earlier i'm not going to cover it all to you in this talk in the include folders that's where we find the headers of the host libraries but are in fact not needed at all to run the tool chain is just an artifact of the we've built those libraries so they've installed their headers but know that justice has been built they are kind of not really needed anymore in Lib we find so last point the host version of G MP MP affair and MPC which are needed by GCC builds for the host machine and then we also find a bunch of other things installed by by decision so the CRT you begin CRT n files which and all constructors destructors and that dues are linked into your executable so that's again one of the work that GCC as the compiler driver does is it calls linker with a bunch of arguments to make sure that your program will will work with all these small object files that are needed to make it work on the target it provides a number of headers provided by the compiler so in fact you'll not only have Enders provided by the kennel and by the C library but also a few others provided by the compiler itself fix includes that so this is the process to fix up header files that come from from the system it's kind of weird stuff and in there it also puts the static variants of the GCC runtime libraries another thing that GCC installs are LD scripts so depending on your architecture it has a huge bunch of linker script which are used to link applications or lingshan libraries and then depending on the flags that you pass to GCC the different set of linker scripts might be used so it just tells the linker out to lay out the final binary that that will be produced I would you combine the sections together and and things like that if we move on to the next part in leap exact that's where we have actual compiler that's where we have 61 c1 plus so the actual compilers and then there's a program called Co like to which is what GCC calls to actually do the linking so it's another wrapper around LG's with doing some stuff and then calling LD and there's a bunch of other things related to a CO and want to get into the details here and finally documentation translation file that's a little less interesting since time is flying gonna speed up a little bit so architecture tuning so GCC provides the number of your time options to tune for a specific arch or simply variant so with arch with CPU with ABI with FPU so what they do is that they define the default value for M arch MCP you ma bi MF pu and and so on so with to specify those that configure time then when you call justice in it will by default produce binaries or libraries optimized for this specific CPU variant they can always be overridden at runtime using the corresponding M options but again as I said earlier beware that parts of the torsion have already been built for the target the C library and the GCC runtime libraries they have already been built as part of the church angel process so you need to make sure that they are they have been built with superior architecture tuning that matches your actual target one thing that's interesting is look at the GCC documentation they have a very long machine dependent options where you can see the on a per Architecture basis what are the accepted options and their values so for arm you can see for M arch what are all the possible values from the different CPU variants that GCC supports another thing that's important to know when discussing tool chains is the story of ABI so API from the point of view of the tool chain it defines the calling convention so calling convention is essentially our function calls are made how will you pass arguments out the the if they are pressed on the stack through registers our return value gets returned the size of basic data types alignment of members and structures and when there's an operating system our system codes are made and from a for a given CPU architecture it provides registers and provide superior instructions they are potentially an infinite number of API is that you can imagine and so it's not necessarily that one CPU architecture has one API you may have multiple aid lines that have been invented or created for that severe architecture and if you have to object files that have been built for different appliance you cannot link them together because of course if they disagree on what the calling conventions are like one thing that to pass the register you do it all on the stack and the other one thinks that the register are the arguments are passed in registers obviously LinkedIn linking them together will not yield something that works very well so when you link the objects mines together they need to use the same API so here is an example an arm which is probably one of the architectures where this is I guess the most confusing there's a history of three a blind on the our bluemix world Oh API which is now completely obsolete and it's not really necessary to talk about it anymore it's not even supported in the in GCC anymore and it's been removed for for a long time I still wanted to mention it to seal it to show that there is a history of a blinds so nowadays we have two API a news on arm e API and ABI HF and the API is an ABI and it allows to mix heart float code so code that uses floating-point instructions with soft flog code so code that emulates the floating-point operations so that it can work on processors that don't have any floating-point hardware and to achieve that and what it does is that it's passing the floating-point arguments into integer registers this way it doesn't assume that there is a floating-point unit in your hardware but it can still use it if there's one but this passing of floating-point arguments in integer register with call encoding kind of a performance slow down on on platforms where you really have always the floating-point unit so people came up with a kind of variation of this API called EAP IHF so the kind of the name suggests it's hard float where do these floating-point arguments are passed in floating-point registers so it requires the use of an arm CPU that as a floating-point unit and it's corresponding registers and of course you can't link behind code with ABI hf code together so if you have a prebuilt binary provided by your vendor which is e ABI then you can only using in EA bi applications with EAP on a breeze another thing I wanted to mention is difference between tool chain and SDK sometimes people are confused buying that tool change the strict sense of it is just the four components that I mentioned the compiler Bennett LZ library and the Cayman headers and that's just it and a number of tools especially build systems they allow you to and create SDKs so SDKs are a tool chain augmented with a number of libraries that match libraries you have available on your target so it's a compiler as your library pin entails Plus maybe networking libraries crypto library graphics libraries and so on and their headers that are available on your target and such SDK allow you to build a user space application that link against those libraries and then of course allow you to run them on your on your target so an SDK is really more than a tool chain and how can you get a cross compilation tool chain you can they get them pre-built from your distro so we're going to Debian and most of the Linux distros these days have pre-built tool chains for various architectures there are also organizations that provide tool chains Solon Aero provides arm and Archie Archduke's for tool chain Mentor Graphics as a few but as I said less than they did in the past imagination which owns the mips architecture provides MIPS tool chains there are other vendors that we can provide them but you can also build it yourself you could do it manually as we've seen it's not that complicated but there are still a few coach has here and there but there are some interesting tools that will help you doing that one of them is cross tool engine which is really specialized in building a cross compilation tool chain so there's a mini config like interface which allows you to say which seed library you wanna use which is this version which can either version which means just version what is the superior kwhitaker Brian you want to tune for and other things like that and then you can just say go and it builds the whole thing for you and produces a nice tool chain that is relocatable and that you can share with other people so it's a very nice tool it's probably the most configurable and versatile option in terms of building tool chains it can produce tool chains that target Windows systems and really other crazy situations but also build systems building embedded Linux systems usually know how to do that as well so if you take Yocto or rope and embeds block root open wrt and many use other tools they know by themselves how to build the tool chain as part of the the overall build process for a pro Linux system and pretty often they can also reuse existing tool chains so if your vendor provides a tool chain pre-built you can usually used use that as an input work for those build systems a few references cross to LNG has a really great documentation on our tool chain is constricted so even if you don't intend to use cross tool engine I recommend looking at that documentation for information and the decision Vinicius documentation are actually interesting it's quite maybe crazy to say it but it's it really contains a lot of details on the GCC runtime and the different options that you can use and and it gives lots of insights on on our the tool chain tool chains are working any questions yes please listen using July or gilepsy yep yes that's that seems kind of weird but then so and so as I said gypsy provides backward compatibility for this it's API so that you can replace gilepsy without recompiling your programs and one of the way does that is by using version symbols so it is the symbol with ads and then the version of the DC library without versioning and then in that case it might break existing application that rely on this version symbols so it's probably a tool chain that has been built without gypsy symbol version yeah so the Canadian build is what you need when you have a build machine that is different from the host machine that is different from the target machine right so I only covered the case we're building a host are the same so it's like you're building on your laptop a tool chain that you want to run on your laptop to produce code for different spirit take your different operating system the Canadian build is what you will need if you want to build on your laptop a tool chain that will for example run on Windows and generate binaries for our Mullenix that's a Canadian build that you need and then more steps are needed to do that because of course for your to allow your laptop to build assist some binaries that run on Windows to build the compiler itself you already need the cross compiler right because you need to allow your laptop to build binaries for for Windows so you're first need to be the cross compiler which you use to build the compiler that will run on that Windows machine and produce binaries for our mornings the next sense yes you still needs to be lips your arm so you also need so you need when one cross-compiler to build the compiler itself and indeed one cross-compiler to build the lips here on forearm and then you can go ahead and build the process you need three tool chain builds I'm not sure why Yocto does it so complicated I'm not sure that's the reason I think the I think the reason why the octo might be more complicated in the processed animation is that made I don't know if they're doing it but they might be bootstrapping GCC which is different from doing a can engine build is that you the problem with the process I highlighted here is that I'm building just a scene with the GCC that is on your machine so if you have a different GCC than the one I have then the tool chain that we produce is different so you can argue that the binaries that do solution will produce may be affected by how the compiler itself was built so if you want to avoid that you need to do a bootstrapping process where you first built with your compiler build the compiler and using that compiler which is the same for you and me we build the cross compiler so maybe they're doing that I'm not but I'm not sure I've never twelve into the octo tool chain logic to build to change for us it takes about 15-20 minutes to build a tool chain something like that even less on on high speed high fast machines yes so as I mentioned if you build an arm tool chain the compiler itself it's not specific to a given architecture variant so your compiler will be able to produce binaries for whichever arm CPU variant you want you can say please build for context aah pill bees for context in 9a 15 arm 9 to 6 and so on and so forth no problem that's the compiler part of it but as part of the tool chain a number of target libraries I've been built the GCC runtime the C library the dynamic linker and all sort of things and so you have to verify that those I've been built as part of the tool chain build process with the CPU variant that matches your target platform or at least is compatible with your target platform does that make sense so yes the compiler can target any CPI variant but the the libraries that are built into the the Trojan as well are a bit for a specific version that you need to make sure matches your your platform yes so you can be not not just like looking at it but you can do where's that most lip stuff yeah yeah if you do GCC - B it's gonna show you the Wiis how it was configure so you can see the - - with arch - - with CPU and so on which it was used to build GCC and most likely those were the tuning that was used to build the C library but you want to be really sure you can use Vivienne F dash a and it's gonna tell you what is the the architecture of RN for which the the C library was produced so it gives you a very sure information that's okay this it has been dead for mv5 or unfit for mv7 with such-and-such optimization level right well thanks a lot for attending I'll be around