This lecture covers the mathematical foundations for visual navigation in flying robots, including vector operations (scaling, addition, dot product, cross product), matrix operations (multiplication, inversion), and coordinate transformations in 2D and 3D space. Key topics include homogeneous coordinates for representing points and lines, Euclidean transformations (translation, rotation), and various rotation representations (rotation matrices, Euler angles, axis-angle, unit quaternions). The lecture also introduces camera models (pinhole, lens), sensor types (IMU, GPS, range sensors), and their applications in robot localization and navigation.
Visual Navigation for Flying Robots: 3D Geometry and Sensors
Added:good then welcome everybody to the second session of visual navigation for flying robots um yeah today we will talk about 3D geometry um which will take most part of the lecture and a little bit on on zensors that are usable for uh flying robots um before we start um there was some confusion last week about the the lab course um so the the um oops sorry and I see that there is a small mistake on the slide the lecture is always here rips leure uh which is here but the the robot lab uh is a few rooms further further down the corridor so in um 0514 um uh we um started or we we have a list for for students to sign up for the lab course and the idea is that you join up with um with your um uh uh colleagues in in teams of three two to three students and um the thing is that we have several people on the waiting list who are interested in joining and um but of course we give priority to the people who signed up first um but the thing is that you have to sign up for a team before before um uh May 2nd um because if you don't then we will give these free places to people who are still on the waiting list um I I I guess that for for all of you who's on the waiting list from people here at the moment to I guess chances are really good to get in so just continue and do the exercises as if you were uh already signed up um good and then the first exercise sheet uh comes out today so Christian will upload it to the to the website later on and it's due next Tuesday before the lecture and the idea you have to send it in by email uh but it's also written on the the sheet good um so today as I said we will look at linear algebra first to give you a short repetition it's uh in principle um it should be easy for you to understand if you so so I just repeat some Concepts like vectors and matrices and all kinds of stuff um and in principle you can just lean back and listen to it and get used to the notation that I will use throughout the course um if you discover something that you have never seen before uh then it's really important that you that you uh study these things again um a good reference for for both of these things is the computer vision book from Richard siski and it's completely online as a PDF so if you get lost somewhere then um have a look at that uh and then I'll briefly talk about the sensors that that the AR drone uses but that are also suitable for other um quadcopters and I'll briefly introduce the exercises from the first sheet good so I guess everybody knows what a vector is um it's um essentially a stack of of numbers um and uh you can interpret it as a as a point in an nend dimensional space so in 2D um a vector for example would be this arrow and it has two components X1 and X2 um you can do several things with vectors like for example uh scaling them uh by um Scala value uh you can also add them um or subtract them I guess that should already be uh uh that that you know that already from school um you can of course also compute the length uh of a vector uh usually we are interested in the oian distance or the two Norm uh which means that you square all the elements and then some the up and take the square root um uh there is also the notion of a normalized vector um which um yeah has length one so you divide a vector by by its length um there is also the dot product um which um in in principle um is the uh the multi um the product of the length of both vectors when the vectors are aligned um uh or it is zero if the vectors are orthogonal and that's a good check to see whether two vectors are um um uh yeah parallel or not or orthogonal uh there is also the notion of linear dependency or linear Independence Vector is said to be dependent um uh from on another on another set of vectors if you can express the vector as a linear combination of these of these vectors um yeah then there is also the cross product that we will use frequently um in several um uh in several algorithms it essentially what it does is uh it um computes an orthogonal Vector to two vectors so imagine you have two vectors in 3D space um pointing in some directions and both vectors form a plane and the cross product product gives you the normal of this plane so this is really useful um for a lot of computations in Geometry 3D geometry um good uh there are there is this definition of the cross product that most of you I guess have seen before um there is also a handy um uh notation that turns a vector into a uh into a matrix um and um it's essentially doing the same thing so this computation is exactly the same as if you would turn a vector into a matrix and then multiply it uh with a vector um it's but it's more convenient as as we will see later um uh yeah in a few weeks from now good then there are of course also matrices um in principle a rectangular array of of numbers uh so to speak um there is sometimes confusion about what the numbers here mean um of course the first element is always X11 uh but then usually people um denote with the first number the row and with the second number um the column and um yeah exactly that's it so so sometimes you see that people refer to individual elements and then they have this indexing scheme and uh you say that a matrix comes from a space n * m and then again the first n refers to the number of rows and the second to the number of columns good then from a matrix you can extract column vectors um uh and row vectors um sometimes useful to to have have individual vectors um in when you when you um interpret a matrix a um as a list of of column vectors there is a nice geometric interpretation um that uh uh says that you can interpret such a matrix as a um as a coordinate basis um yeah uh and um again you can take row vectors of a matrix uh and so on and then there are again several um things that you can do as a matrix uh or special forms of matrices uh like a square Matrix that has the same number of rows and columns a diagonal matrix that just has elements on the diagonal uh there is the concept of the upper and lower triangular Matrix um what does that mean again who knows is e Matrix what mat only the elements theer triang exactly it's all elements on the diagonal and above or all elements on the diagonal and Below um who knows what the symmetric Matrix is Matrix is the same exactly exactly um skew symmetric Matrix what does that mean so now things are getting more complicated from the CR what once again is the the matx that comes from the definition cross uh yes uh um the cross product Matrix is Q symmetric that's true but it's not the definition of a q symmetric Matrix look like multiplication transpose to Matrix uh no no no the cetric Matrix is a matrix that if you transpose it um then then you get the negative of the Matrix um yeah so it's it's similar to a symmetric Matrix except that the sign flips uh when you transpose it um good then U there is a semi positive uh definite Matrix or a definite positive positive definite Matrix who knows what that is values are are and posi are all netive mhm true true um and another um uh way to express that is that if you take any Vector x that you multiply with a matrix um wait there is a transpose somewhere here then uh then this will always be larger than zero or um uh larger or equal to zero so a positive definite Matrix multiplied by a vector on both sides is always a positive value uh good uh what is an invertible [Music] Matrix matx for which the determinant is not yes yeah yeah true um or a matrix that has full rank um and is square uh good what is an authon normal Matrix any ideas yes um it's a matrix where all vectors are rectangular and has have the length one what do you mean with rectangular um our um what's it called orthogonal yeah exactly this is where it comes from or orthogonal vectors uh no no no Vector is linearly dependent on on any other combination of vectors and they have they are normalized have length one good and then there is um The Matrix rank uh what does that say the number of independent column or row exactly very cool good so um good and then again we can do Sy with matrices like uh multiply them with a scalar value add and subtract them transpose we already had that um and then there are two things that we of course need a lot in in 3D geometry M to multiply a matrix as a vector and a matrix as a matrix and to invert them so let's have a look in more detail at what uh it means to multiply a matrix with a vector um as I said before you can interpret a matrix to um to to form uh the basis of a coordinate system and you can for example say just look at the red um coordinate system at the moment we have a point B in in this in this coordinate system um and by multiplying it now by this Matrix um X which is composed of two um uh column vectors uh we get this point B back in the blue coordinate system right so this is one geometric interpretation of course um all of you know how to multiply a matrix as a vector but this is how you can imagine what it actually does so you have coordinates in the red coordinate system you multiply it with the basis vectors X1 and X2 and then uh you get the point B back in the um in the uh blue coordinate system um good uh then there is also a matrix matrix multiplication um which just takes two matrices that have to match so The Columns of the first Matrix have to be the same number than the column uh than the rows in the second Matrix of course and uh you can interpret it again as a transformation of coordinate systems and you can use that to concatenate transforms so to to go back to this examples to this example you could imagine that again you transfer from the blue system to a green system that has even different um coordinate AIS and then you could take the um uh so so the first Matrix would then convert from from the red system to the blue and then you have a second Matrix that converts from blue to green for example and that's that's use useful because you very often have the problem that a robot observes something in its local coordinate frame or the sensor observes the camera observes something in its local coordinate frame and you need to know what it means in in Global World coordinates good uh then some things uh some properties that you that all of you should already know of course uh matrix multiplication is in general not commutative which means that you can't just flip matrices that will give give you a different transformation at the end um it's a associative which means that you can change the um the uh uh the order in which you um not the order but the Precedence or the the parenthesis um the transpose of the multiplication is um uh the uh to to flip uh ah you see you see it how it is good good and then uh we have Matrix inversion and that's also important um because it allows you to do the the inverse thing of the coordinate transform so um to uh for Matrix to be invertible it has to be square and of full Rank and um then uh the inverse Matrix of of a um has a property that if you multiply it with a then you get the identity again and um there are different ways to compute it and um which have different computational complexity um sometimes it's very easy to to to uh to invert a matrix depending on its on its particular form for example it's super easy to to invert a triangular Matrix um um it's also super easy to invert an orthonormal Matrix because then you can just take the transpose um but in a general case it can um can be costly especially when the Matrix is large um yeah and you know from school the gaus elimination is one typical way of um Computing it l decomposition and and so on um good so that was just a brief summary on linear algebra um um we had vectors matrices and oper operators on on these um two entities and now we need to apply that of course to 2D and and 3D geometry um so you you'll notice that most of the examples that I'll do in the lecture will be in 2D because it's easier to visualize but of course especially in the lab course everything will be or no not everything but um yeah even in the lab course we have most things in 2D actually uh but in principle the world is 3D so um keep that in mind that in principle yeah we need we need things in 3D good um as I said we can express a 2d point in the the world uh as a as a vector that comes from R2 um and um in many cases it makes sense to augment this vector by a one in the last column so now we have a three-dimensional Vector uh that still expresses a point in 2D um uh because uh such an augmented Vector is easier um to uh to handle because then we can express translations also as a linear um as a as a matrix multiplication as you will see in a second um and then there is this idea of a homogeneous of homogeneous coordinates which just means that any Vector um what else think I have this on the second slide any any Vector um or any two vectors that differ only by scale actually represent the same point in in 2D um so the the general idea is you have a homogeneous Vector with three consisting of three components and when you divide by the last component then you get here the 2D point back and as you will see this is very useful uh for computing uh things in um yeah for computing things um there is a special case when this W equals zero um then um uh the point gets or the the the smaller this W gets the further the point will move away from from the camera and when w is zero then the point is in principle at infinity and this means that um the point is um uh that we don't have a real point but that we have more or less a direction from the sensor and that can also be sometimes um useful um who uh may I ask who of you already knows homogeneous coordinates okay half and who who has seen homogeneous coordinates for the first time okay then it's good that we uh um look at this um there is also um uh the of course 2D lines um and uh I guess everybody of you knows a 2d line equation which just means that you have a point XY and for that you um uh require that uh uh a * x + B * y + C equals 0 and then you get a 2d line of course and the parameters of this line are these three coefficients a b c and you can just combine them um to a l um and um I put This Ti above the l oh by the way in terms of notation let me go back to this slide so when I talk about a normal Vector in 2D or in 3D I I will just use a bold symbol for that um when we say that we take the same vector and we augment it by a one I have the bar above it and uh when we talk about a 2d or a point in homogeneous coordinates there will always be a till above it and um the same notation also applies to to lines um because in principle you can of course scale these three coefficients as you want it doesn't change the plane equation you can multiply the whole multiply the whole thing by five and will be the same uh the same 2D line um good sometimes it makes sense to normalize the line equation Vector um such that the first two components uh have length one and uh what a geometrically means is that um the first two components are the normal on the line This n here and um and then the third component gives you the distance from the origin or the distance from yeah from the the origin and then you can in principle choose the origin to lie at the place of interest um another uh representation uh is uh to represent a line as Polar coordinates um because in principle the line of course only has two degrees of freedom right you have the an angle so to speak of the line and the distance from the origin and so you can also say uh that you have a Theta and a d that then expresses that and then the the normal uh can be computed from the cosine and S um of this angle and sometimes this is useful uh for example for a half transform for finding lines um or sometimes it's just important to to compute an angle from a line there are two handy formulas um that um uh yeah that it can be useful sometimes you have to compute a line from two points and then it can be arbitarily complex when you try to do it by hand and you don't know this formula so the the nice thing is you can just take the cross product between the two homogeneous points and you will get the line equation and um again sometimes it's important to intersect two lines and then you can of course use G elimination on coordinate systems and so on but even easier is to just just take the uh cross product between two homogeneous line uh coordinates and then you get again uh the intersection point so when you ever find yourself trying to solve geometric equations then maybe remember that there is a very simple formula for that uh good then uh the same thing generalizes exactly in the same way to 3D so we have again a 3D point from R3 we can augment it then we have a vector from R4 but it has only three degrees of freedom um and uh again we can have homogeneous coordinates in in 3D um where remember um two homogeneous vectors are the same if they only differ by scale and the way to get back from homogeneous coordinates to the 3D point is to divide by the last element of the of the homogeneous Vector yes question before if we apply these two equations does it need to be the last element on as the last last elements of all this homogeneous coordinance has to be once uh no no no no wait homogeneous coordinates don't have an one here they can have any number here yes if we apply this equation does it we have to divide them when you want to get back the 3D point and and you need that often for getting back back the intersection point because you want to draw it on the screen then you need normal coordinates and then you need to divide by the last element okay yeah but in the in the in the computation itself it's not required no no it it doesn't matter so the intersection point doesn't change if you multiply it by five any other questions so far good okay so this is supposed to be more or less known by most of you uh it's just to refresh everything and then um and then if there are questions please uh raise your hand um good um again in 3D we have uh 3D planes um which have uh a similar uh form as the 2D the lines in 2D uh so you have four coefficients in this case that multiplied by the augmented vector zero um um and um yeah again you can normalize the plane um sometime it's more more intuitive to have a normalized normal Vector um on the plane and then you also can easily access the distance from the origin which can be useful on the other hand sometimes you start with a point on the plane and the normal Vector that you know from somewhere and then you can also used that to construct um uh the the uh the plane equation good um lines in 3D is a bit more complicated than planes um there are different ways to um to to denote them there are pler coordinates and um I don't know what uh there you can also intersect two uh 3D planes which usually gives you a line in in um except for when the planes are parallel um but the most easiest um way in in my opinion to represent a 3D line is to take two points on the line uh and then to interpolate between the two so you have a point P Point Q uh and you have a Lambda that um um lets you move between these two points um when you're interested in the infinite line then this Lambda can take any value uh if you're just interested in the line segment uh then Lambda would have to be between zero and one uh of course um good then the next step now is uh to look at so now we have been looking at points uh now the question is how um yeah what what Transformations um do uh um or how can we transform these points um or how do we yeah transform between coordinate systems and the easiest thing of course is to have a translation pure translation that shifts your coordinate system by a certain um Direction in X and Y you can have a UK oian transformation which additionally does a rotation um there is also a similarity transform which does everything as the oion but additionally allows you to scale um uh the uh um the side or yeah to change the scale of the world or of your coordinates um an fine U transformation allows you to introduce this Q between the coordinate axis and and the most General cases the projective transformation don't worry we'll look at all of them in in more detail um yeah which um which you use for which you need for example to uh to model uh this um the projection of a camera but now in in more details translation just uh adds um a constant offset to a point so imagine you have a 2d point x and you add uh a 2d translation Vector uh then this is a 2d uh translation um and now we see why we need or why um homogeneous coordinates are um are handy um imagine that we have an augmented vector or a homogeneous Vector uh then we can just um multiply it by a matrix uh with the identity here two um 2x two identity Matrix and the translation vector and that um uh makes it easy because this this is now a multiplication instead of a um an addition and um yeah that makes computations easier uh the same way usually so so here we we change from an augmented vector or a homogeneous coordinates no an augmented Vector to a normal 3D Point uh sorry 2D Point um but usually it's it's um you just stay in homogeneous coordinates so you have a homogeneous or an augmented Vector that you multiply by such a translation Matrix which has the identity here in the upper left and the translation Vector here uh on these two elements and then uh zero and one um in in the in the uh third row um yeah who who sees this for the first time I guess and all the people who haven't okay can everybody who hasn't seen a translation Matrix before raise the hand it's okay good um yeah so one question how many degrees of freedom does this transformation have two okay correction two yes two because it's just these two three degrees of or three parameters of the translation Vector good then uh we look at the oian transformation which is also called the rigid body uh motion or rigid body transformation uh which consists of a rotation and a translation and um that means we have again oh this T should be should be bold uh we have a rotation of the of the point which means to that we change you know remember the coordinate transformation and uh a translation and again in Matrix notation this means that we have a rotation uh Matrix here 2x two and a translation Vector uh 2x1 um and you can apply this directly to an augmented or to an homogeneous point and in 2D uh the rotation Matrix uh looks simply like this you have it's composed of cosine and S if uh this Theta is um is is zero then the rotation Matrix is just the identity and if it's 90° for example then um yeah you get a zero Zer here and here A minus one and a plus one and um one important information about rotation matrices uh a rotation Matrix is always orthonormal it's not supposed to change the scale of the world it just rotates coordinates which means that the length of the vector has to remain the same um and um yeah uh because it's also normal um the inverse of the Matrix is just a the transpose so a rotation Matrix multiplied by the rotation Matrix transpose is the identity and also the other way around so rotation Matrix transpose times rotation Matrix is the identity and um important property of a oian transformation is that the distances and the angles um are preserved SED I mean it just moves and rotates um something in 2D good then we have the similarity transform which adds a scaling scaling factor to this rotation Matrix um this um of course changes the distances equally um uh but it still preserves the angles between lines um yeah so it doesn't change the shape but it changes the size um yeah there is the fine transform uh which just means that you have really six degrees of freedom here uh in this Matrix um it um changes all kinds of things but parallel lines remain parallel after a fine transform uh and then the last thing and most General transformation is the projective transformation uh which allows you essentially to to change all values in this transformation matrix um I should say that uh uh this uh uh projective transformation um or this this projection Matrix here is um uh homogeneous because when we because we multiply it uh by an homogeneous Vector the scale doesn't matter so if you take all values by a factor of two um it will still encode the same projection uh and this means that the uh projective transformation has exactly 8 degrees of freedom and um yeah as you can see you multiply a homogeneous Vector with this protection Matrix yes uh yeah so it has it has nine variables but um uh as I said if if you multiply this Matrix say I I add here a factor of two yeah then so we multiply a homogeneous Vector with this and then by two uh and then we get out a homogeneous Vector again so um uh this this Factor here doesn't really matter um that's not the best way to explain it um I mean take two different projective Transformations um that have different scale um and apply them to to homogeneous vectors that you want to transform then both of them will will lead to the same points at the end because the scale by itself uh doesn't change the the geometry homogeneous vectors or the scale of a homogeneous Vector doesn't matter and this is why uh we have nine free variables here but uh altogether they only have eight degrees of freedom um yeah and the only thing that uh that the projective transform um preserves is actually that if you have a line before or you know various points on a line and you um push them through the projection Matrix then they uh still lie on a line but everything else is in principle lost or can be lost depending on the values good um to summarize that um let's have a look at again at the number um of uh the degrees of freedom we already had this one uh translation pure translation only has two degrees of freedom in 2D because you can shift uh your coordinate system when you additionally allow for a rotation you have one degree of Freedom or this Theta angle from the rotation Matrix uh you can additionally allow for a scale which adds another degree of Freedom um fine uh Matrix has 6° of Freedom um which is a bit more difficult to imagine um but in principle you have do you have the 3° from here um you have you can scale the the uh x-axis you can scale the y- AIS and you can change the SK factor between the two and that then gives you six degrees of freedom and the projective Matrix has as as we saw a degrees of freedom because there is this scale ambiguity then because we apply it to homogeneous coordinates good that that was all very theoretical so let's have a look at uh some um examples that that hopefully help you to uh to to understand these Concepts so usually we have a robot that's located somewhere in space uh in 2D space for the moment uh so we encode the position of the robot by uh its position in X and in y direction and it's heading angle um which usually is denoted by the by y or this um uh uh which symbol is it actually F sorry good okay so um that gives us uh this rotation Matrix here um cosine s you've seen it before and X and Y for the translation and um yeah so this is an element of um it's sometimes denoted by SE E2 special oian group because this is a general oian transformation in 2D space and and therefore this group of matrices that have this shape or this this this um that are oian are called special oian group but of course it's an element of uh or it's 3x3 uh Matrix good so let's plug in some values imagine we have the robot located at 0.7 0.5 and at at a your angle of 45° then we would get this transformation matrix good uh and now imagine the robot moves forward uh a little bit um say with 1 m/s uh in its local frame so this could be the quadopter for example that has this orientation and now we fly we uh command it to fly forward and then of course it will not fly forward along the x-axis because it doesn't know about our x-axis but it flies in its local frame forward so um we could ask for example what is its speed in the global frame and you remember this is exactly the problem that we can solve with multiplying a matrix with with a vector um we have um now first in homogeneous coordinates it moves forward and say in the local frame the x-axis is pointing forward and the y-axis is pointing to the uh uh to the left um uh then we have this this uh velocity 1 we can augment it to homogeneous coordinates which gives us a vector 1 1 and um yeah and now we want to know what is its speed in the global coordinates and um the way to compute that is just to multiply this post Matrix of the robot by its local um uh by by the local coordinates so again this is our post Matrix here we multiply it by our homogene continuous uh speed and that gives us um oh wait yeah but this speed is now larger than there is something wrong uh what happens here I think that I did something wrong with calculating this think there should be a zero here uh I think spe VOR should be one not one because uh like are you moving the robot in just in X time direction or also y direction no no we move the robot in its local frame one meter forward and um yeah then this should not be the velocity but this is its position right we have a position in the low yeah sorry okay this is the mistake um so I fix that on the slides this is not the velocity that we're looking at at the moment but um the absolute position and in the we are looking at a point located in the in the robot's coordinate system at in 1 met distance from the robot um that gives us these homogeneous coordinates in its local frame and uh we transform these local coordinates to uh to global coordinates and that gives us back this point um which is located at 1.4 M and 1.2 M okay sorry the velocity should be the same uh yeah for the velocity we are only interested in the rotation part so for the velocity you would have to multiply the velocity vector by the rotation Matrix to transform it from one coordinate system to the other one but now but in this in this pose we also have the translation included and so it's about absolute points um okay so in so so the thing that that uh that's wrong here is this is not the speed but the position so in the local coordinate system of the robot so imagine we had before this red coordinate system we we now look at the point uh one Z which is located here why is same because if we take tals one the velocity multip T is uh yeah because the velocity so to speak is the the point here and minus the point here would give us the velocity so in in this so if if you multiply a point um uh if you have a 2d pose and you multiply a point uh with it uh then you get um then you get this point in in in in global coordin but if you if you want to multiply a speed uh then you would only multiply it with the rotation the alternative would be to take um so so we have you know our point at the moment which would be for example um 0 0 at the current uh or 0 01 to augment it and we have a second point x P1 afterwards uh and we would say you know we take minus here plus here and that gives us then uh the the the length of the vector that it traveled uh between the two points in time or alternatively you just take uh the rotation Matrix and you multiply it by the speed Vector um to get the [Music] speed that us the the distance yes the travel yes yeah exactly T is one then it's divid the distance divided by one yes is equal to the velocity so it should be the same yeah then it so if if you compute it like this then it would be the same yeah we did comp like this no no we don't have a minus here we we just have we have a point in the local coordinate frame of the robot multiplied by its post which gives us the global coordinate of this point but the local the local coordinates is one Z yes so it's oppos that we start with 0 and to the one yeah yeah so even if you start with z zero you don't get zero at the end um look if you if you have a zero here then you still have this one here and that gives you the absolute position of the robot so for getting out from the pose the absolute position of the robot you would multiply the pose with a vector 0 0 or 0 01 and that would return you this translation Vector here and if you have one Zer in front of the robot then we then we get back this point and you can subtract both of them and then you get the relative distance that the robot moved in global coordinates and a faster way and then you could divide by the time and the faster way to compute that would be to take the rotation Matrix multiply it by the um velocity sorry about this confusion result is just the the the result is correct uh yeah exactly exactly yeah yeah right yeah yeah yeah because that that would be the thing that we would subtract here um yeah okay good um yeah sometimes uh you need to do the inverse so sometimes you're interested in a in a global point or you know a global point that you want where you want to go to for example and you need to know in robot coordinates where this point is located um and um that just means that you you have Global V Global or X Global you want want it in local coordinat so you can just take the inverse of this Matrix um or multiply by the inverse of this Matrix and then um that that allows you to compute the local coordinates um that means of course that you have to invert the pose uh using G elimination for example but because the pose the 2D pose has the special property of having an orthogonal Matrix here there is a faster way to do that so remember again our pose always looks like this you have a rotation Matrix you have a translation vector and um it turns out that or it's clear that the this upper part here uh the rotation Matrix can can be inverted by taking the the transpose because it's an authon normal Matrix and um the the the new translation can be computed by taking minus um transpose rotation times the translation vector and that's of course much faster than running gaus elimination and so on you can just take Matrix transpose um yeah good but in this way you could go from global coordinates to local coordinates and this is um I mean this always uh uh you always need that and you also need it in the first exercise sheet so this is why we do that here in in much detail um good um so now let's consider a different motion um or before we just had a point in front of the robot that we converted between local and global coordinates and now imagine we have a real motion so that means Rober moves a particular distance forward sidewards and turns so that means that we get out a uh posst Matrix again or a transformation matrix again so um this first motion you know gives us a transformation matrix like this uh 10° uh and yeah 20 cm and 10 cm um and now imagine that after this motion um yeah we we we want to know what the pose of the robot is and and um now we can just concatenate these two Transformations um so we have our first post Matrix where the robot was located originally and we multiply that by the new motion and um yeah and that gives us again a 2d post Matrix uh where again the upper left part is a rotation Matrix which is orthonormal and uh the upper right part is the translation then um good yeah and um important property of course the order matters um so it I mean yeah you know it from linear algebra but also it makes intuitively sense it's of course a different story whether I move you know one one meter forward and then turn left or whether I first turn left and then move move one one meter forward um it may sound simple but you may find find yourself in in having this problem at some point in your program so then remember it the order matters of concatenating Transformations good um so now let this was everything in in or this all of this was now in 2D but exactly the same thing generalizes to 3D it only gets harder to imagine and it's it's important to note that it's that it's difficult for everybody to imagine um except for when you have a super mind but usually people have in 3D problems to imagine rotations correctly and so on so you're probably not the only one if you start thinking about 3D rotations then and you're getting lost and it's it's normal sometimes people even construct uh coordinate systems that they can uh look at you know and then think about I have to first around the red axis and then shift forward and so on good so uh a translation in 3D has exactly the same shape EXA except for that the translation Vector now has three elements uh for X Y and Z and uh the identity Matrix here is has a size of three 3x3 um oian transform also has the same shape um uh but again as here the rotation Matrix has a size of 3x3 and again rotation Matrix has to be also normal which means that all the column vectors have to have length one um row vectors have to have length one and rotation Matrix transpose times rotation Matrix is the identity good and you can Define similarly uh scaled rotation fine transformation and projective transform and so on for us most important is the oan oian transformation which again has this name se3 for special oian group um it's um yeah it's interesting to to note that uh the degrees of freedom actually uh do not scale linearly with the number of Dimensions so for translation we have of course three degrees of freedom but for um an oian transformation we already have three degrees of freedom in the rotation remember in 2D we just have one degree of Freedom the Theta angle or the heading angle but in 3D you have a rotation around the X axis y AIS and Z axis so it's three degrees of freedom and um yeah this goes up to 15 degrees of freedom for the projective for a projective transformation um but it's it's of course hard to imagine all these 15 degrees of freedom good uh so as I said most important for us is the Ian transformation um just to make it explicit we have this 3x3 rotation Matrix block and um a 3x1 translation Vector good um the translation is usually easy to imagine or and easy to represent because it just take the the three coordinates um but for the rotation Matrix uh there are different possibilities um the rotation matrix by itself is called uh the special orientation group s SO3 um yeah just because it's uh important and used often it has this uh this name and uh it's also I would say the most common form of um of um defining a rotation and the most um uh um uh how do you say um the best defined one so to speak is the rotation Matrix uh it's just these nine values uh but there are also three other uh forms that are used frequently Oiler angles uh the AIS angle representation uh and unit Quan so um uh yeah in this as a rotation Matrix you can again imagine that the column vectors correspond to coordinate AIS um yeah and it just does a basis transform from from one coordinate system to another coordinate system um and things that that we usually or yeah well yeah that that we usually do with a rotation Matrix is to rotate um uh a point or to concatenate them and to invert them um and this is also where the large Advantage is uh it's very easy to um concatenate rotation matrices because you just multiply them um it's also very easy to invert them how exactly uh rotation Matrix just take the transpose um you can also easily multiply it by a vector yeah just by multiplying it one large disadvantage on the other hand is that it's completely over parametrized so it has nine variables although it just encod three degrees of freedom and that means it's not memory efficient you always need three times more memory than you actually um have to spend uh and uh it's also in terms of optimization hard because if you want to find a rotation Matrix that explains for example the the current view of the or the current pose of the of the robot then you would have to optimize over all these nine variables that are somehow linked because you have to ensure that all of them that the Matrix remains orth normal and so on and that that is very complicated so when you have to find um yeah and this is why this is why there are different representations and the probably most known one um Al not the use most most useful one in practice are the oiler angles and uh that's just a product of three consecutive rotations for example uh in uh aerial navigation um this Ro pit convention is very um frequently used uh which means that you first so you have you have your your plane for example and um uh the first angle um uh gives you the rle which is the rotation around the the long side of the plane or the forward Direction This is the so so to speak the x-axis of the plane uh the second rotation goes around the y- AIS so that's up and down so it's called the pitch of the plane and there is the the Y rotation around the um the set axis of the plane and um it's easy to understand uh but it has yeah it's easy to understand Al of yeah formulas are not so uh easy to remember um what it in principle does is it it rotates by around the x-axis by by an angle then it rotates around the y- axis and then it rotates around the Z axis so when you look at all of these three matrices they are of course simple and then you concatenate them and then you get this large expression here um and similarly there is a conversion step that that from a a rotation Matrix gives you back the three the three angles um and this convention this R pitch your convention is the most frequently one used but in in principle you can have all different things you can also have it the other way around you can first turn rotate around this the set X is then by y then by x uh you could also have X set X and and so on and you would you would laugh but it's in principle there are 24 different types of Oiler angles and it's never clear if somebody gives you a coordinate or a um if somebody gives you a poses of a robot and he says these are Oiler angles then uh you're usually screwed then you have to try out different uh ways you can this would be the first one that you try because it's the most common one but in principle it's it's not defined and then uh for example in the um motion capturing system that we have in Bari um the motion capturing software really allows you to to select then one of this 24 different um uh systems so it can be very confusing um on the other hand it is very intuitive so it's still frequently used um advantage of Aller angles are that it's a minimal representation it just has these three degrees of freedom it has a very easy interpretation um disadvantage as you already saw there are many different alternatives and all of them are actually used or there are several that are frequently used um problem is they are difficult to um uh concatenate because I think I have another slide on this um because it can't just sum up the angles because as we said as you've seen before the the order really matters um and there are so-called singularities so uh let's have a look at this in more detail um so for concatenating Oiler Ang Les there is no other way of going to a rotation Matrix then multiplying this rotation Matrix and then converting this back to Oiler angles and that consumes of course lots of time so not convenient um to invert Oiler angles again you have to go to um the rotation Matrix invert and then convert back and um for suming and and it's you know sometimes you might think yeah when I have two Oiler angles that I want to combine then I just take the sum of them so the uh the sum of the Roll angles and the sum of the pitch angles and the sum of the Y angles it's not not correct it not doesn't give you the right thing might be convenient but it's not not right um the other thing is that uh there is um there are singularities in Oiler angles uh which means that it can happen that all of the rotation um uh uh rotation um uh rotation axis get aligned and then you lose uh one degree of Freedom so you can imagine the oiler angles to be um or you can actually have a a physical construction that um that's uh which is called a gimbal where you have three rings that are uh linked by each other so you can rotate this outer ring around the set axis which is the y axis then you can have a second ring inside of that connected to that that you can rotate around uh the y-axis and then the last one for the roll um that you can turn around around uh the x-axis and um when you bring all of these three rings in uh in the play now position then you suddenly can't rotate anymore around this axis so there is um there is and this is called the gimal lock uh when you yeah when yeah when you can't um when you lose this one degree of freedom and because these kinds of or this kind of construction mechanical construction has been used in in Planes uh before for gyroscopes as we will see later um and this is actually a way to to crash an airplane so if you have a happen to have a a um gyroscope built with a gimbal uh then it can happen that in some particular orientation all three A's get aligned and then um then you're then you're lost then the gyroscope can't can't give you the right orientation of your of your plane good so this means uh uh probably don't take Oiler angles um if if possible um another uh way of representing an uh an orientation is the socalled axis angle representation it's also intuitive because you you say you have a rotation axis uh n uh and you rotate uh around a particular angle around this axis for a particular angle angle Theta so this means that you can uh that you have in principle four parameters because you have this rotation axis and uh the rotation angle uh but you can also say that the uh rotation axis has a length or the the vector encoding the rotation axis has a length of one uh and then um you can use you can um um uh specify the rotation around the axis by the by the length of this of this Vector so you multiply you have a normalized uh Direction around which you rotate and you multiply that by by Theta and then the length of this Vector is the rotation angle and um uh yeah and the direction is the rotation angle and this this representation is also called the angular velocity and um yeah it's it's used a lot also by Christian for example in his current work um good thing is it's also minimal so it has only three degrees of freedom um uh but it's not unique so can anybody imagine um why does is not unique yes toate the L of this thisa then it will always be POS so I can say um so you can you can flip um the direction of this rotation axis and then just take the negative no but wait that would give you the same that would give you the same Vector at the end now but one um easy thing is you can uh I mean always when you have angles there is this 2 pi perod doity so if you rotate by 90° it would be the same as rotating by 90° plus 360 so there would be in principle two vectors that encode the same rotation um yeah good uh there are again some formulas you don't have to remember these formulas um but it's just important to know that you can go of course from the axis angle representation to a rotation Matrix and um uh and back again so if you have a rotation Matrix you can compute the rotation angle and the rotation axis from that um yeah and there is this is if you're interested in this in a more detail there is um uh this book that I would like to recommend an in invitation to 3D Vision and it's also available online but for for the rest of this course it's it's not important it's not important to know how this can be derived good uh there are also quorans um who has heard of quarians before ah okay almost everybody um so a quan has four elements three imaginary um um components and one real component um a typical Quan doesn't have any side constraints but uh it's usually useful to talk about unit quion that have a length of one and um uh should be noted that um uh the the pans uh with a different sign encode the same rotation so um uh yeah you have two half spheres essentially that encode uh rotations and uh opposite points correspond or yeah correspond and encode the same rotation um yeah the cool thing about cence is uh that um the multiplication and inversion um is is very efficient uh there are direct formulas to to concatenate them and to to invert them uh and and so this is very speed efficient so if you have lots of poses uh or this is a reason for for for many people to to choose querian um because they are so efficient especially when you when you have a small embedded system like a quadcopter and you only have a small arm processor then um uh using rotation matrices is maybe not most convenient thing and and and using Oiler angles is maybe also not the best thing because then you need to compute s and cosine and so on and that uh yeah might be slow or this will be slow definitely good um yeah uh you can also multiply equatorian with a with a vector directly so for example if you have um a position in front of the robot as we had before um and you want to convert it from local local frame to the global frame you can also do it directly um yeah and there is there are direct sometimes or there are direct conversion formulas between querian and this X's angle representation um yeah which can be useful good as I said before it's it's really hard to uh read 3D orientations or or um uh rotations in the general case I mean of course everybody can recognize an identity Matrix and knows that there is no rotation but uh as soon as uh you have several rotations around different axes it's usually extremely hard to to read it off the paper uh so don't I mean you can try but um um usually it's hard to see it from the numbers but um um most people find it very easy to to understand a rotation if you if you see it visually um and so um for debugging things like uh this or for for working with 3D poses it's usually useful to have a 3D visualizer um and in the in the lab course we use Arvis for this which will help you a lot as you will see so so whenever you have to to debug a rotation try to send it um to to Arvis and then Arvis will display um the coordinate system or um lines or whatever you want to use to visualize it in 3D and then you can easily see where it goes wrong whether it turns in the right or the wrong direction um it's also important to know that I mean i' I've shown you now some formulas but it's generally not necessary to implement them yourself there are very good uh linear algebra and geometry Li that you can use um or there are yeah yeah um for example you could think that a good choice would be to use a c array to to uh save a rotation Matrix or a standard Vector uh I mean it's of course possible to to store the values in there but it's it's not a good idea because they don't have linear algebra functions to multiply them um the scientific library has been used a lot until recently I think the problem is that it's plain C so um most people use C++ at the moment of course and then um this is a bit harder um to to use um uh Ross messages used uh use boost arrays uh but they have the same problem as the the standard vectors they don't have linear algebra functions on them so what you usually have to do is to when you get when you get um values or uh you have to copy them uh from from the Boost array type to another type and and usually there are already helper functions that do that for you um there is a bullet Library that's used for example by uh Aris a lot uh and and TF I think Al I'm not sure whether it's still used by TF um but the the best Library probably at the moment for for C++ is is the igen library um which has lots of linear algebra functions and also um geometry functions so it has a role pitch your convention you can convert back and forth to oer angles um you can concatenate them you can extra checked Quan and so on so uh my Rec recommendation would be to use that in the lab course um there is uh this TF package I'm not sure whether you've seen that already in in Ross so there is there are some geometry functions that have already been wrapped um or that allow you to to uh to send uh Transformations between between programs and um uh this TF module essentially provides you or is a provides a data structure that allows you to save transformation between coordinates frames and that's a very typical application that you have so for example you have a quadopter that's located somewhere in space and that knows its position so you have a transformation between the Bas link of the quadopter and the the world origin and um now you can um publish this transformation uh and then uh um all nodes that that have a TF listener um um Can can read it out and use it for something and the cool thing about TF is that it can allows you also to to uh to query arbitrary uh Transformations um between between known coordinate systems so for example it could be that the quadricopter itself um has has several coordinate systems for its rotos and uh it has a sensor for example so there is a transformation between the the center of the quadopter and the camera uh that you calibrated somehow and then a camera sees a person for example that uh it wants to follow and um the TF module would then allow you to um to to retrieve um the the coordinates of the person in the map just by concatenating everything so it will look up a path and then concatenate all the [Music] transforms and uh this video this can be arbitrarily complex uh is for example the the PR2 robot uh has of course lots of different coordinate transforms um so you can see this is the the world origin and uh now this is the robot and it has all of these uh red green and blue um lines uh um uh visualize one coordinate frame and when the robot now sees something in the world it can immediately or it sees something with its camera but it can immediately tell um uh the the TF module that it um yeah that it seen something in local coordinates and then TF will compute global coordinates from there good okay and then uh as a last thing um we have uh 3D to 2D projections um uh there are orthographic projections that we are normally not interested in but we are interested in perspective projections because this is what a camera usually does a camera with a normal uh a pinhole camera does so idea is the following we have 3D points somewhere in space and we have um a camera um or we have a yeah we have a point somewhere which is called the focal point um and then uh all the 3D points uh are uh connected to to this focal point and then we have somewhere image plane uh that intersects these rays and uh that gives us a uh projection of uh of this 3D point on the image plane so in terms of notation I will always denote a 3D point with p and a 2d point with X so the idea of a itive projection is that you have have a 3D point that you project on on a plane and uh in in terms of um uh uh protection uh uh projective transformation this looks as follows so we have a 3D Point P that's with coordinates in the camera frame we have and and we want to compute the 2D point x on the image plane and the camera camera the pinhole camera model now says you take the homogeneous coordinates of um of p and you multiply it with this um projection Matrix which has a very simple form in its generic case and then you get back the image coordinates again in homogeneous coordinates and uh as xtia is homogeneous you need to uh normalize it by its last element for getting the real um uh the real 2D coordinates and um uh the problem of course so far is because we don't have any scaling um when we have a 2d when we have a 3D point in meters and we project it on the image plane then also the 2D point will be given in meters on the image plane and that's usually not what you want but you want to have the the 2D point measured in pixels and that means that you have to have some um scaling for the that encodes the size of the pixel in the real world um and usually an offset of where this uh where the where the origin of your sensor is located uh with respect to um uh your Optical axis um good and um this scaling and offset can be um uh expressed as a transformation matrix like this so here you have the focal lengths in X and Y Direction uh which encode the zoom factor or the size of the sensor so to speak you have CX and C Y which encode the um Center the optical center um you know essentially it encodes where the sensor is located and usually it's located in the middle um middle of your um yeah it's yeah so the center is usually in the middle of the image um this skew Factor here determines the angle between uh the X and the y- axis uh with almost all normal cameras the Q factor is zero but in principle you could build a sensor that is somehow tilted and then you can express it with this s good um and um usually a point of course is or depends on your application but it can happen or yeah it can happen that you know the points in World coordinates and then again you need to transform it from first from so for a point in World coordinates when you know want to know its position or it's its pixel position you first need to convert it to local coordinates in the camera frame and uh and then you can project it down to so the full thing looks as this you start with a 3D point in World coordinates you transform it to um local coordinates and project it uh with the pinhole model and then you apply the intrinsic Matrix and that gives you then the pixel coordinates good so this um was the first first part now or the most the largest part of this of today we've looked at uh points lines and planes and two 2D and 3D Transformations and um uh We've looked at different represent presentations for 3D or orientations um Choice depends on the application of course and we briefly looked at 3D to 2D projections um it it's it's in principle really a boring topic so I'm I'm sorry that I had to expose you to that but it's super important because uh you need to really know that by heart for the lab course um to because you need to uh work with with 2D and 3D geometry good so that brings us now to the more relaxing second part of today in the remaining time that we have um so as you know and we've seen last week there are plenty of different sensors of course that you that you can use um and um you can uh categorize them of course by the modality that they observe for example there are tactile sensors um that you can use to detect whether you have hit the ground or um uh with your with a quadcopter there are wheel or motor sensors that directly measure for example example how uh quick how fast um uh rotor turns um you can have uh heading sensors that measure um the orientation of your quadricopter uh you can have uh groundbased beacons that that directly give you the absolute position of your vehicle uh in in a world coordinate system something like GPS for example falls into this uh category uh but you could also have um beacons that you install in the world um that you that you can use to to um compute your position uh then there are um active active range sensors that um send out something uh and then measure the response from the world um there are also uh things like radar uh that measures the speed uh of something or or your own speed uh if you measure measure uh the distance to to to a fixed fixed reference uh and of course Vision sensors um I mean the most um common type of course is a camera but also in the AR drone we already have um integrated packages that uh compute um visual odometry already very close to the camera so if you uh look at uh what what sensors we actually have on the AR drone then you see that we already have a large uh variety of different things so the AR drone measures how quickly the uh rotors turn by measuring the currents in the motors in the motor drivers uh it also has an IMU unit which is composed of a gyroscope and an accelerometer um it uh supports uh beacons uh the AR drone can recognize um uh colored uh colored markers uh that you might have seen I think it's orange and blue and you can for example two there there is a game that you can play with two AR Drones then um uh uh something like a death match you can shoot at each other and then when one AR drone detects the other one then you get a point um but it uses its forward camera for that uh of course um then it has an ultrasound sensor that measures the the distance to the ground um and of course it has uh two cameras One forward-looking camera one downward looking camera and especially the downward looking camera is um is already um packaged with a visual odometry sensor that computes um the the the the speed um of the vehicle and and the AR drone uses that for correcting for drift good and then you know there are of course all kinds of performance measures that you are usually interested in um of course you're interested in the uh frequency of a sensor so a camera gives you images at 20 HZ or at 30 HZ but can go up to 500 or even more there is with some sensors um a significant delay that you have to be aware of um for example GPS provides you at with new poses only every uh um uh 500 milliseconds I think um sensitivity of course how um uh um yeah that tells you how much you can still sense um some sensors especially active sensors have problems with with cross talk so you can't have arbitrarily many ultrasound sensors in the same room because then they will hear each other um there is uh also um different uh accuracy of course of different sensors um some of these um errors that you get can be removed by carefully calibrating your sensor but of course there are also just random errors um noise so to speak that corrupts your data and then other um aspects are the weight uh the power consumption also the price for example depending on your application uh yeah so so now I want to briefly show you um more details on some of these sensors that are relevant for uh quadrocopters in particular and that are cameras gyroscopes accelerometers GPS and uh range sensors good um we had it before the pinhole camera model or the the projection this is where it comes from we have an object somewhere in the world uh it's lit by the Sun or by by a light source and then of course all of all the points on the uh object will radiate uh light and um if you have um somewhere a small opening the pin hole then these Rays uh will uh will shine through the hole and if you have an image plane behind behind the pin hole then you will get an um uh an inverted image then here on the image plane I guess everybody has has seen that before the problem with the pinhole camera as it was originally invented is that this pinhole hole is very small and so only very little light gets in um you can uh increase the amount of light when you add a lens um then this lens will um um uh how do you say um will will um change the direction of the ray so that everything that's um reflected here by this point on the object will converge again on the same point of the image sensor and that gives you of course Much More Much More light um this only works when um the the object is in Focus so you can imagine if you move um the the image plane forward for example then um the the Rays would not yet intersect or on the other hand if you move the object backwards here then the same effect happens um so this means that you have to to focus um your lens or you have to decide on a particular Focus that's relevant for your application and um and then when you're potentially too close or too far away from your object then you it will be blurred on the image so the AR drone has fixed uh Focus lenses so that means that they have been set up in a way that in the operating range it should be sharp but of course when you get too close uh then then it will be um blur blurred um but when you look at real cameras then there are some other problems that you should be aware of um there is especially with um wide angle cameras and also with cheap cameras uh always some radial Distortion which comes from the from the cheap lenses um when you when you buy an expensive SLR then of course um it's possible to manufacture a lens that has less derivations you can also correct for it in software um but with most consumer type cameras there will be some radial distortions and you notice that that when you look at the straight line then it won't be straight in your image so whenever that this happens um you can also um check this with the AR drone when you look at at the window or something that should be straight and it is not straight this is this is a picture from the AR drone from from yakob from Jacob's Master tees actually um you see that it's not straight and that usually means that you should correct for it somewhere somehow and a very e simple model that's that's typically used and that's also used in the open CV library that that we that we use and that Ross uses um uh is to transform the the pixel coordinates in the um um uh in in the distorted image to um yeah to undistorted and yeah good uh then there are some other effects um that uh are interesting to know um uh first of all um the image borders uh usually get less light uh because they are further away from from the scene and then um yeah less Rays actually fall fall onto the image borders um there is also the problem of the Bier pattern um usually you just have a monochrome camera or cameras in principle are monochrome and for getting color images um usually um people add an um an a colored image mask and this means that every pixel um just absorbs either uh blue or green or red um and this means that effectively you have a A reduced resolution by a factor of four on color images uh and then there is a process called debing where you again try to estimate the full color of every pixel uh from its surrounding but you should keep in mind that even if you have a 20 megapixel camera or a 1 megapixel camera uh that gives you color images that probably there are only 1 million photo or light sensitive cells and that um one3 of your um resolution is actually just estim from or one third of the values are just estimated from the surroundings um other problems are rolling shutter um which means that the the image is not read out completely at the same time but that the uh the camera reads out line by line and this means that the the uh lines at the bottom are actually newer than the lines at the top and this could mean for example if you turn the camera really quickly um that um uh yeah that you get a Q uh between between the two coordinate AES um another problem of course is motion blur if you move too fast uh and the the shutter time is too too large then you get blurring another problem is jpeg compression also the AR drone actually compresses the images when it sends them to the ground station and this means that you could get or that you get artifacts and then depending on the compression setting these artifacts might might uh um actually impair your um uh image processing and you have normal noise of course especially when you have little light good then uh let's go to Mechanical gyroscopes uh first so um a gyroscope um is used to measure the orientation so for example um yeah I guess everybody knows this device or um for for children that you can you have a spinning wheel and it always tries to maintain its uh orientation and that makes it uh not fall over um and the same principle can can be used um in in uh in in three dimensions actually so you have again this this Gimbal and in the middle you have a very uh heavy wheel that that you spin and that then from then that moment onward will try will keep its position or will try to keep its position and early uh planes really had a device like this so when you get in the plane you switch on a motor and then it starts up and that is then the gyroscope that's accelerating and um as you've seen before this gyr scope in this construction has a problem of the gimbal lock so depending on how long you fly and in which direction it could happen that all of these AES at some point uh line up uh and then uh then you're U uh screwed um also when you have this um um uh a gyroscope like this and you wait for half a day and then the Earth turns then you will also see that this dis virtually it seems to to rotate but in in in reality of course it's you that rotated and the gyroscope just keeps its position um of course nowadays um uh we we use um uh different sensors um most of them are um based on this Ms technology where you have a small integrated circuit that um that reads out um that that really a mechanical circuit that uh reads out the orientation changes um for example you can have a um vibrating gyroscope where you have a small chip and then it it really VI um has a yeah has vibrates at a certain frequency and whenever you um turn the chip um it the rotation will try sorry the vibration will try to keep its um vibration Direction and then you can use that to read out the um uh the change in in um in orientation different techniques also include and more accurate techniques actually are uh fiber optic Gyros and ring lasers uh problem is that they are larger of course so they are um used on on planes for example uh while on the uh quadopter we have then more things like this which is of course less accurate um an accelerometer measures the external forces that that act or yeah it measures the external forces and that usually um uh means you measure the gravity um you can imagine that as a as a Mass for example that's located between two Springs and um and when you accelerate it to the left then the mass will of course shift to the right for a few seconds and then by looking at how much it shifts you can read out the acceleration and um of course such an acceler accelerometer just measures the acceleration in One Direction so you would have to have three of these to measure the accelerations in all possible directions um yeah of course again in practice uh we use M sensors um they have again in principle the same uh mechanical structure just much smaller so you have a small chip where really one part can um yeah contains contain something like a spring and uh when you move the chip then um uh the um the the plates will uh will come closer or further away and that changes the capacity for example and then you can read yeah read out by how much and so on there are different different principles I think the most common ones are the capacitive accelerometers but uh you can also change the resistance somehow and um uh or do it optically or p p to Electric good okay and um both of these uh sensors the gyroscope and the accelerator or the MS gyroscope and the MS accelerator just provides you with one measurements of one AIS um so to to get the real 3D um rotation you typically combine three gyroscopes that you mount orthogonally to each other um and the same is um is true for the accelerometer so it just take three of them and uh that gives you a threedimensional vector then uh the gyroscope um gives you um the angular um acceleration oh no sorry angular velocity yeah exactly the angular velocity and but what you typically want is of course your orientation in space so you integrate it up to get the angular position and uh problem is of course when you integrate something that is noisy then you get some drift and this this means that the accelerator depending on its quality will drift slowly over time um for example 1 degree by hour per hour 1 degree per minute depending on what sensor you have and this is called the gyro bias and um the accelerometers um provide you with the accelerations and for for getting um speed for example you would have to to integrate over the accelerations and then integrate again to get the position and you can imagine that that would be super uh inaccurate uh super noisy so usually people combine um so people just don't use accelerators for for getting positions but they use um the accelerator only to get uh the direction of gravity and then use that over time to correct for the drift that occurs in the in the gyroscope but for finding the position you usually need some external sensor like a camera and this is why we need visual navigation um good uh how does this look in practice this is a the autopilot Board of the the Pelican quadrocopter uh that we have um um let me look where is the so we have here the the three Gyros 3 four five so here is one here is the second one here's the third one and you see um they are mounted orthogonally to each other um not exactly at the same spot so that that's where a small inaccuracy might come from uh there is the acceler acceleration sensor too that contains all three AES of the accelerometers um yeah and you can also see here there are two um embedded processors that do some of the lowlevel um processing and uh there is also a pressure sensor and um a compass sensor that's mounted somewhere else on on the quadopter compass is always a bit tricky because when you have uh Motors then the motors also work with magnetic fields and um then they distor of course the magnetic field of the of the earth which is very weak uh so you have to make sure that when you mount a compass that you mount it at a place that's not strongly influenced by neither by the electronics nor by your Motors good then uh we're almost through I think the GPS I guess most of you know so you have GPS satellites um in an orbit around Earth and um you measure at least least four of them uh that or you receive the time from four of these satellites and their positions so the satellite transmits its position as well together with the time and then from this you try to estimate your position and um uh yeah that works surprisingly well and uh devices got surprisingly small also disadvantage of course is that you have to have a direct line of sight more or less to the sensors uh to to the satellites um and that doesn't make it useful in buildings um uh and then the satellites of course need to know where they are and that's again done by monitoring station that monitoring stations that exactly measure the position of each satellite and then send corrections to the satellites so that the satellite again can broadcast its position to the user um so there are lots of GPS satellites um the system needs at least 24 of them uh each of them has a 12-hour orbit and they um yeah are located at 20,000 km height um there are six different planes um um with GPS satellites and and each of these uh planes has at least four satellites um and yeah as I said each satellite transmits its location or its um yeah rotation parameters and uh its current time and um you have to receive or you receive these messages uh with your device then and that um because this is transmitted so slowly this really takes takes takes a while so when you switch on a GPS device that doesn't have an internet connection for the very first time it takes at least or it yeah it takes at least 12.5 minutes to receive the full message from every satellite um of course Smartphones at the moment have um have different techniques so they can download the orbit locations from the internet uh and that makes it then makes the initialization um faster and also they get an approximate position which makes the computation of the position uh more efficient um there are two different operation modes of GPS um the most common one is the so-called pseudo range uh where you get measurements of at least four different satellites um the idea is in principle that you I mean why four thing is you have you want to estimate three degrees of freedom that's your X Y and Z position uh but you also don't know your exact time at the moment so you have four degrees of freedom that you need to estimate so you need four measurements um a problem from the pseudo range is that it's uh has a relatively low accuracy um between 3 and 15 M um it see might seem more accurate sometimes when you look on your phone but it's actually um it's actually so noisy and sometimes even more when you are close to buildings or trees um there are different systems that um so so um that additionally use the face shift of the signal uh but this face shift is is um somewhat difficult to or it's um it's Chang by the atmosphere and the atmospheric conditions um and so you need a reference receiver of which you know exactly the position and then you can compute your own position with respect to that um so there are different systems uh differential GPS and real-time kinematic GPS in differential GPS you have base stations that are maybe uh 10 kilometers away that broadcast the signal either over a radio link or over the Internet uh that you get the face shift uh to to the reference station and then you can localize with respect to that which gives you maybe around 20 to 30 cm accuracy and realtime kinematic GPS means that you have an own base station that you have in a car or somewhere on your building um and then you can localize R back to that and that really gives you accuracies down to 1 mm um even but that does require special Hardware or can you use the usual GPS uh it it so it requires um uh a better receiver that can measure the face shift uh but you additionally need for real-time kinematic GPS you need the base station as well with again a good uh antenna and a good receiver I imagine for the dgps um there should be like enough to get them over the internet and stuff uh yeah but you these are commercial Solutions then you have to pay people to to provide you with the correction values yeah and depending on where you live uh the next base station might be uh uh painfully far away good um so this is the last thing um Range sensors um as I said we have uh an ultrasound sensor on the AR drone uh but other types of range sensors are laser range finders that have been very or that are very popular in robotics in particular um and time of FL cameras and then uh sensors like stereo cameras and structured light uh that we will have a look on later so today I'll just want to look at sonar and laser range finders the the general idea of of most range sensors is that they emit a signal um to determine the distance along a ray so um they make use of the propagation speed of of the signal uh and just measure how long how long it takes to reflect the signal and uh receive it again um and then depending on what medium you use or what uh signal you use Sound Speed is around 300 m/s and light speed is much faster so uh for this you can measure yeah so so this changes the the properties of the sensor but also changes of course the hardware that you that you require um ultrasound sensors usually have an operating range between 12 CM and 5 m you have one speaker that emits the signal and um microphone that receives it again um the problem in air is that the you can't really focus sound very well so it usually means that you're you have an opening angle between 20 and 40° and you measure anything in that con um so that makes it a bit imprecise you never really know what you measure um another problem of course is that soft surfaces absorb the sound and then you might not get an echo at all uh the other problem is that you might get a reflection several Reflections uh in a room um that you have something like a multipath um um uh um measurements and that that gives you ghost Reflections that you have to deal with uh good thing about Ultra sound sensors is that they are really lightweight and really cheap so you can buy them for I think 5 if you want to to add something like this to your robot uh and it only weighs a few grams and it's easy to use but the problem is you can never get a good 3D reconstruction out of a ultrasound sensor of course because at this opening angle um yeah but the good thing is so this is on the AR drone and it's it's used to um to measure the height uh a laser scanner um Works similarly it it emits um Light Beam there are actually different measurement techniques um but most laser scanners um uh send out um a modulated puls um and then measure the phase shift um between the two signals the send out signal and the return signal and um um yeah and and with this technique you can um really measure very accurately so a normal laser scanner that people use on robots uh has an accuracy of maybe 1 to 2 cm M and it has an operating range of 50 to 100 m so so this is the reason why it's it's used um heavily on uh on the autonomous cars uh the only problem is that you have rotating Parts in between because I mean in principle you just measure the distance to one point but then you want to measure around the whole car so you have to rotate um uh your scanner and uh that makes it a bit more fragile especially when you have it on a car and you drive through bumps and so on um and it's also expensive so uh a laser scanner like this costs maybe around €5,000 and um these valine scanners that they have used on or that they are using on autonomous cars cost around 50,000 maybe so it's at the moment not clear whether something like this will ever make it into a consumer car at the end because then the sensor would be more expensive than the whole car why is it so expensive I mean the components itself shouldn't be so that's true yeah it might get a bit cheaper of course if you if you start mass production um but nevertheless it's a mechanically very tricky thing so a camera you know it's much simpler you just have one thing it um it nothing moves but that this thing really rotates and it it's heavy it weighs like 5 kilog and so the maintenance is much more complicated uh yeah but yeah I mean there are all cheaper variants but in comparison to a camera I mean a camera you know cost a few Euros but even a cheap laser scanner costs hundred of of of dollars I guess good um yeah sorry now we have um brief look at the first exercise sheet um as I said as you know the the um P of a robot can be P of a robot in 3D can be described by six parameters uh by its position and the oiler angles and um the robot usually makes sensor observations and in particular the AR Drones makes its velocity observations in its entric frame because it doesn't know about the world frame obviously um and um in the the task of the first exercise sheet is to use these egocentric velocity estimates to to integrate up um the position of the quadrocopter um there are two types of motion models that people usually use odometry motion model and velocity motion model on the AR drone we have this second type because we can't really measure distances but we can measure speeds and by knowing the speed and the time we can integrate this up and um and uh obtain the position and uh this that Reckoning or odometry computation is exactly this thing we uh uh it's a mathematical procedure to determine the present location of a vehicle from uh entric velocity observations and um just to give you an example of how this looks like uh don't expect it to be very accurate um um so the idea is that you that you get the velocities and then you integrate them up and then you get a certain uh trajectory out of that and um your task in the exercise sheet is next to getting the so so the first task actually is fun you're supposed to wait are no sorry no it's uh you you get uh data from from the quadopter uh that you have to integrate up and then determine how far it actually flew good um yeah already said that so we have a down looking camera that gives you the motion estimates ultrasound sensor to give you the height uh there is um yeah the gyroscopes and accelerometers that are already packaged into this visual odometry thing and uh then the quadopter gives you it's its IMU readings that we provide in a back file um and uh yeah and from these measurements you have to integrate up the robot position and this is this is the task good so that's it for today um as I said uh exercise sheet is due next Tuesday this Thursday there is again the robot lab with Christian and yakob um so if you have questions then or please come to the robot lab we'll make things easier and then hand in your submission in your group um to this email address yeah good so that's it then are there any questions yes yeah it's so one solution uh is is sufficient per group and then make sure that you write all the names of your group participants uh in the solution yes of group uh on May May 2nd uh and that's I think [Music] in in two weeks yeah but I I would uh really encourage everybody who's who's seriously interested in taking this lecture to um just also do the first exercise sheet and I think chances are really good to to get in for everybody good then thanks and then see you next week
Up Next

The Cross Ratio: Geometry of Roberto Carlos' Free Kick
@numberphile
313.4K views•2018-07-06

JEE Maths | Circles: Pole & Polar Explained with Equations
@TargetPublications
14.4K views•2019-02-23

Fourier Series Introduction: The Big Idea Explained
@DrTrefor
387K views•2021-05-03

The Mathematical Impossibility of Accurate World Maps
@Vox
23.3M views•2016-12-02
Related Study Plans & Knowledge Roadmaps
Structured learning paths in Mathematics







































