How to Find Eigenvalues and Eigenvectors | Linear Algebra

[00:00:00]It’s Professor Dave, I wanna tell you about eigenvalues and eigenvectors.

[00:00:11]Eigenvalues and eigenvectors represent an incredibly useful concept in linear algebra, and we can see their application not just in math, but also in physics, especially quantum physics, which we have begun to learn about in the modern physics series.

[00:00:27]Eigenvalues can be used to solve systems of linear differential equations, describe natural frequencies of vibrations, separate modes of motion, distinguish states of energy, and much more.

[00:00:42]So just what are they exactly?

[00:00:44]Let’s say we have matrix A, a square, n by n, matrix.

[00:00:51]Eigenvectors are vectors that have a special relationship with matrix A, such that when you multiply A times the eigenvector x, you get back that same vector, multiplied by a scalar, lambda.

[00:01:05]These scalars are called eigenvalues, or sometimes characteristic values.

[00:01:11]In addition, these eigenvectors must be nontrivial, meaning that they are not just the zero vector.

[00:01:19]The matrix A can have multiple eigenvalues, but no more than n, the number of rows and columns in the matrix.

[00:01:27]Furthermore, each eigenvalue will have its own eigenvector that is associated with it.

[00:01:35]Before we start trying to solve for eigenvalues or eigenvectors, let’s see them in action, so that we can further cement the definition we just mentioned.

[00:01:45]Let’s say that matrix A is a 2 by 2 matrix with elements -3, 1, -2, and 0.

[00:01:53]Now let’s check that the vector x = (1, 1) is an eigenvector.

[00:02:00]To do this, we will simply multiply A by x and see if we get back x multiplied by some scalar.

[00:02:10]Performing the multiplication, we get (-3 times 1 + 1 times 1, -2 times 1 + zero times 1), or (-3 + 1, -2 + 0), which simplifies to (-2, -2).

[00:02:28]We can simply factor out a -2 from this and get -2 times (1, 1), which is just our original vector multiplied by -2.

[00:02:38]We have not only verified that (1, 1) is in fact an eigenvector of our matrix, but also discovered that -2 is one of its eigenvalues.

[00:02:51]Now that we are better acquainted with eigenvalues and eigenvectors, let’s start learning how to solve for them.

[00:02:58]We know that the eigenvalues and eigenvectors of a square matrix, A, obey the equation Ax = λx.

[00:03:08]In order to solve this, we will first subtract λx from both sides.

[00:03:13]Then, in between lambda and x, we will insert the identity matrix, I, which is the matrix equivalent of multiplying by one, so we are not changing the equation by doing this.

[00:03:26]Now we have Ax – λIx = the zero vector.

[00:03:32]If we factor out the vector, we can rewrite this as the quantity (A - λI) times x = the zero vector.

[00:03:41]Now we want nontrivial solutions to this equation, meaning that x can’t be the zero vector.

[00:03:47]Let’s see what this means for (A - λI).

[00:03:52]If this matrix were invertible, then we could multiply both sides by (A - λI) inverse, and get x = (A - λI) inverse times the zero vector, which would mean that x is zero.

[00:04:07]To avoid this situation, (A - λI) can’t be invertible, and therefore, as we recall from learning about inverse matrices, the determinant of this matrix must be zero.

[00:04:22]The determinant of A - λI will form a polynomial in λ often referred to as the “characteristic polynomial”, and the equation formed when the determinant equals zero is called the “characteristic equation”.

[00:04:38]The solutions to this equation will be our eigenvalues.

[00:04:42]Let’s try a concrete example so that this will make more sense.

[00:04:50]Consider matrix A, which equals 1, 1; 4, 1.

[00:04:55]We will take this matrix and subtract λI, where I is the two by two matrix with ones on the main diagonal and zeros for the other two entries.

[00:05:07]Multiplying by lambda, we simply end up with lambdas on the main diagonal, and so in performing the subtraction, we will simply end up subtracting lambda from the main diagonal of A. That gives us 1 - λ, 1, 4, 1 - λ.

[00:05:28]Now we must take the determinant of this matrix and set it equal to zero.

[00:05:33]The determinant will be (1 – lambda times 1 – lambda) - the quantity (4 times 1).

[00:05:41]Foiling out this first term leaves us with 1 - 2λ + λ2 - 4.

[00:05:48]If we simplify and then set this equal to zero, that will give us a characteristic equation of λ2 - 2λ – 3 = 0.

[00:05:58]If we recall our basic algebra, we can easily factor this, rewriting it as (λ - 3)(λ + 1) = 0.

[00:06:08]This means that we have two solutions, 3 and -1.

[00:06:12]These are the eigenvalues.

[00:06:15]So to summarize what we just did, we found the determinant of A – λI and set it equal to zero, which was our characteristic equation, and then solving that equation gave us a pair of eigenvalues.

[00:06:31]Once we have found the eigenvalues for a matrix, we can start solving for the eigenvectors.

[00:06:38]This takes a bit more effort, and it must be done for each eigenvalue separately.

[00:06:45]We will once again start the process by examining the A - λI part of (A - λI) times x = the zero vector.

[00:06:54]This time we will actually plug in one of the eigenvalues we found for lambda and get a new matrix.

[00:07:01]This new matrix, multiplied by our unknown vector x, equals zero, so in essence what we are left with is a system of equations that we can solve by any method we’ve learned so far.

[00:07:13]The most consistent method here is to use row operations to get the matrix into row echelon form.

[00:07:21]There will be times when we must “choose” values for the components of the eigenvectors, but it doesn’t really matter what we choose, because the solution we find by doing so only represents the form of eigenvectors.

[00:07:37]Any scalar multiple of the vector we find through this method will also be an eigenvector.

[00:07:44]We can easily demonstrate this as follows.

[00:07:47]Assume the vector x is an eigenvector of A, so Ax = λx.

[00:07:53]Now if we consider the vector cx, where c is a scalar, then A times this new vector can also be written as c times Ax since c is just a scalar.

[00:08:05]We already know Ax = λx so this becomes cλx which can be rewritten as λ(cx).

[00:08:15]So we get A(cx) = λ(cx) which makes the vector cx an eigenvector, by definition.

[00:08:25]To make this process of solving for eigenvectors a little more clear, let’s once again consider the matrix A equals 1, 1; 4, 1.

[00:08:35]We have already found the eigenvalues of this matrix to be 3 and -1, so let’s use these to find the related eigenvectors.

[00:08:44]We can start with λ = 3 and plug this into A - λI.

[00:08:49]We end up getting threes on this main diagonal, which means we will be subtracting 3 from the diagonal elements of A, leaving us with -2, 1, 4, -2.

[00:09:02]Now once we have this, we will have to use row operations to get this in row echelon form.

[00:09:08]In this case, we can simply add two times the first row to the second row, to get a new second row.

[00:09:15]This turns the second row into zeros, and we are left with -2 and 1 in the first row.

[00:09:22]Recall that we are solving the matrix equation (A - λI)x = 0, so this matrix multiplied by x equals zero.

[00:09:31]Writing x as (x1, x2) and doing matrix multiplication, we end up with the equation -2x1 + x2 = 0.

[00:09:41]Moving 2x1 to the other side, that leaves us with x2 = 2x1.

[00:09:47]Let’s now “choose” x1 to be equal to 1, which makes x2 equal to 2.

[00:09:55]So for the eigenvalue λ = 3, we get the eigenvector x = (1, 2).

[00:10:01]However, recall that this vector only represents the form eigenvectors take for this eigenvalue.

[00:10:07]We can have any multiple of this vector, and it will still be an eigenvector.

[00:10:12]To put it simply, any vector where the second element is twice the first, will be an eigenvector.

[00:10:20]Now let’s do the same thing for λ = -1.

[00:10:23]We once again go back to A - λI, this time plugging in -1 for lambda.

[00:10:29]In this case, by subtracting negative 1 from the main diagonal of A, we find the result to be 2, 1, 4, 2.

[00:10:38]For our row operation this time, we will subtract two times the first row from the second row to get a new second row.

[00:10:46]We are left with 2 and 1 in the first row, which this time makes our equation 2x1 + x2 = 0.

[00:10:54]Moving x1 to the other side we are left with x2 = -2x1.

[00:11:00]Keeping it simple, let’s again choose 1 for x1, and we get x2 = -2, making our vector x = (1, -2).

[00:11:09]Again, any multiple of this vector will also qualify.

[00:11:14]So the eigenvectors for λ = -1 are vectors with their second element being equal to -2 times the first element.

[00:11:24]Now that we’ve gotten our feet wet, let’s go through one more example to really make sure we understand this process.

[00:11:30]This time let’s take our matrix A to be 1, 0, 0; 3, -2, 0; 2, 3, 4 and for this we will find the eigenvalues and eigenvectors.

[00:11:41]First, to find the eigenvalues, let’s find A - λI, which will mean subtracting lambda from the terms in the main diagonal of A. Then we will take the determinant of this new matrix, and set the polynomial equal to zero.

[00:11:58]Thanks to the form of our matrix, with two zeros in the top row, the determinant of A - λI is relatively simple.

[00:12:06]We get (1 - λ) times [(-2 - λ)(4 - λ) – (0 times 3)], then minus zero, and then plus zero.

[00:12:19]This simplifies to the product of these three binomials, and setting this equal to zero, we end up with a characteristic equation that is already completely factored.

[00:12:30](1 - λ)(-2 - λ)(4 - λ) = 0.

[00:12:37]So our solutions are λ = 1, λ = -2, and λ = 4, and these are the eigenvalues for this matrix.

[00:12:48]Now that we have the eigenvalues, we can solve for the eigenvectors that are related to each of them.

[00:12:54]Let’s start with λ = 1.

[00:12:57]We will go back to our matrix given by A – λI, and plugging in λ = 1, we will subtract 1 from the terms in the main diagonal to get the matrix 0, 0, 0; 3, -3, 0; 2, 3, 3.

[00:13:14]We could go through row operations from here, but the form of our matrix in this case makes it simple to solve if we jump right into equation form.

[00:13:22]The top row gives no information, but the second row tells us 3x1 - 3x2 = 0.

[00:13:30]Solving this we see that x1 = x2, and for this case let’s choose x2 to be 1.

[00:13:36]So both x1 and x2 are therefore equal to one.

[00:13:41]The final row tells us that 2x1 + 3x2 + 3x3 = 0, but we’ve already chosen x1 and x2 to be equal to 1, so plugging those values in, we get 2 + 3 + 3x3 = 0.

[00:13:57]We end up getting x3 = -5/3, making our eigenvectors any multiple of (1, 1, -5/3).

[00:14:08]Next we will find the eigenvectors for λ = -2.

[00:14:12]Plugging this value into A – λI, we subtract -2 from the main diagonal to get 3, 0, 0; 3, 0, 0; 2, 3, 6.

[00:14:24]Expressing this in equation form, the first row tells us that 3x1 = 0, so x1 must be zero.

[00:14:31]The second row tells us the same thing.

[00:14:34]Meanwhile, the final row becomes 2x1 + 3x2 + 6x3 = 0.

[00:14:41]We already know that x1 is 0, so we are left with 3x2 + 6x3 = 0, or x2 = -2x3.

[00:14:52]We will now choose x3 to be 1, making x2 = -2.

[00:14:57]So the eigenvectors have the form (0, -2, 1).

[00:15:04]And finally let’s do the third eigenvalue λ = 4.

[00:15:08]The matrix A - λI requires subtracting 4 from the main diagonal, so this becomes -3, 0, 0; 3, -6, 0; 2, 3, 0.

[00:15:21]The first row tells us that -3x1 = 0, so once again x1 is zero.

[00:15:27]The second row tells us 3x1 - 6x2 = 0, but since x1 is 0, we are left with -6x2 = 0.

[00:15:37]So x2 must also be zero.

[00:15:39]The third row becomes 2x1 - 3x2 = 0.

[00:15:44]We already know that both x1 and x2 are zero, but we have no information about x3.

[00:15:50]This makes x3 a free variable.

[00:15:53]In this case we can just choose 1, making our eigenvectors have the form (0, 0, 1).

[00:16:02]Now that we have sufficiently discussed the important process of finding eigenvalues and eigenvectors, let’s check comprehension.