Systems of Differential Equations (Part 3. Review : Matrices and Vectors)

Review : Matrices and Vectors

This section is intended to be a catch all for many of the basic concepts that are used occasionally in working with systems of differential equations.  There will not be a lot of details in this section, nor will we be working large numbers of examples.  Also, in many cases we will not be looking at the general case since we won’t need the general cases in our differential equations work.

Let’s start with some of the basic notation for matrices.  An n x m (this is often called the size or dimension of the matrix) matrix is a matrix with n rows and m columns and the entry in the i^th row and j^th column is denoted by a_ij.  A short hand method of writing a general n x m matrix is the following.

The size or dimension of a matrix is subscripted as shown if required.  If it’s not required or clear from the problem the subscripted size is often dropped from the matrix.

Special Matrices

There are a few “special” matrices out there that we may use on occasion.  The first special matrix is the square matrix.  A square matrix is any matrix whose size (or dimension) is n x n.  In other words it has the same number of rows as columns.  In a square matrix the diagonal that starts in the upper left and ends in the lower right is often called the main diagonal.

The next two special matrices that we want to look at are the zero matrix and the identity matrix.  The zero matrix, denoted 0_{n x m} , is a matrix all of whose entries are zeroes.  The identity matrix is a square n x n matrix, denoted I_n, whose main diagonals are all 1’s and all the other elements are zero.  Here are the general zero and identity matrices.

In matrix arithmetic these two matrices will act in matrix work like zero and one act in the real number system.

The last two special matrices that we’ll look at here are the column matrix and the row matrix.  These are matrices that consist of a single column or a single row.  In general they are,

We will often refer to these as vectors.

Arithmetic

We next need to take a look at arithmetic involving matrices.  We’ll start with addition and subtraction of two matrices.  So, suppose that we have two n x m matrices, A and B.  The sum (or difference) of these two matrices is then,

The sum or difference of two matrices of the same size is a new matrix of identical size whose entries are the sum or difference of the corresponding entries from the original two matrices.  Note that we can’t add or subtract entries with different sizes.

Next, let’s look at scalar multiplication.  In scalar multiplication we are going to multiply a matrix A by a constant (sometimes called a scalar) α.  In this case we get a new matrix whose entries have all been multiplied by the constant, α.

Example 1  Given the following two matrices,                                      compute A-5B. Solution There isn’t much to do here other than the work.                                              We first multiplied all the entries of B by 5 then subtracted corresponding entries to get the entries in the new matrix.

The final matrix operation that we’ll take a look at is matrix multiplication.  Here we will start with two matrices, A_{n x p} and B_{p x m} .  Note that A must have the same number of columns as B has rows.  If this isn’t true then we can’t perform the multiplication.  If it is true then we can perform the following multiplication.

The new matrix will have size n x m and the entry in the i^th row and j^th column, c_ij,  is found by multiplying row i of matrix A by column j of matrix B.  This doesn’t always make sense in words so let’s look at an example.

Example 2  Given                            compute AB. Solution The new matrix will have size 2 x 4.  The entry in row 1 and column 1 of the new matrix will be found by multiplying row 1 of A by column 1 of B.  This means that we multiply corresponding entries from the row of A and the column of B and then add the results up.  Here are a couple of the entries computed all the way out.                                            Here’s the complete solution.

In this last example notice that we could not have done the product BA since the number of columns of B does not match the number of row of A.  It is important to note that just because we can compute AB doesn’t mean that we can compute BA.  Likewise, even if we can compute both AB and BA they may or may not be the same matrix.

Determinant

The next topic that we need to take a look at is the determinant of a matrix.  The determinant is actually a function that takes a square matrix and converts it into a number.  The actual formula for the function is somewhat complex and definitely beyond the scope of this review. 

The main method for computing determinants of any square matrix is called the method of cofactors.  Since we are going to be dealing almost exclusively with 2 x 2 matrices and the occasional 3 x 3 matrix we won’t go into the method here.  We can give simple formulas for each of these cases.  The standard notation for the determinant of the matrix A is.

Here are the formulas for the determinant of 2 x 2 and 3 x 3 matrices.

Example 3  Find the determinant of each of the following matrices.                            Solution For the 2 x 2 there isn’t much to do other than to plug it into the formula.                                     For the 3 x 3 we could plug it into the formula, however unlike the 2 x 2 case this is not an easy formula to remember.  There is an easier way to get the same result.  A quicker way of getting the same result is to do the following.  First write down the matrix and tack a copy of the first two columns onto the end as follows.                                                Now, notice that there are three diagonals that run from left to right and three diagonals that run from right to left.  What we do is multiply the entries on each diagonal up and the if the diagonal runs from left to right we add them up and if the diagonal runs from right to left we subtract them. Here is the work for this matrix.

You can either use the formula or the short cut to get the determinant of a 3 x 3. 

If the determinant of a matrix is zero we call that matrix singular and if the determinant of a matrix isn’t zero we call the matrix nonsingular.  The 2 x 2 matrix in the above example was singular while the 3 x 3 matrix is nonsingular.

Matrix Inverse

Next we need to take a look at the inverse of a matrix.  Given a square matrix, A, of size n x n if we can find another matrix of the same size, B such that,

then we call B the inverse of A and denote it by B=A^-1.

Computing the inverse of a matrix, A, is fairly simple.  First we form a new matrix,

and then use the row operations from the previous section and try to convert this matrix into the form,

If we can then B is the inverse of A.  If we can’t then there is no inverse of the matrix A.

Example 4  Find the inverse of the following matrix, if it exists.                                                          Solution We first form the new matrix by tacking on the 3 x 3 identity matrix to this matrix.  This is                                                     We will now use row operations to try and convert the first three columns to the 3 x 3 identity.  In other words we want a 1 on the diagonal that starts at the upper left corner and zeroes in all the other entries in the first three columns. If you think about it, this process is very similar to the process we used in the last section to solve systems, it just goes a little farther.  Here is the work for this problem.                                                                                               So, we were able to convert the first three columns into the 3 x 3 identity matrix therefore the inverse exists and it is,

So, there was an example in which the inverse did exist.  Let’s take a look at an example in which the inverse doesn’t exist.

Example 5  Find the inverse of the following matrix, provided it exists.                                                              Solution In this case we will tack on the 2 x 2 identity to get the new matrix and then try to convert the first two columns to the 2 x 2 identity matrix.                                      And we don’t need to go any farther.  In order for the 2 x 2 identity to be in the first two columns we must have a 1 in the second entry of the second column and a 0 in the second entry of the first column.  However, there is no way to get a 1 in the second entry of the second column that will keep a 0 in the second entry in the first column.  Therefore, we can’t get the 2 x 2 identity in the first two columns and hence the inverse of B doesn’t exist.

We will leave off this discussion of inverses with the following fact.

Fact

Given a square matrix A. 1.      If A is nonsingular then A-1 will exist. 2.      If A is singular then A-1 will NOT exist.

I’ll leave it to you to verify this fact for the previous two examples.

Systems of Equations Revisited

We need to do a quick revisit of systems of equations.  Let’s start with a general system of equations.

(1)

Now, covert each side into a vector to get,

The left side of this equation can be thought of as a matrix multiplication.

Simplifying up the notation a little gives,

(2)

where,  is a vector whose components are the unknowns in the original system of equations.  We call (2) the matrix form of the system of equations (1) and solving (2) is equivalent to solving (1).  The solving process is identical.  The augmented matrix for (2) is

Once we have the augmented matrix we proceed as we did with a system that hasn’t been wrote in matrix form.

We also have the following fact about solutions to (2).

Fact

Given the system of equation (2) we have one of the following three possibilities for solutions. 1.      There will be no solutions. 2.      There will be exactly one solution. 3.      There will be infinitely many solutions.

In fact we can go a little farther now.  Since we are assuming that we’ve got the same number of equations as unknowns the matrix A in (2) is a square matrix and so we can compute its determinant.  This gives the following fact.

Fact

Given the system of equations in (2) we have the following. 1.      If A is nonsingular then there will be exactly one solution to the system. 2.      If A is singular then there will either be no solution or infinitely many solutions to the system.

The matrix form of a homogeneous system is

(3)

where  is the vector of all zeroes.  In the homogeneous system we are guaranteed to have a solution, .  The fact above for homogeneous systems is then,

Fact

Given the homogeneous system (3) we have the following. 1.      If A is nonsingular then the only solution will be . 2.      If A is singular then there will be infinitely many nonzero solutions to the system.

Linear Independence/Linear Dependence

This is not the first time that we’ve seen this topic.  We also saw linear independence and linear dependence back when we were looking at second order differential equations.  In that section we were dealing with functions, but the concept is essentially the same here.  If we start with n vectors,

If we can find constants, c₁,c₂,…,c_n with at least two nonzero such that

(4)

then we call the vectors linearly dependent.  If the only constants that work in (4) are c₁=0, c₂=0, …, c_n=0 then we call the vectors linearly independent.

If we further make the assumption that each of the n vectors has n components, i.e. each of the vectors look like,

we can get a very simple test for linear independence and linear dependence.  Note that this does not have to be the case, but in all of our work we will be working with n vectors each of which has n components.

Fact

Given the n vectors each with n components,                                                   form the matrix,                                           So, the matrix X is a matrix whose ith column is the ith vector, .  Then, If X is nonsingular (i.e. det(X) is not zero) then the n vectors are linearly independent, and if X is singular (i.e. det(X) = 0) then the n vectors are linearly dependent and the constants that make (4) true can be found by solving the system             where  is a vector containing the constants in (4).

Example 6  Determine if the following set of vectors are linearly independent or linearly dependent.  If they are linearly dependent find the relationship between them.                                          Solution So, the first thing to do is to form X and compute its determinant.                                This matrix is non singular and so the vectors are linearly independent.

Example 7  Determine if the following set of vectors are linearly independent or linearly dependent.  If they are linearly dependent find the relationship between them.                                          Solution As with the last example first form X and compute its determinant.                            So, these vectors are linearly dependent.  We now need to find the relationship between the vectors.  This means that we need to find constants that will make (4) true. So we need to solve the system                                                                     Here is the augmented matrix and the solution work for this system. Now, we would like actual values for the constants so, if use  we get the following solution , , and .  The relationship is then.

Calculus with Matrices

There really isn’t a whole lot to this other than to just make sure that we can deal with calculus with matrices.

First, to this point we’ve only looked at matrices with numbers as entries, but the entries in a matrix can be functions as well.  So we can look at matrices in the following form,

Now we can talk about differentiating and integrating a matrix of this form.  To differentiate or integrate a matrix of this form all we do is differentiate or integrate the individual entries.

So when we run across this kind of thing don’t get excited about it.  Just differentiate or integrate as we normally would.

In this section we saw a very condensed set of topics from linear algebra.  When we get back to differential equations many of these topics will show up occasionally and you will at least need to know what the words mean.

The main topic from linear algebra that you must know however if you are going to be able to solve systems of differential equations is the topic of the next section.