## Determinants as Area and Volume

The Area of a Parallelogram

Let $v=(v_1,v_2)$ and $w=(w_1,w_2)$ be two linearly independent vectors in $\mathbb{R}^2$. Then they span a parallelogram as shown in Figure 1.

Figure 1. Parallelogram spanned by two linearly independent vectors

The area $A$ of the parallelogram is
\begin{align*}
A&=||v||||w||\sin\theta\\
&=||v\times w||\\
&=\left|\begin{array}{cc}
v_1 & v_2\\
w_1 & w_2
\end{array}\right|.
\end{align*}
In general, the resulting determinant is not necessarily positive. If it is negative, we need to take the absolute value of the determinant for the area.

Exercise. Given two vectors $v=(v_1,v_2,v_3)$ and $w=(w_1,w_2,w_3)$ in $\mathbb{R}^3$, show that $||v||||w||sin\theta=||v\times w||$ where $\theta$ is the angle between $v$ and $w$. (Hint. Note that $||v||^2||w||^2\sin^2\theta=||v||^2||w||^2-(v\cdot w)^2$.)

The Volume of a Parallelepiped

Let $u=(u_1,u_2,u_3)$, $v=(v_1,v_2,v_3)$, $w=(w_1,w_2,w_3)$ be three linearly independent vectors in $\mathbb{R}^3$. Then they span a parallelepiped as shown in Figure 2.

Figure 2. Parallelepiped spanned by three linearly independent vectors

The volume $V$ of the parallelepiped is
\begin{align*}
V&=||u||\cos\theta ||v\times w||\\
&=u\cdot (v\times w)\\
&=\left|\begin{array}{ccc}
u_1 & u_2 & u_3\\
v_1 & v_2 & v_3\\
w_1 & w_2 & w_3
\end{array}\right|.
\end{align*}
In general, the resulting determinant is not necessarily positive. If it is negative, we need to take the absolute value of the determinant for the volume.

## Inverse of a Matrix

Let $A$ be an $n\times n$ matrix with $\det A\ne 0$. (A square matrix whose determinant is not equal to $0$ is called non-singular.) Let $X=(x_{ij})$ be an unknown $n\times n$ matrix such that
$AX=I$. Then
$$x_{1j}A^1+\cdots+x_{nj}A^n=E^j.$$
This is a system of linear equations and as we studied here, it can be solved by Cramer’s Rule as
\begin{align*}
x_{ij}&=\frac{\det(A^1,\cdots,E^j,\cdots A^n)}{\det A}\\
&=\frac{1}{\det A}\left|\begin{array}{ccccc}
a_{11} & \cdots & 0 & \cdots & a_{1n}\\
\vdots &        & \vdots &   & \vdots\\
a_{j1} & \cdots & 1 & \cdots & a_{jn}\\
\vdots &        & \vdots &   & \vdots\\
a_{n1} & \cdots & 0 & \cdots & a_{nn}
\end{array}\right|\\
&=\frac{1}{\det A}(-1)^{i+j}\det(A_{ji})
\end{align*}
for $i=1,\cdots,n$. If we show that $XA=I$ as well, then $X$ would be the inverse $A^{-1}$ and
$$A^{-1}=\frac{1}{\det A}{}^t((-1)^{i+j}\det(A_{ij})).$$
$\det({}^tA)=\det A\ne 0$, so we can find an $n\times n$ matrix $Y$ such that ${}^tAY=I$. Taking transposes, we obtain ${}^tYA=I$. Now,
$$I={}^tYA={}^tYIA={}^tY(AX)A={}^tYA(XA)=XA.$$

Example. Let $A=\begin{pmatrix} a & b\\ c & d \end{pmatrix}$ with $\det A=ad-bc\ne 0$. Then $A^{-1}$ is given by
$$A^{-1}=\frac{1}{ad-bc}\begin{pmatrix} d & -b\\ -c & a \end{pmatrix}.$$

Example. Find the inverse of the matrix
$$A=\begin{pmatrix} 3 & 1 & -2\\ -1 & 1 & 2\\ 1 & -2 & 1 \end{pmatrix}.$$

Solution. $\det A=16$. We find
\begin{align*}
A_{11}=\begin{pmatrix}
1 & 2\\
-2 & 1
\end{pmatrix},\ A_{12}=\begin{pmatrix}
-1 & 2\\
1 & 1
\end{pmatrix},\ A_{13}=\begin{pmatrix}
-1 & 1\\
1 & -2
\end{pmatrix},\\
A_{21}=\begin{pmatrix}
1 & -2\\
-2 & 1
\end{pmatrix},\ A_{22}=\begin{pmatrix}
3 & -2\\
1 & 1
\end{pmatrix},\ A_{23}=\begin{pmatrix}
3 & 1\\
1 & -2
\end{pmatrix},\\
A_{31}=\begin{pmatrix}
1 & -2\\
1 & 2
\end{pmatrix},\ A_{32}=\begin{pmatrix}
3 & -2\\
-1 & 2
\end{pmatrix},\ A_{33}=\begin{pmatrix}
3 & 1\\
-1 & 1
\end{pmatrix}.
\end{align*}

Hence by the formula we obtain
\begin{align*}
A^{-1}&=\frac{1}{16}{}^t\begin{pmatrix}
\det(A_{11}) & -\det(A_{12}) & \det(A_{13})\\
-\det(A_{21}) & \det(A_{22}) & -\det(A_{23})\\
\det(A_{31}) &-\det(A_{32}) & \det(A_{33})
\end{pmatrix}\\
&=\frac{1}{16}{}^t\begin{pmatrix}
5 & 3 & 1\\
3 & 5 & 7\\
4 & -4 & 4
\end{pmatrix}\\
&=\frac{1}{16}\begin{pmatrix}
5 & 3 & 4\\
3 & 5 &-4\\
1 & 7 & 4
\end{pmatrix}.
\end{align*}

We introduce the following theorem without proof:

Theorem. For any two $n\times n$ matrices $A$, $B$,
$$\det(AB)=\det A\det B.$$

As a special case, we obtain:

Corollary. For an invertible matrix $A$,
$$\det(A^{-1})=\frac{1}{\det A}.$$

Proof. $AA^{-1}=I$, so by the theorem
$$1=\det(AA^{-1})=\det A\det (A^{-1}).$$
Thus proving the formula for the inverse.

## Cramer’s Rule

Consider a system of $n$ linear equations in $n$ unknowns
$$x_1A^1+\cdots+x_nA^n=B,$$
where $x_1,\cdots,x_n$ are variables and $A^1,\cdots,A^n,B$ are column vectors of dimension $n$. Suppose that $\det(A^1,\cdots,A^n)\ne 0$. Recall that this condition ensures that the linear system has a unique solution as seen here. Let us consider the determinant of the matrix obtained by replacing $j$-th column $A^j$ by $B$. Then using the properties of determinants studied here
\begin{align*}
\det (A^1,\cdots,B,\cdots,A^n)&=\det (A^1,\cdots,x_1A^1+\cdots+x_nA^n,\cdots,A^n)\\
&=\det (A^1,\cdots,x_1A^1,\cdots,x^nA^n)+\cdots\\&+\det (A^1,\cdots,x_jA^j,\cdots,x_nA^n)+\cdots\\&+\det (A^1,\cdots,x_nA^n,\cdots,x_nA^n)\\
&=x_1\det (A^1,\cdots,A^1,\cdots,A^n)+\cdots\\&+x_j\det (A^1,\cdots,A^j,\cdots,A^n)+\cdots\\&+x_n\det (A^1,\cdots,A^n,\cdots,A^n).
\end{align*}
Two column vectors are the same in every term except the $j$-th term, and so every term except the $j$-th term. Hence, we have
$$\det (A^1,\cdots,B,\cdots,A^n)=x_j\det (A^1,\cdots,A^j,\cdots,A^n),$$
i.e.
\begin{align*}
x_j&=\frac{\det (A^1,\cdots,B,\cdots,A^n)}{\det (A^1,\cdots,A^j,\cdots,A^n)}\\
&=\frac{\left|\begin{array}{ccccc}
a_{11} & \cdots & b_1 & \cdots & a_{1n}\\
a_{21} & \cdots & b_2 & \cdots & a_{2n}\\
\vdots & &\vdots& &\vdots\\
a_{n1} & \cdots & b_n & \cdots & a_{nn}
\end{array}\right|}{\left|\begin{array}{ccccc}
a_{11} & \cdots & a_{1j} & \cdots & a_{1n}\\
a_{21} & \cdots & a_{2j} & \cdots & a_{2n}\\
\vdots & &\vdots& &\vdots\\
a_{n1} & \cdots & a_{nj} & \cdots & a_{nn}
\end{array}\right|}
\end{align*}
for $j=1,\cdots,n$. This is called the Cramer’s Rule.

Example. Solve the system of Linear equations:
\begin{align*}
3x+2y+4z&=1,\\
2x-y+z&=0,\\
x+2y+3z&=1.
\end{align*}

Solution. By the Caramer’s Rule, we have
$$x=\frac{\left|\begin{array}{ccc} 1 & 2 & 4\\ 0 & -1 & 1\\ 1 & 2 & 3 \end{array}\right|}{\left|\begin{array}{ccc} 3 & 2 & 4\\ 2 & -1 & 1\\ 1 & 2 & 3 \end{array}\right|}=-\frac{1}{5},\ y=\frac{\left|\begin{array}{ccc} 3 & 1 & 4\\ 2 & 0 & 1\\ 1 & 1 & 3 \end{array}\right|}{\left|\begin{array}{ccc} 3 & 2 & 4\\ 2 & -1 & 1\\ 1 & 2 & 3 \end{array}\right|}=0,\ z=\frac{\left|\begin{array}{ccc} 3 & 2 & 0\\ 2 & -1 & 1\\ 1 & 2 & 0 \end{array}\right|}{\left|\begin{array}{ccc} 3 & 2 & 4\\ 2 & -1 & 1\\ 1 & 2 & 3 \end{array}\right|}=\frac{2}{5}.$$

Posted in Linear Algebra | 1 Comment

## The Rank of a Matrix 2

A test for the linear dependence of vectors may be given in terms of determinant.

Theorem. Let $A^1,\cdots,A^n$ be column vectors of dimension $n$. They are linearly dependent if and only if
$$\det(A^1,\cdots,A^n)=0.$$

Corollary. If a system of $n$ linear equations in $n$ unknowns has a matrix of coefficients whose determinants is not 0, then this system has a unique solution.

Proof. A system of $n$ linear equations in $n$ unknowns may be written as
$$x_1A^1+\cdots+x_nA^n=B,$$
where $A^1,\cdots,A^n$ are the column vectors of dimension $n$ of the matrix of coefficients and $B$ is a column vector of dimension $n$. Since $\det(A^1,\cdots,A^n)\ne 0$, $A^1,\cdots,A^n$ are linearly independent by the theorem. So there exists a unique solution $x_1,\cdots,x_n$ of the system.

Since determinants can be used to test linear dependence , they can be also used to determine the rank of a matrix in stead of using row operations as seen here.

Example. Let
$$A=\begin{pmatrix} 3 & 1 & 2 & 5\\ 1 & 2 & -1 & 2\\ 1 & 1 & 0 & 1 \end{pmatrix}.$$
Since $A$ is a $3\times 4$ matrix, its rank is at most 3. If we can find three linearly independent column vectors, the rank is 3. In fact,
$$\left|\begin{array}{ccc} 1 & 2 & 5\\ 2 & -1 & 2\\ 1 & 0 & 1 \end{array}\right|=4.$$
So, the rank is exactly 3.

Example. Let
$$B=\begin{pmatrix} 3 & 1 & 2 & 5\\ 1 & 2 & -1 & 2\\ 4 & 3 & 1 & 7 \end{pmatrix}.$$
Every $3\times 3$ subdeterminant has value 0, so the rank of $B$ is at most 2. The first two rows of $B$. The first two rows are linearly independent since teh determinant
$$\left|\begin{array}{cc} 3 & 1\\ 1 & 2 \end{array}\right|$$
is not 0. Hence, the rank is 2.

Posted in Linear Algebra | 1 Comment

## Determinants II: Determinants of Order $n$

A determinant of order $n$ can be calculated by expanding it in terms of determinants of order $n-1$. Let $A=(a_{ij})$ be an $n\times n$ matrix and let us denote by $A_{ij}$ the $(n-1)\times (n-1)$ matrix obtained by deleting the $i$-th row and the $j$-th column from $A$:

Then $\det A$ is given by the Laplace expansion
\begin{align*}
\det A&=(-1)^{i+1}a_{i1}\det A_{i1}+\cdots+(-1)^{i+n}a_{in}\det A_{in}\\
&=(-1)^{1+j}a_{1j}\det A_{1j}+\cdots+(-1)^{n+j}a_{nj}\det A_{nj}.
\end{align*}

All the properties of the determinants of order 2 we studied here still hold in general for the determinants of order $n$. In particular,

Theorem. Let $A^1,\cdots,A^n$ be column vectors of dimension $n$. They are linearly dependent if and only if
$$\det(A^1,\cdots,A^n)=0.$$

Example. Let us calculate the determinant of the following $3\times 3$ matrix
$$A=\begin{pmatrix} a_{11} & a_{12} & a_{13}\\ a_{21} & a_{22} & a_{23}\\ a_{31} & a_{32} & a_{33} \end{pmatrix}.$$
You may use any column or row to calculate $\det A$ using the Laplace expansion. In this example, we use the first row to calculate $\det A$. By the Laplace expansion,
\begin{align*}
\det A&=a_{11}\det A_{11}-a_{12}\det A_{12}+a_{13}\det A_{13}\\
&=a_{11}\left|\begin{array}{cc}
a_{22} & a_{23}\\
a_{32} & a_{33}
\end{array}\right|-a_{12}\left|\begin{array}{cc}
a_{21} & a_{23}\\
a_{31} & a_{33}
\end{array}\right|+a_{13}\left|\begin{array}{cc}
a_{21} & a_{22}\\
a_{31} & a_{32}
\end{array}\right|.
\end{align*}
Replay this with
$$A=\begin{pmatrix} 2 & 1 & 0\\ 1 & 1 & 4\\ -3 & 2 & 5 \end{pmatrix}.$$
Since the first row or the third column contains 0, you may want to use the first row or the third column to do the Laplace expansion.

For $3\times 3$ matrices, there is a quicker way to calculate the determinant as shown in the following figure. You multiply three entries along each indicated arrow. When you multiply three entries along each red arrow, you also multiply by $-1$. This is called the Rule of Sarrus named after a French mathematician Pierre Frédéric Sarrus. Please be warned that the rule of Sarrus works only for $3\times 3$ matrices.

The Rule of Sarrus

Example. [Cross Product] Let $v=v_1E_1+v_2E_2+v_3E_3$ and $w=w_1E_1+w_2E_2+w_3E_3$ be two vectors in Euclidean 3-space $\mathbb{R}^3$. The cross product is defined by
$$v\times w=\left|\begin{array}{ccc} E_1 & E_2 & E_3\\ v_1 & v_2 & v_3\\ w_1 & w_2 & w_3 \end{array}\right|.$$
Note that the cross product is perpendicular to both $v$ and $w$.

Clearly, if there are many 0 entries in a given determinant, it would be easier to calculate the determinant since you will have a lesser than usual number of terms that you actually have to calculate in the Laplace expansion. For any given determinant, we can indeed make it happen. Recall the theorem we studied here:

Theorem. If one adds a scalar multiple of one column (row) to another column (row), then the value of the determinant does not change.

Using the particular column (row) operation in the Theorem, we can turn a given determinant into one with more 0 entries.

Example. Find
$$\left|\begin{array}{cccc} 1 & 3 & 1 & 1\\ 2 & 1 & 5 & 2\\ 1 & -1 & 2 & 3\\ 4 & 1 & -3 & 7 \end{array}\right|.$$

Solution. By the above Theorem,
\begin{align*}
\left|\begin{array}{cccc}
1 & 3 & 1 & 1\\
2 & 1 & 5 & 2\\
1 & -1 & 2 & 3\\
4 & 1 & -3 & 7
\end{array}\right|&=\left|\begin{array}{cccc}
0 & 4 & -1 & -2\\
2 & 1 & 5 & 2\\
1 & -1 & 2 & 3\\
4 & 1 & -3 & 7
\end{array}\right|\ (\mbox{add $-1$ times row 3 to row 1})\\
&=\left|\begin{array}{cccc}
0 & 4 & -1 & -2\\
0 & 3 & 1 & -4\\
1 & -1 & 2 & 3\\
4 & 1 & -3 & 7
\end{array}\right|\ (\mbox{add $-2$ times row 3 to row 2})\\
&=\left|\begin{array}{cccc}
0 & 4 & -1 & -2\\
0 & 3 & 1 & -4\\
1 & -1 & 2 & 3\\
0 & 5 & -11 & -5
\end{array}\right|\ (\mbox{add $-4$ times row 3 to row 4})\\
&=\left|\begin{array}{ccc}
4 & -1 & -2\\
3 & 1 & -4\\
5 & -11 & -5
\end{array}\right|\ (\mbox{Laplace expansion along column 1})
\end{align*}
You may compute the resulting determinant of order 3 using the rule of Sarrus or you may further simpify it. For instance, you may do:
\begin{align*}
\left|\begin{array}{ccc}
4 & -1 & -2\\
3 & 1 & -4\\
5 & -11 & -5
\end{array}\right|&=\left|\begin{array}{ccc}
7 & 0 & -6\\
3 & 1 & -4\\
5 & -11 & -5
\end{array}\right|\ (\mbox{add row 2 to row 1})\\
&=\left|\begin{array}{ccc}
7 & 0 & -6\\
3 & 1 & -4\\
38 & 0 & -49
\end{array}\right|\ (\mbox{add 11 times row 2 to row 3})\\
&=\left|\begin{array}{cc}
7 & -6\\
38 & -49
\end{array}\right|\ (\mbox{Laplace expansion along column 2})\\
&=-115.
\end{align*}

## Determinants I: Determinants of Order 2

Let $A=\begin{pmatrix} a & b\\ c & d \end{pmatrix}$. Then we define the determinant $\det A$ by
$$\det A=ad-bc.$$
$\det A$ is also denoted by $|A|$ or $\left|\begin{array}{ccc} a & b\\ c & d \end{array}\right|$. In terms of the column vectors $A^1,A^2$, the determinant of $A$ may also be written as $\det(A^1,A^2)$.

Example. If $A=\begin{pmatrix} 2 & 1\\ 1 & 4 \end{pmatrix}$, then $\det A=8-1=7$.

Property 1. The determinant $\det (A^1,A^2)$ may be considered as a bilinear map of the column vectors. As a bilinear map $\det (A^1,A^2)$ is linear in each slot. For example, if $A^1=C+C’$ then
$$\det(A^1,A^2)=\det(C,A^2)+\det(C’,A^2).$$
If $x$ is a number,
$$\det(xA^1,A^2)=x\det(A^1,A^2).$$

Property 2. If the two columns are equal, the determinant is 0.

Property 3. $\det I=\det (E^1,E^2)=1$.

Combining Properties 1-3, we can show that:

Theorem. If one adds a scalar multiple of one column to another, then the value of the determinant does not change.

Proof. We prove the theorem for a particular case.
\begin{align*}
\det(A^1+xA^2,A^2)&=\det(A^1,A^2)+x\det(A^2,A^2)\\
&=\det(A^1,A^2).
\end{align*}

Theorem. If the two columns are interchanged, the determinant changes by a sign i.e.
$$\det(A^2,A^1)=-\det(A^1,A^2).$$

Proof. \begin{align*}
0&=\det(A^1+A^2,A^1+A^2)\\
&=\det(A^1,A^2)+\det(A^2,A^1).
\end{align*}

Theorem. $\det A=\det {}^tA$.

Proof. This theorem can be proved directly from the definition of $\det A$.

Remark. Because of this theorem, we can also say that if one adds a scalar multiple of one row to another row, then the value of the determinant does not change.

Theorem. The column vectors $A^1,A^2$ are linearly dependent if and only if $\det(A^1,A^2)=0$.

Proof. Suppose that $A^1,A^2$ are linearly dependent. Then there exists numbers $x,y$, not all equal to 0 such that $xA^1+yA^2=0$. Let us say $x\ne 0$. Then $A^1=-\frac{y}{x}A^2$. So,
\begin{align*}
\det(A^1,A^2)&=\det\left(-\frac{y}{x}A^2,A^2\right)\\
&=-\frac{y}{x}\det(A^2,A^2)\\
&=0.
\end{align*}
To prove the converse, suppose that $A^1,A^2$ are linearly independent. Then $E^1,E^2$ can be written as linear combinations of $A^1,A^2$:
\begin{align*}
E^1&=xA^1+yA^2,\\
E^2&=zA^1+wA^2.
\end{align*}
Now,
\begin{align*}
1&=\det(E^1,E^1)\\
&=xw\det(A^1,A^2)+yz\det(A^2,A^1)\\
&=(xw-yz)\det(A^1,A^2).
\end{align*}
Hence, $\det(A^1,A^2)\ne 0$.

Posted in Linear Algebra | 2 Comments

## Orthogonal Bases

Let $V$ be a vector space with a positive definite scalar product $\langle\ ,\ \rangle$. A basis $\{v_1,\cdots,v_n\}$ of $V$ is said to be orthogonal if $\langle v_i,v_j\rangle=0$ if $i\ne j$. In addition, if $||v_i||=1$ for all $i=1,\cdots,n$, then the basis is said to be orthonormal.

Example. $E_1,\cdots,E_n$ of $\mathbb{R}^n$ form an orthonormal basis of $\mathbb{R}^n$.

Why having a orthonormal basis is big deal? To answer this question, let us suppose that $e_1,\cdots,e_n$ is an orthonormal basis of a vector space $V$. Let $v,w\in V$. Then
\begin{align*}
v&=v_1e_1+\cdots+v_ne_n,\\
w&=w_1e_1+\cdots+w_ne_n.
\end{align*}
Since $\langle e_i,e_j\rangle=\delta_{ij}$,
$$\langle v,w\rangle=v_1w_1+\cdots+v_nw_n=v\cdot w.$$
Hence, once an orthonormal basis is given, the scalar product $\langle\ ,\ \rangle$ is identified with the dot product. Next question is then, can we always come up with an orthonormal basis? The answer is affirmative. Given a basis, we can construct a new basis which is orthonormal through a process called the Gram-Schmidt orthogonalization process. Here is how it works. Let $w_1,\cdots,w_n$ be a basis of a vector space $V$. Let $v_1=w_1$ and
$$v_2=w_2-\frac{\langle w_2,v_1\rangle}{\langle v_1,v_1\rangle}v_1.$$
Then $v_2$ is perpendicular to $v_1$. Note that if $w_2$ is already perpendicular to $v_1=w_1$, then $v_2=w_2$.

Gram-Schmidt Orthogonalization Process

Let
$$v_3=w_3-\frac{\langle w_3,v_1\rangle}{\langle v_1,v_1\rangle}v_1-\frac{\langle w_3,v_2\rangle}{\langle v_2,v_2\rangle}v_2.$$
Then $v_3$ is perpendicular both $v_1$ and $v_2$ as seen in the following figure.
Continuing this process, we have
$$v_n=w_n-\frac{\langle w_n,v_1\rangle}{\langle v_1,v_1\rangle}v_1-\cdots-\frac{\langle w_n,v_{n-1}\rangle}{\langle v_{n-1},v_{n-1}\rangle}v_{n-1}$$
and $v_1,\cdots,v_n$ are mutually perpendicular.

Gram-Schmidt Orthogonalization Process

Therefore, we have the following theorem holds.

Theorem. Let $V\ne\{O\}$ be a finite dimensional vector space with a positive definite scalar product. Then $V$ has an orthonormal basis.

Example. Find an orthonormal basis for the vector space generated by
$$A=(1,1,0,1),\ B=(1,-2,0,0),\ C=(1,0,-1,2).$$
Here the scalar product is the dot product.

Solution. Let
\begin{align*}
A’&=A,\\
B’&=B-\frac{B\cdot A’}{A’\cdot A’}A’\\
&=\frac{1}{3}(4,-5,0,1),\\
C’&=C-\frac{C\cdot A’}{A’\cdot A’}-\frac{C\cdot B’}{B’\cdot B’}B’\\
&=\frac{1}{7}(-4,-2,-7,6).
\end{align*}
Then $A’,B’,C’$ is an orthogonal basis. We obtain an orthonormal basis by normilizing each basis member:
\begin{align*}
\frac{A’}{||A’||}&=\frac{1}{\sqrt{3}}(1,1,0,1),\\
\frac{B’}{||B’||}&=\frac{1}{\sqrt{42}}(4,-5,0,1),\\
\frac{C’}{||C’||}&=\frac{1}{\sqrt{105}}(-4,-2,-7,6).
\end{align*}

Theorem. Let $V$ be a vector space of dimension $n$ with a positive definite scalar product $\langle\ ,\ \rangle$. Let $\{w_1,\cdots,w_r,u_1,\cdots,u_s\}$ with $r+s=n$ be an orthonormal basis of $V$. Let $W$ be a subspace generated by $w_1,\cdots,w_r$ and let $U$ be a subspace generated by $u_1,\cdots,u_s$. Then $U=W^{\perp}$ and $\dim V=\dim W+\dim W^{\perp}$.

## Scalar Products

Let $V$ be vector space. A scalar product is a map $\langle\ ,\ \rangle: V\times V\longrightarrow\mathbb{R}$ such that

SP 1. $\langle v,w\rangle=\langle w,v\rangle$

SP 2. $\langle u,v+w\rangle=\langle u,v\rangle+\langle u,w\rangle$

SP 3. If $x$ is a number, then
$$\langle xu,v\rangle=x\langle u,v\rangle=\langle u,xv\rangle$$

Additionally, we also assume the condition

SP 4. $\langle v,v\rangle>0$ if $v\ne O$

A scalar product with SP 4 is said to be positive definite.

Remark. If $v=O$, then $\langle v,w\rangle=0$ for any vector $w$ in $V$. This follows immediately from SP 3.

Example. Let $V=\mathbb{R}^n$ and define
$$\langle X,Y\rangle=X\cdot Y.$$
Then $\langle\ ,\ \rangle$ is a positive definite scalar product.

Example. Let $V$ be the function space of all continuous real-valued function on $[-\pi,\pi]$. For $f,g\in V$, we define
$$\langle f,g\rangle=\int_{-\pi}^{\pi}f(t)g(t)dt.$$
Then $\langle\ ,\ \rangle$ is a positive definte scalar product.

Using a scalar product, we can introduce the notion of orthogonality of vectors. Two vectors $v,w$ are said to be orthogonal or perpendicular if $\langle v,w\rangle=0$.

Let $S\subset V$ be a subspace of $V$. Let $S^{\perp}=\{w\in V: \langle v,w\rangle=0\ \mbox{for all}\ v\in V\}$.Then $S^{\perp}$ is also a subspace of $V$. (Check for yourself.) It is called the orthogonal space of $S$.

Define the length or the norm of $v\in V$ by
$$||v||=\sqrt{\langle v,v\rangle}.$$
It follows from SP 3 that
$$||cv||=|c|||v||$$
for any number $c$.

The Projection of a Vector onto Another Vector

For vectors in $\mathbb{R}^2$ or $\mathbb{R}^3$, the vector projection of a vector $v$ onto another vector $w$ is
$$||v||\cos\theta\frac{w}{||v||}=\langle v,w\rangle\frac{w}{||w||^2}=\frac{\langle v,w\rangle}{\langle w,w\rangle}w$$
as seen in the above Figure. The number $c=\frac{\langle v,w\rangle}{\langle w,w\rangle}$ is called the component of $v$ along $w$. Note that the vector projection of $v$ onto $w$
$$\frac{\langle v,w\rangle}{\langle w,w\rangle}w$$
can still be defined in any vector space with a scalar product.

Proposition. The vector $v-cw$ is perpendicular to $w$.

Proof. \begin{align*}
\langle v-cw,w\rangle&=\langle v,w\rangle-c\langle w,w\rangle\\
&=\langle v,w\rangle-\langle v,w\rangle\\
&=0.
\end{align*}

Example. Let $V=\mathbb{R}^n$. Then the component of $X=(x_1,\cdots,x_n)$ along $E_i$ is
$$X\cdot E_i=x_i.$$

Example. Let $V$ be the space of continuous functions on $[-\pi,\pi]$. Let $f(x)=\sin kx$, where $k$ is a positive integer. Then
$||f||=\sqrt{\pi}$. The component of $g(x)$ along $f(x)$ is
$$\frac{\langle g,f\rangle}{\langle f,f\rangle}=\frac{1}{\pi}\int_{-\pi}^{\pi}g(x)\sin kxdx.$$
It is called the Fourier coefficient of $g$ along $f$.

The following two inequalities are well-known for vectors in $\mathbb{R}^n$. They still hold in any vector space with a positive definite scalar product.

Theorem [Schwarz Inequality] Let $V$ be a vector space with a positive definite scalar product. For any $v,w\in V$,
$$|\langle v,w\rangle|\leq ||v||||w||.$$

Theorem [Triangle Inequality] Let $V$ be a vector space with a positive definite scalar product. For any $v,w\in V$,
$$||v+w||\leq ||v||+||w||.$$

## Sage: Basic Matrix Operations

For using Sage: Sage is an open source math software whose interface is a web browser (in particular firefox). You don’t have to install Sage in your computer to use it. You can access any sage server including the main Sage server. I am running a Sage server at sage.st.usm.edu. If you are a student at the University of Southern Mississippi, you are more than welcome to create an account at sage.st.usm.edu and use it.

Matrix Constructions

In Sage, $2\times 3$ matrix
$$\begin{pmatrix} 1 & 1 & -2\\ -1 & 4 & -5 \end{pmatrix}$$ can be created as follows. Let us say we want to call the matrix $A$. Type the following command in the blank line of your sage worksheet:

A=matrix([[1,1,-2],[-1,4,-5]])

In case you are familiar with Maple, not like Maple you will not see your matrix $A$ as an output when you click on “evaluate”. To see your matrix, you need to type

A

in the next blank line and click on “evaluate”again:

[ 1  1 -2]
[-1  4 -5]

Scalar Multiplication

If you want to multiply the matrix $A$ by a number 5, type the command

5*A

and click on “evaluate”. The output is

[  5   5 -10]
[ -5  20 -25]

To perform addition of two matrices:
$$\begin{pmatrix} 1 & 1 & -2\\ -1 & 4 & -5 \end{pmatrix}+\begin{pmatrix} 2 & 1 & 5\\ 1 & 3 & 2 \end{pmatrix}$$, first call the second matrix $B$:

B=matrix([[2,1,5],[1,3,2]])

and do

A+B

the output is

[ 3  2  3]
[ 0  7 -3]

The linear combination $3A+2B$ can be calculated by the command

3*A+2*B

and the output is

[  7   5   4]
[ -1  18 -11]

Transpose of a Matrix

To find the transpose of the matrix $A$ do

A.transpose()

and the output is

[ 1 -1]
[ 1  4]
[-2 -5]

Matrix Multiplication

An $m\times n$ matrix can be multiplied by a $p\times q$ matrix as long as $n=p$. The resulting multiplication is an $m\times q$ matrix. Let $C=\begin{pmatrix} 3 & 4\\ -1 &2\\ 2 &1 \end{pmatrix}$. The number of columns of $A$ and the number of rows of $C$ coincide as 3, so we can perform $AC$ and this can be done in Sage as:

A*C

The output is

[ 1 -1]
[ 1  4]
[-2 -5]

## Inverses

Let $F: V\longrightarrow W$ be a mapping. $F$ is said to be invertible if there exists a map $G: W\longrightarrow V$ such that
$$G\circ F:I_V,\ F\circ G=I_W,$$
where $I_V: V\longrightarrow V$ and $I_W: W\longrightarrow W$ the identity maps on $V$ and $W$, respectively. The map $G$ is called the inverse of $F$ and is denoted by $F^{-1}$.

Theorem. A map $F: V\longrightarrow W$ has an inverse if and only if it is one-to-one (injective) and onto (surjective).

Remark. If a map $F: V\longrightarrow W$ has an inverse, it is unique.

Example. The inverse map of $T_u$ is $T_{-u}$.

Example. Let $A$ be a square matrix. Then $L_A: \mathbb{R}^n\longrightarrow\mathbb{R}^n$ is invertible if and only if $A$ is invertible.

Proof. If $A$ is invertible, then one can immediately see that $L_A\circ L_{A^{-1}}=L_{A^{-1}}\circ L_A=I$. On the other hand, any linear map from $\mathbb{R}^n$ to $\mathbb{R}^n$ may be written as $L_B:\mathbb{R}^n\longrightarrow\mathbb{R}^n$ for some $n\times n$ square matrix $B$ as seen here. Suppose that $L_B:\mathbb{R}^n\longrightarrow\mathbb{R}^n$ is an inverse of $L_A: \mathbb{R}^n\longrightarrow\mathbb{R}^n$. Then $L_A\circ L_B=L_B\circ L_A=I$ i.e. for any $X\in\mathbb{R}^n$,
$$(AB)X=(BA)X=IX=X.$$
This implies that $AB=BA=I$. (Why?) Therefore, $A$ is invertibale and $B=A^{-1}$.

Theorem. Let $F: U\longrightarrow V$ be a linear map, and assume that this map has an inverse map  $G:V\longrightarrow U$. Then $G$ is a linear map.

Proof. The proof is straightforward and is left for an exercise.

Recall that a linear map $F: V\longrightarrow W$ is injective if and only if $\ker F=\{O\}$ as seen here.

Theorem. Let $V$ be a vector space of $\dim n$. If $W\subset V$ is a subspace of $\dim n$, then $W=V$.

Proof. It follows from the fact that a basis of a vector space is the maximal set of linearly independent vectors.

Theorem. Let $F: V\longrightarrow W$ be a linear map. Assume that $\dim V=\dim W$. Then

(i) If $\ker F=\{O\}$ then $F$ is invertible.

(ii) If $F$ is surjective then $F$ is invertible.

Proof. It follows from the formula
$$\dim V=\dim\ker F+\dim\mathrm{Im}F.$$

Example. Let $F:\mathbb{R}^2\longrightarrow\mathbb{R}^2$ be the linear map such that
$$F(x,y)=(3x-y,4x+2y).$$
Show that $F$ is invertible.

Solution. Suppose that $(x,y)\in\ker F$. Then
\left\{\begin{aligned} 3x-y&=0,\\ 4x+2y&=0. \end{aligned}\right.
This system of linear equations has only the trvial solution $(x,y)=(0,0)$, so $\ker F=\{O\}$. Therefore, $F$ is invertible.

A linear map $F: V\longrightarrow W$ which is invertible is called an isomorphism. If there is an isomorphism from $V$ onto $W$, we say $V$ is isomorphic to $W$ and write $V\cong W$ or $V\stackrel{F}{\cong}W$ in case we want to specify the isomorphism $F$. If $V\cong W$, as vector spaces they are identical and we do not distinguish them.

The following theorem says that there is essentially only one vector space of dimension $n$, $\mathbb{R}^n$.

Theorem. Let $V$ be a vector space of dimension $n$. Then $\mathbb{R}^n\cong V$.

Proof. Let $\{v_1,\cdots,v_n\}$ be a basis of $V$. Define a map $L:\mathbb{R}^n\longrightarrow V$ by
$$L(x_1,\cdots,x_n)=x_1v_1+\cdots+x_nv_n$$
for each $(x_1,\cdots,x_n)\in\mathbb{R}^n$. Then $L$ is an isomorphism.

Theorem. A square matrix $A$ is invertible if and only if its columns $A^1,\cdots,A^n$ are linearly indepdendent.

Proof. Let $L_A:\mathbb{R}^n\longrightarrow\mathbb{R}^n$ be the map such that $L_AX=AX$. For $X=\begin{pmatrix} x_1\\ \vdots\\ x_n \end{pmatrix}\in\mathbb{R}^n$,
$$L_AX=x_1A^1+\cdots+x_nA^n.$$
Hence, $\ker L_A=\{O\}$ if and only if $A^1,\cdots,A^n$ are linearly independent.