\documentclass{article}% \usepackage{amsmath} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{graphicx}% \setcounter{MaxMatrixCols}{30} %TCIDATA{OutputFilter=latex2.dll} %TCIDATA{Version=5.00.0.2552} %TCIDATA{CSTFile=40 LaTeX article.cst} %TCIDATA{Created=Tuesday, August 18, 2015 14:51:12} %TCIDATA{LastRevised=Monday, December 07, 2015 11:29:16} %TCIDATA{} %TCIDATA{} %TCIDATA{} %TCIDATA{Language=American English} \newtheorem{theorem}{Theorem} \newtheorem{acknowledgement}[theorem]{Acknowledgement} \newtheorem{algorithm}[theorem]{Algorithm} \newtheorem{axiom}[theorem]{Axiom} \newtheorem{case}[theorem]{Case} \newtheorem{claim}[theorem]{Claim} \newtheorem{conclusion}[theorem]{Conclusion} \newtheorem{condition}[theorem]{Condition} \newtheorem{conjecture}[theorem]{Conjecture} \newtheorem{corollary}[theorem]{Corollary} \newtheorem{criterion}[theorem]{Criterion} \newtheorem{definition}[theorem]{Definition} \newtheorem{example}[theorem]{Example} \newtheorem{exercise}[theorem]{Exercise} \newtheorem{lemma}[theorem]{Lemma} \newtheorem{notation}[theorem]{Notation} \newtheorem{problem}[theorem]{Problem} \newtheorem{proposition}[theorem]{Proposition} \newtheorem{remark}[theorem]{Remark} \newtheorem{solution}[theorem]{Solution} \newtheorem{summary}[theorem]{Summary} \newenvironment{proof}[Proof]{\noindent\textbf{#1.} }{\ \rule{0.5em}{0.5em}} \begin{document} \title{Math 413/513 Chapter 6 (from Friedberg, Insel, \& Spence)} \author{David Glickenstein} \maketitle \section{Inner product spaces} In this chapter, we will only consider the fields $\mathbb{R}$ and $\mathbb{C}$. \begin{definition} Let $V$ be a vector space over $F=\mathbb{R}$ or $\mathbb{C}$. An \emph{inner product} on $V$ is a function $V\times V\rightarrow F,$ denoted $\left( x,y\right) \rightarrow\left\langle x,y\right\rangle ,$ such that the following hold for all $x,y,z\in V$ and $c\in F$: \begin{enumerate} \item $\left\langle x+z,y\right\rangle =\left\langle x,y\right\rangle +\left\langle z,y\right\rangle$. \item $\left\langle cx,y\right\rangle =c\left\langle x,y\right\rangle .$ \item $\left\langle y,x\right\rangle =\overline{\left\langle x,y\right\rangle }$ where the bar denotes complex conjugation. \item $\left\langle x,x\right\rangle >0$ if $x\neq0.$ \end{enumerate} \end{definition} Here are some observations: \begin{itemize} \item The first two conditions could be called being \emph{linear in the first component}. \item The third condition is called being \emph{conjugate symmetric} (or \emph{symmetric} if $F=\mathbb{R}$). \item Linearity in the first component and conjugate symmetry imply linearity in the second component, and being linear in both components is called being\emph{ bilinear}. \item Notice that conjugate symmetry implies that $\left\langle x,x\right\rangle \in\mathbb{R}$ even if $F=\mathbb{C}$ since $\left\langle x,x\right\rangle =\overline{\left\langle x,x\right\rangle }.$ \item The second condition also implies that $\left\langle \vec{0},\vec {0}\right\rangle =0,$ which together with the fourth condition is called being \emph{positive definite}. \item From all of this, we could have just specified that $\left\langle \cdot,\cdot\right\rangle$ is bilinear, conjugate symmetric, and positive definite. \item We often write $\left\Vert v\right\Vert ^{2}$ to represent $\left\langle v,v\right\rangle .$ Since $\left\Vert v\right\Vert ^{2}\geq0,$ it has a unique positive square root that we call $\left\Vert v\right\Vert .$ \end{itemize} \begin{example} The first example is the dot product for $\mathbb{R}^{n},$ where if $x=\left( x_{1},\ldots,x_{n}\right)$ and $y=\left( y_{1},\ldots,y_{n}\right)$ then $\left\langle x,y\right\rangle =x\cdot y=\sum_{i=1}^{n}x_{i}y_{i}.$ \end{example} \begin{example} The standard inner product on $\mathbb{C}^{n}$ is $\left\langle x,y\right\rangle =\sum_{i=1}^{n}x_{i}\overline{y_{i}}.$ Check the properties. \end{example} \begin{example} Given any inner product $\left\langle \cdot,\cdot\right\rangle ,$ we can multiply it by a positive real number $r>0$ to get another inner product $\left\langle x,y\right\rangle ^{\prime}=r\left\langle x,y\right\rangle .$ Note that it would not be an inner product if $r\leq0$ or $r$ is not a real number. \end{example} \begin{example} For continuous, real-valued functions on $\left[ 0,1\right] ,$ there is an inner product $\left\langle f,g\right\rangle =\int_{0}^{1}f\left( t\right) g\left( t\right) dt.$ Note that it is important that the functions be continuous to ensure that $\left\langle f,f\right\rangle =\int_{0}^{1}f\left( t\right) ^{2}dt>0$ if $f\neq\vec{0}.$ \end{example} \begin{example} Consider the vector space $\mathbb{C}^{n\times n}.$ We define the conjugate transpose, or adjoint, $A^{\ast}$ of a matrix $A$ by specifying the entries as:% $\left( A^{\ast}\right) _{ij}=\overline{A_{ji}}.$ We can now define an inner product by $\left\langle A,B\right\rangle =tr\left( B^{\ast}A\right) .$ Let's check it is an inner product. First, $\left\langle A+A^{\prime },B\right\rangle =tr\left( B^{\ast}\left( A+A^{\prime}\right) \right) =tr\left( B^{\ast}A+B^{\ast}A^{\prime}\right) =tr\left( B^{\ast}A\right) +tr\left( B^{\ast}A^{\prime}\right) =\left\langle A,B\right\rangle +\left\langle A^{\prime},B\right\rangle .$ Also,% \begin{align*} \left\langle B,A\right\rangle & =tr\left( A^{\ast}B\right) \\ \overline{\left\langle A,B\right\rangle } & =tr\left( \overline{B^{\ast}% A}\right) =tr\left( A^{\ast}B\right) . \end{align*} Also,% $\left\langle A,A\right\rangle =tr\left( A^{\ast}A\right) =\sum_{i}\sum _{j}A_{ij}^{\ast}A_{ji}=\sum_{j,i}\overline{A_{ji}}A_{ji}=\sum_{j,i}\left\vert A_{ji}\right\vert ^{2}%$ and so this is nonnegative and equals zero if and only if $A=0.$ \end{example} \begin{definition} A vector space $V$ together with an inner product is called an \emph{inner product space}. If the field is $\mathbb{C}$ then we call it a \emph{complex inner product space}, and if the field is $\mathbb{R}$ we call it a \emph{real inner product space}. \end{definition} The following are basic properties of inner product spaces. \begin{theorem} Let $V$ be an inner product space. Then for $x,y,z\in V$ and $c\in F$ the following statements are true: \begin{enumerate} \item $\left\langle x,y+z\right\rangle =\left\langle x,y\right\rangle +\left\langle x,z\right\rangle .$ \item $\left\langle x,cy\right\rangle =\bar{c}\left\langle x,y\right\rangle .$ \item $\left\langle x,\vec{0}\right\rangle =\left\langle \vec{0}% ,x\right\rangle =0.$ \item $\left\langle x,x\right\rangle =0$ if and only if $x=\vec{0}.$ \item If $\left\langle x,y\right\rangle =\left\langle x,z\right\rangle$ for all $x\in V,$ then $y=z.$ \end{enumerate} \end{theorem} \begin{proof} Most of these follow pretty easily. The last one can be shown as follows. Suppose $\left\langle x,y\right\rangle =\left\langle x,z\right\rangle$ for all $x\in V.$ Then $\left\langle x,y-z\right\rangle =0$ for all $x.$ In particular, taking $x=y-z,$ we get that $\left\langle y-z,y-z\right\rangle =0.$ But that implies $y-z=\vec{0}.$ \end{proof} Note that the first two statements in the above theorem are called being \emph{conjugate linear} in the second component. Recall the definition of the length, or norm:\ $\left\Vert x\right\Vert =\sqrt{\left\langle x,x\right\rangle }.$ This generalizes the Euclidean norm $\left\Vert \left( x_{1},\ldots,x_{n}\right) \right\Vert =\sqrt{x_{1}% ^{2}+\cdots+x_{n}^{2}}.$ Many properties are still true for inner product spaces: \begin{theorem} Let $V$ be an inner product space over $F=\mathbb{R}$ or $\mathbb{C}$. The for $x,y\in V$ and $c\in F,$ the following are true: \begin{enumerate} \item $\left\Vert cx\right\Vert =\left\vert c\right\vert \cdot\left\Vert x\right\Vert .$ \item $\left\Vert x\right\Vert =0$ iff $x=\vec{0}.$ In general, $\left\Vert x\right\Vert \geq0.$ \item (Cauchy-Schwarz inequality) $\left\vert \left\langle x,y\right\rangle \right\vert \leq\left\Vert x\right\Vert \left\Vert y\right\Vert .$ \item (Triangle inequality) $\left\Vert x+y\right\Vert \leq\left\Vert x\right\Vert +\left\Vert y\right\Vert .$ \end{enumerate} \end{theorem} \begin{proof} We will just prove the last two. Consider $x-cy$ and notice that \begin{align*} 0 & \leq\left\Vert x-cy\right\Vert ^{2}\\ & =\left\langle x-cy,x-cy\right\rangle \\ & =\left\Vert x\right\Vert ^{2}-\left\langle cy,x\right\rangle -\left\langle x,cy\right\rangle +\left\Vert cy\right\Vert ^{2}\\ & =\left\Vert x\right\Vert ^{2}-2\operatorname{Re}\bar{c}\left\langle x,y\right\rangle +\left\vert c\right\vert ^{2}\left\Vert y\right\Vert ^{2}. \end{align*} Notice that if we take $c=\frac{\left\langle x,y\right\rangle }{\left\Vert y\right\Vert ^{2}}$ then $0\leq\left\Vert x\right\Vert ^{2}-2\frac{\left\vert \left\langle x,y\right\rangle \right\vert ^{2}}{\left\Vert y\right\Vert ^{2}}% +\frac{\left\vert \left\langle x,y\right\rangle \right\vert ^{2}}{\left\Vert y\right\Vert ^{2}}=\left\Vert x\right\Vert ^{2}-\frac{\left\vert \left\langle x,y\right\rangle \right\vert ^{2}}{\left\Vert y\right\Vert ^{2}},$ which implies the Cauchy-Schwarz inequality. We can now show% \begin{align*} \left\Vert x+y\right\Vert ^{2} & =\left\langle x+y,x+y\right\rangle \\ & =\left\Vert x\right\Vert ^{2}+2\operatorname{Re}\left\langle x,y\right\rangle +\left\Vert y\right\Vert ^{2}\\ & \leq\left\Vert x\right\Vert ^{2}+2\left\Vert x\right\Vert \left\Vert y\right\Vert +\left\Vert y\right\Vert ^{2}\\ & =\left( \left\Vert x\right\Vert +\left\Vert y\right\Vert \right) ^{2}% \end{align*} since $2a\leq a^{2}+b^{2}=\left\vert a+bi\right\vert ^{2}$ and so $2\operatorname{Re}\left\langle x,y\right\rangle \leq\left\vert \left\langle x,y\right\rangle \right\vert \leq\left\Vert x\right\Vert \left\Vert y\right\Vert .$ \end{proof} \begin{definition} Let $V$ be an inner product space. Vectors $x$ and $y$ in $V$ are \emph{orthogonal} (\emph{perpendicular}) if $\left\langle x,y\right\rangle =0.$ As subset $S\subseteq V$ is \emph{orthogonal} if any two distinct vectors in $S$ are orthogonal. A vector $x$ in $V$ is a \emph{unit vector} if $\left\Vert x\right\Vert =1.$ A subset $S\subseteq V$ is \emph{orthonormal} if $S$ is orthogonal and consists entirely of unit vectors. \end{definition} Note that $S=\left\{ x_{1},\ldots,x_{k}\right\}$ is orthonormal iff $\left\langle x_{i},x_{j}\right\rangle =\delta_{ij}.$ Also note that we can make an orthonormal set from an orthogonal set by replacing each vector $x$ by $\frac{1}{\left\Vert x\right\Vert }x.$ This will not change the orthogonality since $\left\langle \frac{x}{\left\Vert x\right\Vert },\frac{y}{\left\Vert y\right\Vert }\right\rangle =\frac{1}{\left\Vert x\right\Vert \left\Vert y\right\Vert }\left\langle x,y\right\rangle$ since $\left\Vert y\right\Vert \in\mathbb{R}$. We call this process \emph{normalizing} the set. \begin{proposition} If $V$ is an inner product space and $S\subseteq V$ is orthogonal and nonzero, then $S$ is linearly independent. \end{proposition} \begin{proof} We first note that if $S$ is not the set consisting only of zero, then zero cannot be in $S.$ Suppose $a_{1}x_{1}+\cdots+a_{k}x_{k}=\vec{0}$ for scalars $a_{1},\ldots,a_{k}$ and vectors $x_{1},\ldots,x_{k}$ in $S.$ Then we see that $0=\left\langle a_{1}x_{1}+\cdots+a_{k}x_{k},x_{i}\right\rangle =a_{i}% \left\Vert x_{i}\right\Vert ^{2}%$ and since $\left\Vert x_{i}\right\Vert ^{2}\neq0,$ we must have $a_{i}=0$. This can be done for all $i.$ \end{proof} \section{Problems} \begin{itemize} \item FIS Section 6.1 exercises 2, 3, 8, 10, 11, 16, 17, 19, 20, 22 \end{itemize} \section{Orthonormal bases} \begin{definition} Let $V$ be an inner product space. A subset of $V$ is an \emph{orthonormal basis} for $V$ if it is an ordered basis that is orthonormal. \end{definition} The standard basis is orthonormal for the usual inner product. So is any rotation of the standard basis! Orthonormal bases make it easier to write the coefficients than in any old basis. \begin{theorem} Let $V$ be an inner product space and $S=\left\{ v_{1},v_{2},\ldots ,v_{k}\right\}$ be an orthogonal subset of $V$ consisting of nonzero vectors. If $y\in\operatorname{span}S,$ then $y=\sum_{i=1}^{k}\frac{\left\langle y,v_{i}\right\rangle }{\left\langle v_{i},v_{i}\right\rangle }v_{i}.$ \end{theorem} \begin{proof} Since $y\in\operatorname{span}S,$ we must have that there exist scalars $a_{1},\ldots,a_{k}$ such that $y=\sum_{i=1}^{k}a_{i}v_{i}.$ We can now take the inner product with $v_{j}$ for $j=1,\ldots,k$ and find that \begin{align*} \left\langle y,v_{j}\right\rangle & =\left\langle \sum_{i=1}^{k}a_{i}% v_{i},v_{j}\right\rangle \\ & =\sum_{i=1}^{k}a_{i}\left\langle v_{i},v_{j}\right\rangle \\ & =a_{j}\left\langle v_{j},v_{j}\right\rangle \end{align*} and so (since $\left\Vert v_{j}\right\Vert \neq0$), $a_{j}=\frac{\left\langle y,v_{j}\right\rangle }{\left\langle v_{j},v_{j}\right\rangle }.$ $y=\sum_{i=1}^{k}\frac{\left\langle y,v_{i}\right\rangle }{\left\langle v_{i},v_{i}\right\rangle }v_{i}.$ \end{proof} Note that if $S$ is orthonormal, then the denominators are all $1.$ We can always take a linearly independent set and use it to find an orthogonal set with the same span. We do this by considering orthogonal projections of vectors onto a subspace. \begin{theorem} Let $W$ be a finite dimensional subspace of the inner product space $V.$ Then for a vector $y\in V,$ there is a unique vector $u\in W$ that minimizes $\left\Vert y-w\right\Vert ^{2}$ for all $w\in W.$ \end{theorem} \begin{proof} Suppose there is a $u\in W$ such that $\left\langle w,y-u\right\rangle =0$ for any $w\in W.$ Then if $w\in W$ (and hence so is $u-w$), \begin{align*} \left\Vert y-w\right\Vert ^{2} & =\left\Vert u+\left( y-u\right) -w\right\Vert ^{2}\\ & =\left\langle u-w+\left( y-u\right) ,u-w+\left( y-u\right) \right\rangle \\ & =\left\Vert u-w\right\Vert ^{2}+\left\langle u-w,y-u\right\rangle +\left\langle y-u,u-w\right\rangle +\left\Vert y-u\right\Vert ^{2}\\ & =\left\Vert u-w\right\Vert ^{2}+\left\Vert y-u\right\Vert ^{2}\\ & \geq\left\Vert y-u\right\Vert ^{2}. \end{align*} We can do this if $W$ is finite dimensional using the following theorem. \end{proof} \begin{definition} We call the assignment of $u$ the orthogonal projection of $y$ onto $W,$ denoted $u=P_{W}\left( y\right) .$ \end{definition} It is an important fact that $\left\langle y-P_{W}\left( y\right) ,w\right\rangle =0$ for all $w\in W.$ \begin{definition} The orthogonal complement of $W,$ written $W^{\bot}$ (pronounced \textquotedblleft W perp\textquotedblright), is the set of all vectors $v\in V$ such that $\left\langle v,w\right\rangle$ for all $w\in W$. \end{definition} Note that $W^{\bot}$ is a vector space. \begin{proposition} $W^{\bot}$ is a vector space. \end{proposition} \begin{proof} It is straightforward to see that $\left\langle \vec{0},w\right\rangle =0$ for all $w\in W,$ so $\vec{0}\in W^{\bot}.$ if $v,u\in W^{\bot}$ and $c\in F$ then $\left\langle cv+u,w\right\rangle =c\left\langle v,w\right\rangle =\left\langle u,w\right\rangle =0$ so $cv+u\in W^{\bot}.$ \end{proof} We can construct an orthogonal set from a linearly independent set by starting with the first vector and then projecting the next vector into \begin{theorem} Let $V$ be an inner product space and $S=\left\{ w_{1},\ldots,w_{n}\right\}$ be a linearly independent subset of $V.$ Define $S^{\prime}=\left\{ v_{1},\ldots,v_{n}\right\}$ by $v_{1}=w_{1}$ and $v_{k}=w_{k}-\sum_{j=1}^{k-1}\frac{\left\langle w_{k},v_{j}\right\rangle }{\left\langle v_{j},v_{j}\right\rangle }v_{j}%$ for $k=2,\ldots,n.$ Then $S^{\prime}$ is an orthogonal set of nonzero vectors such that $\operatorname{span}S^{\prime}=\operatorname{span}S.$ \end{theorem} \begin{proof} We show inductively that $v_{k+1}$ is orthogonal to $v_{1},\ldots,v_{k}.$ It is clear that \begin{align*} \left\langle v_{2},v_{1}\right\rangle & =\left\langle w_{2}-\frac {\left\langle w_{2},v_{1}\right\rangle }{\left\langle v_{1},v_{1}\right\rangle }v_{1},v_{1}\right\rangle \\ & =\left\langle w_{2},v_{1}\right\rangle -\frac{\left\langle w_{2}% ,v_{1}\right\rangle }{\left\langle v_{1},v_{1}\right\rangle }\left\langle v_{1},v_{1}\right\rangle =0. \end{align*} We then can use the inductive hypothesis to assume $\left\langle v_{i}% ,v_{j}\right\rangle =0$ for $i,j\leq k$ and see that \begin{align*} \left\langle v_{k},v_{i}\right\rangle & =\left\langle w_{k}-\sum_{j=1}% ^{k-1}\frac{\left\langle w_{k},v_{j}\right\rangle }{\left\langle v_{j}% ,v_{j}\right\rangle }v_{j},v_{i}\right\rangle \\ & =\left\langle w_{k},v_{i}\right\rangle -\frac{\left\langle w_{k}% ,v_{i}\right\rangle }{\left\langle v_{i},v_{i}\right\rangle }\left\langle v_{i},v_{i}\right\rangle =0. \end{align*} Thus $S^{\prime}$ is orthogonal. Hence $S^{\prime}$ is linearly independent and since each element of $S^{\prime}$ is in the span of $S,$ $\operatorname{span}S^{\prime}\subseteq\operatorname{span}S,$ and hence $\operatorname{span}S^{\prime}=\operatorname{span}S$ (since they have the same dimension). \end{proof} Note: this process of producing an orthogonal set is called the \emph{Gram-Schmidt process}. \begin{theorem} Suppose that $S=\left\{ v_{1},\ldots,v_{k}\right\}$ is an orthonormal set in a $n$-dimensional inner product space $V.$ Then \begin{enumerate} \item $S$ can be extended to an orthonormal basis $\left\{ v_{1},\ldots ,v_{k},v_{k+1,}\ldots,v_{n}\right\}$ for $V.$ \item If $W=\operatorname{span}S,$ then $S_{1}=\left\{ v_{k+1},\ldots ,v_{n}\right\}$ is an orthonormal basis for $W^{\bot}.$ \item If $W$ is any subspace of $V,$ then $\dim V=\dim W+\dim W^{\bot}.$ \end{enumerate} \end{theorem} \begin{proof} By the replacement theorem, $S$ can be extended into a basis, and then the Gram-Schmidt process can be used to turn this into an orthogonal set. Then normalizing gives an orthonormal set. $S_{1}$ is clearly a linearly independent subset of $W^{\bot}.$ Since $\left\{ v_{1},\ldots,v_{n}\right\}$ is a basis, any vector in $W^{\bot}$ can be written as a linear combination of these vectors. However, since $w\in W^{\bot}$ satisfies $\left\langle w,v_{i}\right\rangle =0$ for $i=1,\ldots,k,$ $w$ is in the span of $S_{1},$ hence $S_{1}$ is a basis. The dimension statement is clear now that we know that $S$ is a basis for $S,$ $S^{\prime}$ is a basis for $W^{\bot},$ and $\left\{ v_{1},\ldots,v_{n}\right\}$ is a basis for $V.$ \end{proof} \section{Problems} \begin{itemize} \item FIS Section 6.2 exercises 4, 5, 7, 8, 10, 13, 22. \end{itemize} \section{Adjoints and eigenvalues} \begin{definition} Suppose $T$ is a linear operator on an inner product space $V.$ If $T^{\ast}$ is a linear operator on $V$ such that $\left\langle T\left( x\right) ,y\right\rangle =\left\langle x,T^{\ast }\left( y\right) \right\rangle$ for all $x,y\in V,$ we say $T^{\ast}$ is the \emph{adjoint} of $T.$ We read $T^{\ast}$ as \textquotedblleft$T$ star.\textquotedblright \end{definition} \begin{theorem} \label{thm: adjoint exists}Let $V$ be a finite-dimensional inner product space and let $T$ be a linear operator on $V.$ Then there exists a unique function $T^{\ast}:V\rightarrow V$ that for all $x,y\in V,$% $\left\langle T\left( x\right) ,y\right\rangle =\left\langle x,T^{\ast }\left( y\right) \right\rangle .$ Furthermore, $T^{\ast}$ is linear and hence the adjoint of $T.$ \end{theorem} In order to prove this theorem, we need to show that every linear map on a finite-dimensional vector space can be represented in terms of the inner product. We will then use this idea to construct the adjoint. \begin{theorem} \label{thm: inner product linear}Let $V$ be a finite dimensional vector space over $F$ and let $g:V\rightarrow F$ be a linear transformation. Then there exists a unique $y\in V$ such that $g\left( x\right) =\left\langle x,y\right\rangle$ for all $x\in V.$ \end{theorem} \begin{proof} Let $\beta=\left\{ v_{1},\ldots,v_{n}\right\}$ be an orthonormal basis for $V.$ For any $x\in V,$ we must have $y=\sum b_{i}v_{i}$ for some $b_{i},$ so in order for $g\left( x\right) =\left\langle x,y\right\rangle$ we must have that $g\left( v_{i}\right) =\left\langle v_{i},y\right\rangle =\bar{b}_{i}.$ Hence any $y$ that satisfies the equation must have $b_{i}=\overline{g\left( v_{i}\right) }$ (this proves uniqueness). We now confirm that $y=\sum \overline{g\left( v_{i}\right) }v_{i}$ satisfies the theorem: if $x=\sum a_{i}v_{i}$ then \begin{align*} \left\langle x,y\right\rangle & =\left\langle \sum_{i=1}^{n}a_{i}v_{i}% ,\sum_{j=1}^{n}\overline{g\left( v_{j}\right) }v_{j}\right\rangle \\ & =\sum_{i=1}^{n}\sum_{j=1}^{n}a_{i}g\left( v_{j}\right) \left\langle v_{i},v_{j}\right\rangle \\ & =\sum_{i=1}^{n}a_{i}g\left( v_{i}\right) =g\left( x\right) . \end{align*} \end{proof} \begin{proof} [Proof of Theorem \ref{thm: adjoint exists}]Let $\beta=\left\{ v_{1}% ,\ldots,v_{n}\right\} ,$ be an orthonormal basis for $V$ and define $T_{i}:V\rightarrow F$ by $T_{i}\left( x\right) =\left\langle T\left( x\right) ,v_{i}\right\rangle .$ By Theorem \ref{thm: inner product linear}, there exists a unique $y_{i}\in V$ such that $T_{i}\left( x\right) =\left\langle x,y_{i}\right\rangle ,$ and so $\left\langle T\left( x\right) ,v_{i}\right\rangle =\left\langle x,y_{i}\right\rangle .$ We can thus define a linear tranformation $T^{\ast}$ by defining it on the basis to be $T^{\ast}\left( v_{i}\right) =y_{i}$ and extending it linearly to a transformation on $V.$ Now, for any vector $y\in V,$ $y=\sum b_{i}v_{i}$ for some scalars $b_{1},\ldots,b_{n}$ and we check that \begin{align*} \left\langle T\left( x\right) ,y\right\rangle & =\left\langle T\left( x\right) ,\sum b_{i}v_{i}\right\rangle \\ & =\sum\bar{b}_{i}\left\langle T\left( x\right) ,v_{i}\right\rangle \\ & =\sum\bar{b}_{i}\left\langle x,T^{\ast}\left( v_{i}\right) \right\rangle \\ & =\sum\left\langle x,b_{i}T^{\ast}\left( v_{i}\right) \right\rangle \\ & =\left\langle x,T^{\ast}\left( y\right) \right\rangle . \end{align*} $T^{\ast}$ is unique since if there were another linear transformation $U$ satisfying the properties, then for any $y\in V,$ $\left\langle x,T^{\ast}\left( y\right) \right\rangle =\left\langle T\left( x\right) ,y\right\rangle =\left\langle x,U\left( y\right) \right\rangle$ must be true for all $x\in V,$ implying that $T^{\ast}\left( y\right) =U\left( y\right) .$ (See 6.1, problem 9.) \end{proof} Notice that the adjoint works the other way as well:% $\left\langle x,T\left( y\right) \right\rangle =\overline{\left\langle T\left( y\right) ,x\right\rangle }=\overline{\left\langle y,T^{\ast}\left( x\right) \right\rangle }=\left\langle T^{\ast}\left( x\right) ,y\right\rangle .$ Note our proof of the existence of an adjoint used a basis, and it turns out that for a linear operator on an infinite dimensional vector space, the existence of an adjoint is not guaranteed. However, most properties we will derive are true if the adjoint exists (sometimes it does). Recall that for a matrix $A,$ we denote its conjugate transpose by $A^{\ast}$ (conjugate transpose means we take a transpose and replace each entry with its complex conjugate). This is related to the adjoint. \begin{theorem} Let $V$ be a finite dimensional inner product space and $\beta$ be an orthonormal basis for $V.$ If $T$ is a linear operator on $V,$ then $\left[ T^{\ast}\right] _{\beta}=\left[ T\right] _{\beta}^{\ast}.$ \end{theorem} It might help to be clear about the content here. If one writes the matrix for the adjoint transformation $T^{\ast}$, this matrix is the same as the conjugate transpose of the matrix for the transformation $T$ (provided the basis used in both cases is orthonormal). \begin{proof} If we write $A=\left[ T\right] _{\beta}$ and $B=\left[ T^{\ast}\right] _{\beta}$ then by orthonormality we have that $B_{ij}=\left\langle T^{\ast}\left( v_{j}\right) ,v_{i}\right\rangle =\left\langle v_{j},T\left( v_{i}\right) \right\rangle =\overline {\left\langle T\left( v_{i}\right) ,v_{j}\right\rangle }=\bar{A}_{ji}.$ It follows that $B=A^{\ast}.$ \end{proof} \begin{corollary} Left multiplication by an $n\times n$ matrix $A$ satisfies $L_{A^{\ast}}=L_{A}^{\ast}.$ \end{corollary} \begin{theorem} Let $V$ be an inner product space and $T,U$ be linear operators on $V$ that have an adjoint (this is always true if $V$ is finite dimensional). Then \begin{enumerate} \item $\left( T+U\right) ^{\ast}=T^{\ast}+U^{\ast}$ \item $\left( cT\right) ^{\ast}=\bar{c}T^{\ast}$ for any $c\in F$ \item $\left( TU\right) ^{\ast}=U^{\ast}T^{\ast}$ \item $T^{\ast\ast}=T$ \item $I_{V}^{\ast}=I_{V}.$ \end{enumerate} \end{theorem} \begin{proof} The proofs are pretty straightforward. We prove the first and third: for any $x,y\in V$% \begin{align*} \left\langle \left( T+U\right) x,y\right\rangle & =\left\langle T\left( x\right) +U\left( x\right) ,y\right\rangle \\ & =\left\langle T\left( x\right) ,y\right\rangle +\left\langle U\left( x\right) ,y\right\rangle \\ & =\left\langle x,T^{\ast}\left( y\right) \right\rangle +\left\langle x,U^{\ast}\left( y\right) \right\rangle \\ & =\left\langle x,T^{\ast}\left( y\right) +U^{\ast}\left( y\right) \right\rangle \\ & =\left\langle x,\left( T^{\ast}+U^{\ast}\right) \left( y\right) \right\rangle . \end{align*} Also, \begin{align*} \left\langle TU\left( x\right) ,y\right\rangle & =\left\langle T\left( U\left( x\right) \right) ,y\right\rangle \\ & =\left\langle U\left( x\right) ,T^{\ast}\left( y\right) \right\rangle \\ & =\left\langle x,U^{\ast}T^{\ast}\left( y\right) \right\rangle . \end{align*} \end{proof} \section{Problems} \begin{itemize} \item FIS Section 6.3 exercises 4, 6, 8, 10, 12, 13, 15 \end{itemize} \section{Schur's theorem and the (baby) spectral theorem} We would like to understand when there exists an orthonormal basis of eigenvectors. We first have Schur's theorem, that says that we can represent a linear transformation by an upper triangular matrx. \begin{theorem} [Schur]Let $T$ be a linear operator on a finite dimensional inner product space $V.$ Suppose the characteristic polynomial of $T$ splits. The there exists an orthonormal basis $\beta$ of $V$ such that $\left[ T\right] _{\beta}$ is upper triangular. \end{theorem} The main idea behind the proof is that we can restrict $T$ to a subspace. If $W$ is a subspace of $V$ and $T\left( W\right) \subseteq W,$ then we say that $W$ is $T$\emph{-invariant}. Hence there is a linear operator $T_{W}% \in\mathcal{L}\left( W\right)$ that is just $T_{W}\left( x\right) =T\left( x\right) .$ If $W$ is $T$-invariant and $\beta$ is a basis for $V$ that is an extension of a basis for $W,$ then $\left[ T\right] _{\beta}$ has block form $\left[ \begin{array} [c]{cc}% \left[ T_{W}\right] & \ast\\ 0 & \ast \end{array} \right]$ where the $\ast$'s are unknown but generally not zero. We now see that if $z$ is an eigenvector of $T^{\ast}$ and if $z^{\bot }=\left\{ x\in V:\left\langle x,z\right\rangle =0\right\} ,$ then $z^{\bot}$ is $T$-invariant, i.e., $\left\langle T\left( y\right) ,z\right\rangle =\left\langle y,T^{\ast }\left( z\right) \right\rangle =\left\langle y,\lambda z\right\rangle =\bar{\lambda}\left\langle y,z\right\rangle =0.$ \begin{proof} We first notice that the characteristic polynomial of $T^{\ast}$ satisfies $\det\left( T^{\ast}-\lambda I\right) =\det\left( T-\bar{\lambda}I\right)$ since determinant is invariant under trace. Hence if the characteristic polynomial of $T$ splits, then there is an eigenvector $z$ of $T^{\ast}.$ Then $z^{\bot}$ is $T$-invariant, and so we can use the inductive hypothesis to show that $\left[ T\right] _{\beta}$ is upper triangular. This completes the proof. \end{proof} Notice the following consequence: \begin{theorem} Suppose $T\in\mathcal{L}\left( V\right)$ for a finite dimensional inner product space $V.$ If $T$ is self-adjoint, i.e., $T^{\ast}=T$, and its characteristic polynomial splits, then $V$ has an orthonormal basis of eigenvectors of $T.$ \end{theorem} \begin{proof} Since $T=T^{\ast},$ we have that $\left[ T^{\ast}\right] _{\beta}=\left[ T\right] _{\beta}^{\ast}.$ If we choose the basis from the Schur theorem, $\left[ T\right] _{\beta}$ is upper triangular and $\left[ T\right] _{\beta}^{\ast}$ is lower triangular and since $\left[ T\right] _{\beta}=\left[ T^{\ast}\right] _{\beta}=\left[ T\right] _{\beta}^{\ast}%$ we must have that $\left[ T\right] _{\beta}$ is triangular. Note that we needed that the characteristic polynomials splits, which is not known for real inner product spaces, so we need the following lemma to complete the proof. \end{proof} \begin{lemma} Let $T$ be a self-adjoint operator on a finite dimensional inner product space $V$. Then \begin{enumerate} \item $T$ has real eigenvalues \item The characteristic polynomial of $T$ splits. \end{enumerate} \end{lemma} \begin{proof} If $\lambda$ is an eigenvalue with eigenvector $v,$ then $\lambda\left\langle v,v\right\rangle =\left\langle T\left( v\right) ,v\right\rangle =\left\langle v,T^{\ast}\left( v\right) \right\rangle =\left\langle v,T\left( v\right) \right\rangle =\left\langle v,\lambda v\right\rangle =\bar{\lambda}\left\langle v,v\right\rangle$ and so $\lambda=\bar{\lambda}$ and $\lambda$ is real. The second statement follows from the fundamental theorem of algebra if $F=\mathbb{C}$. If $F=\mathbb{R}$ then we can consider $A=\left[ T\right] _{\beta}$ for some basis $\beta,$ and then consider the linear transformation $L_{A}% :\mathbb{C}^{n}\rightarrow\mathbb{C}^{n}$ given by $L_{A}\left( x\right) =Ax.$ Since $T$ is self-adjoint, $A=A^{\ast}$ and hence $L_{A}$ is self-adjoint and hence has real eigenvalues. Since all polynomials split over the complex numbers, the characteristic polynomial for $L_{A}$ splits, and since the eigenvalues are real, the polynomial has real coefficients. The characteristic polynomial for $L_{A}$ is the same as that for $T,$ and since the eigenvalues are real, that means that the characteristic polynomial for $T$ splits. \end{proof} \begin{remark} The theorem above is if and only if, actually, for real inner product spaces. For complex inner product spaces, self-adjoint can be relaxed to normal, which means that $T$ and $T^{\ast}$ commute, i.e. $TT^{\ast}=T^{\ast}T.$ For a normal operator, any eigenvector $x$ of $T,$ with eigenvalue $\lambda,$ is an eigenvector for $T^{\ast}$ since \begin{align*} 0 & =\left\langle Tx-\lambda x,Tx-\lambda x\right\rangle \\ & =\left\langle x,T^{\ast}Tx\right\rangle -\lambda\left\langle x,Tx\right\rangle -\bar{\lambda}\left\langle Tx,x\right\rangle +\left\vert \lambda\right\vert ^{2}\left\langle x,x\right\rangle \\ & =\left\langle x,TT^{\ast}x\right\rangle -\lambda\left\langle T^{\ast }x,x\right\rangle -\bar{\lambda}\left\langle x,T^{\ast}x\right\rangle +\left\vert \lambda\right\vert ^{2}\left\langle x,x\right\rangle \\ & =\left\langle T^{\ast}x-\bar{\lambda}x,T^{\ast}x-\bar{\lambda}% x\right\rangle \end{align*} and so $x$ is an eigenvector for $T^{\ast}$ with eigenvalue $\bar{\lambda}.$ \end{remark} \section{Problems} \end{document}