Basic Course CPS615 Contact Points

• Instructor: Geoffrey Fox gcf@npac.syr.edu (315)443-2163 Room 3-131 CST

• Backup: Nancy McCracken njm@npac.syr.edu (315)443-4687 Room 3-234 CST

• TA: John Houle houle@npac.syr.edu

• Administrative Support: Nora Downey-Easter nora@npac.syr.edu (315)443-4740 Room 3-210 CST

• CPS615 Powers that be above can be reached at cps615ad@npac.syr.edu

• CPS615 Students can be reached by mailing cps615@npac.syr.edu

• Homepage will be: http://www.npac.syr.edu/projects/cps615fall95

Course Structure

• Graded on the basis of Approximately 8 Homeworks, which will be due Wednesday of week following day given out (Monday or Wednesday)

• Plus one small project at the end of class

• No finals or written exams

• All material will be placed on World Wide Web (WWW)

• Preference given to work returned on the Web—optional lecture will be given on how to use WWW

Definition of a Matrix

• Consider matrix with elements A_{ij}

• It is an operator A in a vector space of dimension n where A is n × n.

• There are n^{2} numbers A_{ij} defining A.

• The vector space is spanned by a “complete basis” \underline{e}\underline{k}, k = 1 … n.

• One choice of \underline{e}_{k} is

\underline{e}_{1} = (1, 0, 0, … 0)
\underline{e}_{2} = (0, 1, … 0)
\underline{e}_{n} = (0, 0, … 1)

The Definition of a Vector

• Any vector \underline{x} = (x_{1}, x_{2}, … x_{n}) can be written in terms of the complete basis

\underline{x} = _{k}^{∑} x_{k}\underline{e}_{k}

• A_{ij} is representation of A in the basis \{\underline{e}_{k}\}

• The operator A operating on vector \underline{x} is

\underline{y} = A\underline{x}
\underline{y} = _{l}^{∑} y_{l} \underline{e}_{l}
\underline{x} = _{k}^{∑} x_{k} \underline{e}_{k}
Components are y_{l} = _{k}^{∑} A_{lk} x_{k}

• Physicists often write \underline{x} = |x > called a ket

The Definition of Scalar Products

and Orthonormality

• \underline{x}^{T}A\underline{x} is a number (scalar) written by physicists ⟨y|A|x⟩ where ⟨y| is called a Bra.

• The Bra-ket is ⟨y|x⟩ or scalar product

• Therefore,

A_{lk} = \underline{e}_{l}^{T} A \underline{e}_{k}
= ⟨e_{l}|A|e_{k}⟩

The components of A with respect to basis \{\underline{e}_{k}\} • The basis {e_{k}} is orthonormal

⟨\underline{e}_{k}|\underline{e}_{l}⟩ = δ_{lk}
= 0  l ≠ k
= 1  l = k

• δ_{lk} = 0 unless l = k when δ_{lk} = 1 is Kronecker delta

Types of Matrices

• Useful Symbols A^{T} = Transpose (A^{T} )_{ij} = A_{ji} A^{∗} = Complex Conjugate (A^{∗})_{ij} = A_{ij}^{∗} A^{+} = Hermitean Conjugate (A^{+})_{ij} = A_{ji}^{∗}

• A = A^{+} Hermitean Matrix

• In physics, all nice matrices are hermitean—we can understand why when we study eigenvalues and eigenvectors.

• Hermitean operators have a complete set of eigenvectors with real eigenvalues.

Eigenvectors and Eigenvalues

• Suppose A\underline{v} = λ\underline{v} (\underline{v} is not all zeros), or A|v >= λ|v >, where \underline{v} is an eigenvector of A with eigenvalue λ.

• Suppose \underline{e}_{1} = (1, 0, 0 … 0) is an eigenvector of A with eigenvalue λ_{1}.

A\underline{e}_{1} = λ_{1}\underline{e}_{1} = _{l}^{∑} (λ_{1}δ_{l1}) \underline{e}_{l}
λ_{1}δ_{l1} = _{k}^{∑} A_{l1}δ_{k1}

A = [ λ_{1} 0 0  ANY 0  VALUE 0 ⋮ ]

Structure of Matrix

• If A is also symmetric (Hermitean)

A = [ \begin{matrix}λ_{1} & 0 & 0 & 0 & 0 & … & 0 \\ 0 \\ 0 \\ 0 & & A & N & Y \\ 0 & & V & A & L & U & E \\ 0 \end{matrix} ]

• If \underline{e}_{2} … \underline{e}_{n} are also eigenvectors with eigenvalues λ_{2} … λ_{n},

\underline{e}_{2} = (0, 1, 0, … 0)
\underline{e}_{n} = (0, 0, … 1)

Then

A = [ \begin{matrix}λ_{1} \\ & λ_{2} & & 0 \\ & & λ_{3} \\ & 0 & & ... \\ & & & & λ_{n} \end{matrix} ]

A is diagonal with diagonal entries λ_{i}.

CPS615MATRIX, G.C.Fox, Oct 95  8  gcf@npac.syr.edu

Equations for Eigenvalues—I

• If A\underline{v} = λ\underline{v} (A − λI)\underline{v} = 0

• I is identity matrix

• B = A − λI. Then we have

B\underline{v} = 0 and \underline{v} ≠ [0, … 0]
b_{11}v_{1} + b_{12}v_{2} … +b_{1n}v_{n} = 0
⋮
b_{n1}v_{1} . . . . . . b_{nn}v_{n} = 0

• n linear equations for n unknowns v_{i}. This will normally only have solution

v_{1} = v_{2} = … = v_{k} = … = v_{n} = 0

Equations for Eigenvalues—II

• Condition for nontrivial \underline{v} to solve B\underline{v} = 0 is: determinant (B) = 0

• This corresponds to

• n = 2  det B = b_{11} b_{22} − b_{12}b_{21}
• n = 3  det B = b_{11} det [ \begin{matrix}b_{22} & b_{23} \\ b_{32} & b_{33} \end{matrix} ]
− b_{12} det [ \begin{matrix}b_{21} & b_{23} \\ b_{31} & b_{33} \end{matrix} ]
+ b_{13} det [ \begin{matrix}b_{22} & b_{23} \\ b_{32} & b_{33} \end{matrix} ]

•and so on recursively

Polynomial Equations for Eigenvalues

• det (A − λI) =

det [ \begin{matrix}a_{11} − λ & a_{12} \\ a_{21} & a_{22} − λ \\ & a_{32} \\ & & a_{nn} − λ \end{matrix} ]

• so det (A − λI) = 0 is a polynomial of degree n in λ. Coefficient of λ^{n} is (−1)^{n}, i.e., always nonzero.

• On general principles, we will ALWAYS have n solutions

λ = λ_{1}, λ_{2}, … λ_{n}

of det (A − λI) = 0

General Properties of Eigenvalues

• The solutions are usually COMPLEX and in special cases degenerate

• e.g., matrix A = I, the unit matrix has n degenerate eigenvalues λ_{k} = 1.

• Now consider special case when A is real and symmetric, or more generally, complex but hermitean when we will find eigenvalues are real.

Eigenvalues of Hermitean Matrices—I

• For Hermitean A, suppose

Av_{k} = λ_{k}v_{k}
A_{ij}v_{k})_{j} = λ_{k} v_{k})_{i}

• Multiply both sides by v_{k})_{i}^{∗} and sum over i = 1 … n

•

⟨v_{k}|A|v_{k}⟩ = λ_{k}⟨v_{k}|v_{k}⟩ or
n ; ∑ ; i,j=1 v_{k})_{i}^{∗} A_{ij} v_{k})_{j} = λ_{k} _{i}^{∑} v_{k})_{i}^{∗} v_{k})_{i}

• Write this LHS = λ_{k} RHS. The right hand side (RHS) is clearly real and nonzero

RHS = _{i}^{∑} |v_{k})_{i}|^{2}
= ⟨v_{k}|v_{k}⟩

Eigenvalues of Hermitean Matrices—II

• On the other hand, complex conjugate equation,

LHS = n ; ∑ ; i,j=1 v_{k})_{i}^{∗} A_{ij} v_{k})_{j}
LHS^{∗} = n ; ∑ ; i,j=1 v_{k})_{i} \underbrace{A_{ij}^{∗}}_{as hermitean A_{i,j}^{∗}=A_{ji}} v_{k})_{j}^{∗}
= n ; ∑ ; i,j=1 v_{k})_{i} A_{ji} v_{k})_{j}^{∗}

• Now Interchange i and j labels

LHS* = n ; ∑ ; i,j=1 v_{k})_{i}^{∗} A_{ij} v_{k})_{j} = LHS

Therefore, LHS is real

• Formally, we can show this for

⟨x|A|y⟩^{∗} = ⟨y|A^{+}|x⟩
= ⟨y|A|x⟩if A Hermitean

• Taking ⟨x| = ⟨y|, we see ⟨x|A|x⟩^{∗} = ⟨x|A|x⟩ if A Hermitean

• It follows that

λ_{k} = \frac{∑_{i,j=1}^{n} v_{k}^{∗})_{i} A_{ij} v_{k})_{j}}{∑_{i=1}^{n} v_{k}^{∗})_{i} v_{k})_{i}}

is ALWAYS real

CPS615MATRIX, G.C.Fox, Oct 95  14  gcf@npac.syr.edu

Orthogonality of Eigenvectors

of Hermitean Matrices

• Now take any two eigenvectors/values

A|v_{1} > = λ_{1}|v_{1} >
A|v_{2} > = λ_{2}|v_{2} >
⟨v_{2}|A|v_{1}⟩ = λ_{1}⟨v_{2}|v_{1}⟩ (α)
⟨v_{1}|A|v_{2}⟩ = λ_{2}⟨v_{1}|v_{2}⟩

• Take complex conjugate of this equation

⟨v_{2}|A^{+}|v_{1}⟩ = λ_{2}^{∗}⟨v_{2}|v_{1}⟩
⟨v_{2}|A|v_{1}⟩ = λ_{2}⟨v_{2}|v_{1}⟩ (β)
as A = A^{+} λ_{2} = λ_{2}^{∗}

• Subtract (α) and (β)

(λ_{1} − λ_{2}) ⟨v_{2}|v_{1}⟩ = 0

• either λ_{1} = λ_{2} or ⟨v_{2}|v_{1}⟩ = 0 so eigenvectors of different eigenvalues are orthogonal.

• Note if |v_{1} > and |v_{2} > are distinct eigenvectors of same degenerate eigenvalue λ_{1} = λ_{2}, then we can replace |v_{2} > by |v′_{2} > where |v′_{2} > is a linear combination of |v_{1} > and |v_{2} >, and < v_{1}|v′_{2} >= 0.

Orthonormal Eigenvectors and

Unitary Transformation Matrices

• We choose to normalize each eigenvector

⟨v_{k}|v_{k}⟩ = n ; ∑ ; i=1 v_{k}^{∗})_{i} v_{k} )_{i} = 1

• and we have proved

⟨v_{k}|v_{l}⟩ = 0 l ≠ k

• eigenvectors are ORTHOnormal whether or not eigenvalues are degenerate

• Define a matrix U so that columns of U^{+} are orthonormal eigenvectors

U^{+} = [ \begin{matrix}v_{1})_{1} & v_{n})_{1} \\ v_{1}) & 2 \\ v_{1}) & 3 \\ v_{1})_{n} & v_{n})_{n} \end{matrix} ]

• U is UNITARY U^{+}U = I

• This unitary condition is exactly the orthonormality condition.

Diagonal Form of General Hermitean Matrix—I

• Consider transformed matrix Ã using U defined previously

Ã = U^{∗} [ \begin{matrix}λ_{1} \\ & λ_{2} & 0 \\ 0 \\ & & & λ_{n} \end{matrix} ] U

↑

Diagonal

• By inspection

Ã|v_{k} >= λ_{k}|v_{k} >

i.e., Ã and A give same result on each eigenvector.

Diagonal Form of General Hermitean Matrix—II

• But ANY vector

|x >= n ; ∑ ; k=1 x_{k}|v_{k} >

as |v_{k} > must be complete as ORTHOnormal and n of them

• Then

Ã|x > = n ; ∑ ; k=1 x_{k}Ã|v_{k} >
= n ; ∑ ; k=1 x_{k}A|v_{k} >
= A|x >
and we have proven Ã ≡ A

• Thus,

A = U^{+} Diag. [ \begin{matrix}λ_{1} \\ & . & . & . \\ & & & & λ_{n} \end{matrix} ] U

Eigenvectors and Eigenvalues of

Symmetric Matrices

• Above we have taken hermitean operator A^{+} = A. If A real then A^{+} = A^{T} and A^{T} = A.

• Then eigen\underline{vectors} can be taken to be real as equation (A − λ_{i}I)|v_{k} > has real coefficients (we showed λ_{k} real).

• Then U is real

U^{+} = U^{T}
U^{T}U = I
U is ORTHOGONAL

Finite and Infinite Dimensional

Matrices as Operators

• We have discussed matrices. These are “operators” in finite (n) dimensional linear space.

• You can generalize to infinite dimensional space.

— e.g., space of “suitable” functions f(x)

• \frac{d}{dx} is an operator!

• \frac{d}{dx}f(x) = g(x) is another function.

— e.g., another operator

\frac{d}{dx} as an Operator and Its Scalar Products

• Consider the Integral

I = ∫_{a}^{b} g(x)\frac{d}{dx}f(x)dx = ⟨g|D|f⟩

where D is operator \frac{d}{dx}.

• Integrate by parts.

I = − ∫_{a}^{b} f(x) \frac{d}{dx} g(x) dx
+ [f(x)g(x)]_{a}^{b}

• The last term [f(x)g(x)]_{a}^{b} = f(b) g(b) − f(a) g(a)

• We need to get rid of this surface (endpoint) term.

i\frac{d}{dx} as a Hermitean Operator

• Suppose we consider periodic functions

f(b) = f(a)
g(b) = g(a)

— i.e., impose particular boundary conditions

• Then surface term vanishes and

∫_{a}^{b} g(x)\frac{d}{dx}f(x) dx = − ∫_{a}^{b} f(x)\frac{d}{dx}g(x) dx

• The operator L = i\frac{d}{dx} with \underline{boundary conditions} f(a) = f(b) is Hermitean. The i in L cancels minus sign.

• Must include boundary conditions when discussing operators in infinite dimensional space

The Laplacian as a (Matrix) Operator

• ∇^{2} = \frac{d^{2}}{dx^{2}} = −L L

• (∇^{2})^{†} = −L^{†}L^{†} = −L L = ∇^{2}

• Note: (A B)^{†} = B^{†}A^{†} in general

• Thus, ∇^{2} (\frac{d^{2}}{dx^{2}} in one dimension) is Hermitean (symmetric) operator

Mapping of Function Spaces

to a Finite Number of Dimensions

• In solving differential equations numerically, we take infinite dimensional operators on space of functions and convert them to finite matrix operators in space of n dimensional vectors.

• The different numerical approaches correspond to different ways of mapping the infinite dimensional space to a finite space—there is no unique way and some are much better than others.