Orthogonality: small idea, used everywhere

What \(u^\top v=0\) means, what \(Q^\top Q=I\) means, and why symmetric eigenvectors are orthogonal.

1. Vectors: orthogonal vs orthonormal

Two vectors are orthogonal if their dot product is zero: \(u^\top v = 0\). They meet at 90°.
They are orthonormal if additionally each has length 1: \(\|u\|=\|v\|=1\).

Read \(u^\top v=0\) as: “u-transpose v equals zero”, i.e. “u dot v is zero”. Length is \(\|v\|^2=v^\top v\).

2. Matrices: \(Q\) is orthogonal if its columns are orthonormal

\(Q\) is orthogonal iff \(Q^\top Q = I\), i.e. \(Q^{-1}=Q^\top\). Columns are orthonormal; for square \(Q\) the rows are too.

Read \(Q^\top Q=I\) as: “Q-transpose Q equals I”. Read \(Q^{-1}=Q^\top\) as: “Q-inverse is Q-transpose” — no elimination needed.

What Q preserves: lengths and angles. \(\|Qx\|=\|x\|\) and \((Qx)^\top(Qy)=x^\top y\). Geometrically: rotations and reflections only.

3. The theorem you keep using

Theorem (symmetric ⇒ orthogonal eigenvectors). If \(S^\top=S\) and \(Sv_i=\lambda_i v_i\), \(Sv_j=\lambda_j v_j\) with \(\lambda_i\neq\lambda_j\), then \(v_i^\top v_j=0\). The spectral theorem extends this: every symmetric \(S\) has a full orthonormal eigenbasis, \(S=PDP^\top\) with \(P^\top P=I\).

This is why \(A^TA\) and \(AA^T\) in lesson 0002 give us orthonormal \(V\) and \(U\) for free. Symmetry manufactures orthogonality.

4. Concrete check (reuse from SVD)

Let \(Q=\begin{bmatrix}0.6&-0.8\\0.8&0.6\end{bmatrix}\) (the \(U\) from lesson 0002).

Columns \(q_1=[0.6,0.8]^T\), \(q_2=[-0.8,0.6]^T\): \(q_1^\top q_2=-0.48+0.48=0\), \(\|q_1\|=\|q_2\|=1\). So \(Q^\top Q=I\). Take \(x=[1,2]^T\): \(\|x\|^2=5\), \(Qx=[-1.0,2.0]^T\), \(\|Qx\|^2=1+4=5\). Length preserved.

5. If the professor says: “Prove it”

1Q preserves dot products

(Qx)ᵀ(Qy) = xᵀQᵀQy = xᵀy; ‖Qx‖ = ‖x‖

Say ► “Q-x transpose Q-y equals x-transpose Q-transpose Q y, which is x-transpose y since Q-transpose Q is I. Put y equal x to get norm Q-x equals norm x.”

↓

2Symmetric eigenvectors are orthogonal

λᵢvᵢᵀvⱼ = (Svᵢ)ᵀvⱼ = vᵢᵀSvⱼ = λⱼvᵢᵀvⱼ ⇒ (λᵢ−λⱼ)vᵢᵀvⱼ = 0

Say ► “Lambda-i times v-i transpose v-j equals S-v-i transpose v-j, which equals v-i transpose S v-j, which is lambda-j times v-i transpose v-j. Since lambda-i differs from lambda-j, v-i transpose v-j must be zero.”

If grilled: “Does orthogonal mean \(Q^\top=Q\)?” → No, it means \(Q^\top=Q^{-1}\). “Are rows orthonormal?” → Yes for square orthogonal \(Q\), since \(QQ^\top=I\) too.

6. Check yourself

Pick an answer — feedback is immediate. Answers matched in length so formatting gives no hints.

Q1. Q-inverse equals?
Q2. When are u,v orthogonal?
Q3. What does Q preserve?

7. Go to the source

Strang §4.4 / §5.1 (Orthogonality, Orthogonal Matrices) or MIT 18.06 Lecture 14–15. Same two facts: \(Q^\top Q=I\) preserves geometry; symmetric matrices get orthonormal eigenvectors.

Prev: 0002 — SVD · 0001 — diagonalization