Projection

Projection separates a vector into a part that we keep and a part that we discard. In orthogonal projection, the discarded part is perpendicular to the target subspace.

• For one direction u, the vector v decomposes as v = v_⊥ + v_∥, where v_∥ lies along u
• For a matrix U with independent columns, the projection of v onto col(U) is U (Uᵀ U)⁻¹ Uᵀ v
• The particular case U = [ u ] collapses the matrix formula back to projection onto a single direction
• Orthogonal projection is a special case of general projection, where the residual lies in a chosen complementary subspace
• Projection matrices are idempotent: applying P twice gives the same result as applying it once
• The matrix P = U (Uᵀ U)⁻¹ Uᵀ can also be read as a serial transformation from ℝ³ to ℝ² and back to ℝ³

The chapter moves from vector form to matrix form, then compares orthogonal and general projection side by side and ends with the transformation view of the projection matrix.

Orthogonal projection chooses the projected vector so that the residual is perpendicular to the target subspace. That perpendicularity makes the projected vector unique and gives the standard formula P = U (Uᵀ U)⁻¹ Uᵀ.

The chapter is available as a PDF. Page links below are best-effort: most browsers support them but some viewers may ignore the page hint.

Because once a vector has already been projected into the target subspace, projecting it again does nothing new. The first application of P removes the residual, and the second application has nothing left to remove.

Quick feedback helps us improve the site.