GraphMath

Projection

From one direction to a subspace: orthogonal and general projection

How do we split a vector into a projected part and a residual?

Projection decomposes a vector into two parts: one part lying in the chosen subspace and the other lying in a complementary residual subspace. This chapter starts with orthogonal projection onto one vector, extends it to the columns of a matrix, compares orthogonal and general projection, develops the main properties of projection matrices and ends with the serial-transformation view of P = U(UᵀU)⁻¹Uᵀ.

Open chapter PDF Key ideas Chapter index Back to Linear Algebra

Key ideas

Projection separates a vector into a part that we keep and a part that we discard. In orthogonal projection, the discarded part is perpendicular to the target subspace.

  • For one direction u, the vector v decomposes as v = v⊥ + v∥, where v∥ is a scalar multiple of u and v⊥ is orthogonal to u
  • For a matrix U with independent columns, the orthogonal projection of v onto col(U) is U(UᵀU)⁻¹Uᵀv
  • General projection also projects onto col(U), but its residual lies in null(Sᵀ), where SᵀU is invertible
  • Orthogonal projection is the special case S = U, so the residual lies in null(Uᵀ), the orthogonal complement of col(U)
  • Every projection matrix is idempotent: P² = P; orthogonal projection matrices are also symmetric
  • The only eigenvalues of a projection matrix are 0 and 1, corresponding to discarded and preserved directions
  • The complementary projection I − P keeps the residual component instead of the projected component
  • P = U(UᵀU)⁻¹Uᵀ can be read as a serial transformation from ℝ³ to ℝ² and back to ℝ³

The chapter moves from vector form to matrix form, compares orthogonal and general projection side by side, develops the algebraic properties of P and ends with the transformation view of the projection matrix.

What makes orthogonal projection special?

Orthogonal projection chooses the projected vector so that the residual is perpendicular to the target subspace. That perpendicularity makes the projected vector unique and gives the standard formula P = U (Uᵀ U)⁻¹ Uᵀ.

Related chapters

  • The four subspaces develops the orthogonal decompositions behind col(A), null(A), row(A) and ℓ-null(A)
  • Least squares, best solution and projection uses projection onto col(A) to explain best-fit solutions of inconsistent systems
  • QRF, Gram-Schmidt, part 1 develops orthogonal vectors, projection ideas and the structure used in Gram-Schmidt
  • Linear equations introduces consistent and inconsistent systems, which motivate projection onto reachable outputs
  • Inverse & Pseudoinverse uses projection operators such as A⁺A and AA⁺ to describe what part of a map is preserved
  • 3D rotation uses projection onto the axis and onto the orthogonal plane as the starting decomposition for Rodrigues algorithm

Chapter contents

The chapter is available as a PDF. Page links below are best-effort: most browsers support them but some viewers may ignore the page hint.

Topic Pages
Orthogonal projection of vector v onto direction of vector u 1–2
Orthogonal projection of vector v onto columns of U = [ u₁, u₂, ..., uᵣ ] 2–4
Orthogonal vs general projection (optional reading) 4–8
Properties of P 8–10
P = U(UᵀU)⁻¹Uᵀ as a serial transformation 11
Open PDF

Why are the only projection eigenvalues 0 and 1?

Because P² = P. If Pv = λv, then applying P again gives λ²v = λv, so λ² = λ. Therefore λ is either 0, for directions eliminated by the projection, or 1, for directions preserved by it.

Was this chapter helpful?

Quick feedback helps us improve the site.

Yes No