Singular Vector

Intuition (SVD)

• The SVD captures how a matrix maps between two subspaces:
• Input (domain): the row space (equivalently, the space spanned by columns of ).
• Output (codomain): the column space (the space spanned by columns of ).

• Even a single singular triplet (, , ) can reveal a lot:
• = “which input direction matters”
• = “how strongly it is scaled”
• = “which output direction it becomes”

Left singular vectors ()

• Each left singular vector is an orthonormal direction in the column space of .

• Intuitively: the dominant output patterns produced by .

Right singular vectors ()

• Each right singular vector is an orthonormal direction in the row space of (the input directions).

• Why does the SVD use (not ) in ?
• Because we want the matrix to act on an input vector as:
• first rotate/express in the basis of right singular vectors via
• then scale by
• then rotate into the output space via .

• can be viewed as spanning directions that are mapped into the column space of under .

Singular values ()

• Singular values quantify the strength of along each pair of singular directions.

• Geometric view: maps the unit sphere to an ellipsoid whose axis lengths are (axes aligned with in input space and in output space).