Definition: Two vectors (\mathbf{u}) and (\mathbf{v}) are orthogonal if their inner product (dot product) is zero: [\mathbf{u} \perp \mathbf{v} \iff \langle \mathbf{u}, \mathbf{v} \rangle = 0]
In (\mathbb{R}^n) with the standard dot product: [\mathbf{u} \cdot \mathbf{v} = \sum_{i=1}^n u_i v_i = 0]
Orthogonal projection: The projection of (\mathbf{v}) onto a nonzero vector (\mathbf{u}) is: [\text{proj}_{\mathbf{u}}(\mathbf{v}) = \frac{\mathbf{v} \cdot \mathbf{u}}{\mathbf{u} \cdot \mathbf{u}} \mathbf{u}]
This gives the closest point on the line spanned by (\mathbf{u}) to (\mathbf{v}). The error (\mathbf{v} - \text{proj}_{\mathbf{u}}(\mathbf{v})) is orthogonal to (\mathbf{u}).
Example: The projection of ((3,4)) onto ((1,0)) is ((3,0)). The error ((0,4)) is orthogonal to ((1,0)).
Orthogonal decomposition: Any vector (\mathbf{v}) can be written as (\mathbf{v} = \mathbf{v}W + \mathbf{v}{W^\perp}) where (\mathbf{v}W) lies in a subspace (W) and (\mathbf{v}{W^\perp}) is orthogonal to every vector in (W).
**Definition:** Two vectors \(\mathbf{u}\) and \(\mathbf{v}\) are *orthogonal* if their inner product (dot product) is zero:
\[\mathbf{u} \perp \mathbf{v} \iff \langle \mathbf{u}, \mathbf{v} \rangle = 0\]
In \(\mathbb{R}^n\) with the standard dot product:
\[\mathbf{u} \cdot \mathbf{v} = \sum_{i=1}^n u_i v_i = 0\]
**Orthogonal projection:** The projection of \(\mathbf{v}\) onto a nonzero vector \(\mathbf{u}\) is:
\[\text{proj}_{\mathbf{u}}(\mathbf{v}) = \frac{\mathbf{v} \cdot \mathbf{u}}{\mathbf{u} \cdot \mathbf{u}} \mathbf{u}\]
This gives the closest point on the line spanned by \(\mathbf{u}\) to \(\mathbf{v}\). The error \(\mathbf{v} - \text{proj}_{\mathbf{u}}(\mathbf{v})\) is orthogonal to \(\mathbf{u}\).
**Example:** The projection of \((3,4)\) onto \((1,0)\) is \((3,0)\). The error \((0,4)\) is orthogonal to \((1,0)\).
**Orthogonal decomposition:** Any vector \(\mathbf{v}\) can be written as \(\mathbf{v} = \mathbf{v}_W + \mathbf{v}_{W^\perp}\) where \(\mathbf{v}_W\) lies in a subspace \(W\) and \(\mathbf{v}_{W^\perp}\) is orthogonal to every vector in \(W\).