Matrices transform space

The idea

A matrix is a compact rule for transforming vectors: its columns show exactly where the basis vectors land. Scaling, rotation, reflection, and shear all become a single matrix multiplication — the same trick used throughout graphics, machine learning, and other data transformations.

Matrix multiplication composes transformations. If $B$ acts first and $A$ acts second, then $A(Bx)=(AB)x$ : the rightmost matrix is applied first. This also explains why two linear layers without a nonlinearity collapse to one matrix product.

Real-world: stretching a game sprite

A 2D game doubles a character's width but keeps its height, so every point $(x, y)$ is transformed to $(2x, y)$ by the matrix $\begin{bmatrix} 2 & 0 \\ 0 & 1 \end{bmatrix}$ . The sprite point $(3, 4)$ moves to $(6, 4)$ — the $x$ -coordinate doubles while $y$ stays unchanged.

The matrix scales the vector’s x-coordinate by 2; y stays fixed.

Worked example

Apply the transform in one calculation:

\begin{bmatrix}2&0\\0&1\end{bmatrix} \begin{bmatrix}2\\2\end{bmatrix} = \begin{bmatrix} 2(2)+0(2)\\ 0(2)+1(2) \end{bmatrix} = \begin{bmatrix}4\\2\end{bmatrix}.

Cheat sheet

$\begin{bmatrix} a & b \\ c & d \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} ax+by \\ cx+dy \end{bmatrix}$

Each column of the matrix is where a basis vector ( $\begin{bmatrix} 1 \\ 0 \end{bmatrix}$ or $\begin{bmatrix} 0 \\ 1 \end{bmatrix}$ ) lands after the transform.
The identity matrix $\begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}$ leaves every vector unchanged.
A diagonal matrix scales each axis independently by its diagonal entry.
Matrix multiplication composes transformations right-to-left: $ABx=A(Bx)$ applies $B$ first, then $A$ .
Matrix multiplication applies the same linear transformation consistently to every point.