Some notes on Linear Algebra

Span and Basis

• The ‘span’ of two vectors is the set of all of their linear combinations. $a\vec{v}+b\vec{u}$ where $a$ and $b$ are scalars.
  • It’s basically the set of all the vectors reachable with $\vec{u}$ and $\vec{u}$ as basis vectors.
  • The span of a single vector is simply the line through the vector in the entire plane, i.e., $\lambda \vec{v}$
  • The span of 2 vectors in a 3D space is the plane passing through them - given the vectors are not both 0 or same vector but scaled differently. This redundancy is called linear dependence - or that one vector is a linear combination of the others. In that case, their span is a 1D line.
• Basis of a vector space is a set of linearly independent vectors that span the full space. In 2D, we have $\hat{i}$ and $\hat{j}$ unit vectors forming a basis. Add $\hat{k}$ to get 3D space.

Linear Transformations

• Linear if - all lines remain lines - and the origin remains fixed in place. ( No bias term)
  • Or more formally, if $f(\vec{a}+\vec{b})=f(\vec{a})+f(\vec{b})$ and $f(c\vec{a})=cf(\vec{a})$ then $f$ is a linear transform.
• Given some basis vectors, any vector under any linear transform still is the same linear combination of the original basis vectors now transformed. So if $\vec{v}=5\hat{i}-2\hat{j}$, it still remains the same, with now the basis vectors $\vec{i}$ and $\vec{j}$ are transformed.
• Matrix $\times$ Vector is basically a transform on the basis of the vector.
• Affine transformation is Linear transformation plus possibility of shifting the origin too - so can have a bias term. So Linear transforms are a subset of affine transform.

Eigenvectors and Eigenvalues

• eigenvector is a vector that under a transformation $\vec{A}$ remains on its span. If its magnitude changes by a scalar $\lambda$, then $\lambda$ is called the eigenvalue. $$A\overrightarrow{\mathbf{v}}=\lambda \overrightarrow{\mathbf{v}}$$ $$A\overrightarrow{\mathbf{v}}-\lambda I\overrightarrow{\mathbf{v}}=0$$ $$(A-\lambda I)\overrightarrow{\mathrm{v}}=0$$ $$\det (A-\lambda I)=0$$
• Eigenbasis - when the basis vectors themselves are eigenvectors under the given transformation.
  • This means if we make our basis into two eigenvectors, the transformation matrix will be diagonal and will be trivial to take powers of.