3. Cheatsheet

Vector
Scalar Vector Multiplication
Dot Product
Cross Product ()
Returns a vector orthogonal to the two vectors
Vector Space
Closure under addition
Closure under scalar multiplication
Commutativity of addition
Associativity of addition
Additive identity
Additive inverse
Scalar multiplication (compatibility)
Distributivity over vector addition
Distributivity over scalar addition
Multiplicative identity
Subspace

Non-emptiness

Closure under addition

Closure under scalar multiplication

 A subspace is a subset of a vector space that is itself a vector space, satisfying the same axioms as the original. If is a vector space in , then the subspace is always contained in , meaning
Vector Addition
Dot Product
Orthogonality Angle between the two vectors is
Angle between vectors
Norm (Manhattan)
Norm (Euclidean)
Distance (Manhattan)
Distance (Euclidean)
Projection
Linear Independence

A set of vectors is linearly independent if no vector in the set can be written as a linear combination of the others

A set of vectors is linearly independent if the only solution to the the equation

is

Transformation
  • Additivity

  • Homogeneity
  • Surjective (onto)

Every element in is the image of at least one element in . The transformation covers the entire codomain.

  • Injective (one-to-one)

Different inputs in map to different outputs in .The transformation is information-preserving — doesn’t collapse distinct vectors together

Or equivalently:
Domain
Set of all input vectors that the transformation acts on
Codomain
Set of all possible output vectors to which elements of the domain are mapped under the transformation
Transpose
Image

The image of a transformation is the set of all possible outputs for :

  • It is a subspace of the codomain

  • If is represented by a matrix , the image of is the column space of

Preimage
The preimage of a transformation refers to the set of all elements in the domain that map to a particular element or subset in the codomain
Span The span of a set of vectors is the collection of all possible linear combinations of those vectors
Composition
  • Additivity

  • Homogeneity

  • Identity Transformation
Column Space (Range)

Or equivalently

 The column space (or range) of a matrix is the set of all linear combinations of its columns
Determinant

The determinant of a square matrix A measure of the "scale factor" by which the matrix A transforms a space

    • does not collapse the space
    • has full rank
    • ‘s columns are linearly independent
    • is invertable

    • collapses the space into lower dimension
    • does not have full rank
    • ‘s columns are linearly dependent
    • is non-invertable (singular)
Invertibility

Basis

Linear Independence

Spanning
  • A basis of a vector space is a set of linearly independent vectors that span the space
  • Every vector in can be uniquely written as a linear combination of the basis vectors

E.g.:
Dimension

Number of linearly independent vectors (basis) in a vector space

is a line through the origin in
is a plane through the origin in
is a -dimensional flat subspace of

  • has dimension 2:

    • A basis:

  • has dimension 3:

    • A basis:
Rank
  • The rank of a matrix is the dimension of its column space (or row space)
  • Number of linearly independent columns (or rows)
Eigen

Set of all nonzero vectors such that when the transformation represented by matrix is applied to , the result is a scaled version of itself

These vectors lie along directions that are preserved by the transformation:

  • : stretched
  • : shrunk
  • : flipped
  • : stay the same
Null Space (kernel) The null space of a matrix is the set of all input vectors that get mapped to the zero vector when you multiply them by
Identity Matrix
Matrix Inverse
RREF
  1. Row Swapping (Interchange)
  1. Row Scaling (Multiplication)
  1. Row Addition (Replacement)

3.1. Matrix

3.1.1. Matrix Vector Product

3.1.2. Matrix Multiplication
: Row Representation
: Column Representation

3.1.3. Transpose

3.2. Matrix Factorization

3.2.1. LU Decomposition

where:

  1. Matrix (with )
  1. Matrix (with )

see QR Decomposition

3.3. Singular Value Decomposition (SVD)
Example
  1. Null Space

The null-space of , denoted , consists of all vectors such that . The set of all such vectors is the pre-image of the zero vector under the transformation defined by . In other words, , which represents the set of vectors that maps to zero.

To find the null space of the matrix , we can use the row-reduced echelon form (RREF). By augmenting the matrix with a zero column and performing row operations, we reduce it to the form:

The dimension of the null-space is the number of vectors in this basis, which is 2. This is important because the dimension of the null space gives us insight into how many degrees of freedom exist in the system of equations

  1. Column Space
  1. Basis
  1. Rank

Number of vector in the basis of our column space