Linear Algebra and NumPy

In the following discussion, the vector space is of dimension $d$ and has basis $x_1, x_2, \dots, x_d$

Vector Space¶

a vector space over the real numbers is a set $V$ together with two binary operations $+$ (mapping $V \times V$ to $V$) and $\cdot$ (mapping $\R \times V$ to $V$) satisfying the following axioms for any $x, y, z \in V$ and $a, b \in \R$

• closure: $x + y \in V$ and $a \cdot x = ax \in V$
• associativity of addition: $(x + y) + z = x + (y + z)$
• transitivity of addition: $x + y = y + x$
• negation: there exists a $(-x)$ such that $x + (-x) = 0$
• zero element: $0 \in V$ such that $0 + x = x + 0 = x$
• associativity of scalar multiplication: $a(bx) = b(ax) = (ab)x$
• multiplication by one: $1v = v$
• distributivity: $a(x+y) = ax + ay$ and $(a+b)x = ax + bx$

Vector¶

Any vector $v$ in the vector space can be written uniquely as the following for some real numbers $\alpha_1, \alpha_2, \ldots$

Therefore, to represent $v$ on a computer, it suffices to store $\alpha_1$, $\alpha_2$, $\ldots$, and $\alpha_d$. We store these alphas in an array and this gets us back to our CS-style notion of what a vector is.

Linear Map¶

We say a function $F$ from a vector space $U$ to a vector space $V$ is a linear map if for any $x, y \in U$ and any $a \in \R$,

Note if we want to define a linear map, it suffices to just write out $F(x_i)$ for all basis. Say $|x_i| = d =m, |F(x_i)| = n$, we can then represent a linear map with $m\times n$ number: each $F(x_i)$ needs $n$ numbers and there are $m$ $F(x_i)$

Matrix¶

We have basically introduced the “matrix” in above. We call this two-dimensional-array representation of a linear map a matrix.

We use multiplication to denote the effect of a matrix operating on a vector (this is equivalent to applying a multilinear map as a function). E.g. if $F$ is the multilinear map corresponding to matrix $A$:

We can add two matrices, scale a matrix by a scalar, and these two operations follow the “axioms” described above for a vector space. This means that the set of matrices $\R^{n \times m}$ is itself a vector space. (This was covered in MATH3360)

Broadcasting¶

When operate on vectors / matrices, numpy broadcasts by acting as if there are infinite $1\times$ to the left of dimension:

if dimension of array not same:
    append newaxis to the lesser one until same
    if on a specific axis, one array has dim 1 and the other has > 1:
        broadcast that 1 axis to match dimension
    else:
        fail

However, it is good practice for you to always manually add dimension before broadcasting.

x = numpy.array([2,3])
x1 = x[numpy.newaxis,:]