[Linear Algebra] 2. Basis vector, linear combination, span and dependent

Basis vectors

In previous chapter, we describe vector as coordinate, where each coordinate value is location (or movement) of $x$-axis or $y$-axis.

 

There is another view of coordinate value:

If there is a pair of number $\begin{bmatrix} v_1 \\ v_2 \end{bmatrix}$ that describe a vector $\mathbf{\vec{v}}$, think each coordinate value as a scalar.

In XY-coordinate system, there are two special vectors: 1). pointing to the right with length $1$, denoted $\mathbf{\hat{i}}$ and called i-hat or unit vector in $x$-direction. 2). pointing straight up with length $1$, denoted $\mathbf{\hat{j}}$ and called j-hat or unit vector in the $y$-direction.

Let's think $v_1$ as a scalar that scales $\mathbf{\hat{i}}$. and $v_2$ as scalar that scales $\mathbf{\hat{j}}$. In this sense, the vectors are the sum of two scaled vectors.

 

$\mathbf{\hat{i}}$ and $\mathbf{\hat{j}}$ are the basis vectors of XY-coordinate system. This means that when we think coordinate values as scalars, the basis vectors are what those scalars actually scale.

 

What if we chose different basis vectors?

Instead of $\mathbf{\hat{i}}$ and $\mathbf{\hat{j}}$, let's think about another basis vectors. We could choose different basis vectors, and by that, we get completely different coordinate system. A new pair of basis vectors gives valid way to go back and forth between list-of-number and arrow, but it is definitely different from one that uses standard basis $\mathbf{\hat{i}}$ and $\mathbf{\hat{j}}$. In later chapter, we will think about the relation between different coordinate systems. So for right now, we implicitly use $\mathbf{\hat{i}}$ and $\mathbf{\hat{j}}$ as basis vectors to describe vectors numerically.

Linear combination

Scaling two vectors $\mathbf{\vec{v}}$ and $\mathbf{\vec{w}}$ and adding them is linear combination of two vectors.

$$
\mathbf{\vec{v}} =
\begin{bmatrix} v_1 \\ v_2 \end{bmatrix}
\
\mathbf{\vec{w}} =
\begin{bmatrix} w_1 \\ w_2 \end{bmatrix}
\qquad
a \mathbf{\vec{v}} + b \mathbf{\vec{w}} =
a\begin{bmatrix} v_1 \\ v_2 \end{bmatrix} +
b\begin{bmatrix} w_1 \\ w_2 \end{bmatrix}
$$

Span

If both scalars, $a$ and $b$, range freely, what are all possible vectors got by linear combination of $\mathbf{\vec{v}}$ and $\mathbf{\vec{w}}$? There are three possible cases that can happen:

1. For most pairs of vectors, it can reach every possible point in the plane.
2. If vectors are lined up, it reach single line pass through the origin.
3. If both vectors are zero, it just be stuck at the origin.

The set of all possible vectors, reached by a linear combination of given vectors $\mathbf{\vec{v}}$ and $\mathbf{\vec{w}}$, is called the span of two vectors $\mathbf{\vec{v}}$ and $\mathbf{\vec{w}}$. In other words, the span of two vectors is the set of all possible linear combinations of these vectors.

 

The span of two vectors is basically a way of asking, "What are all the possible vectors you can reach using only these two fundamental operations, vector addition and scalar multiplication?".

Linear dependent & Linear independent

Randomly sampled three 3-D vectors($\mathbf{\vec{u}}$, $\mathbf{\vec{v}}$ and $\mathbf{\vec{w}}$) are expected to span 3-D space. But it could span 2-D plane, if $\mathbf{\vec{u}}$ is on the span of $\mathbf{\vec{v}}$, $\mathbf{\vec{w}}$(plane).  Like this, if there is a vector like $\mathbf{\vec{u}}$ that does not reduce span when it's removed, it's linear dependent. We can rephrase like this: "one of the vectors can be expressed as linear combination of the others".

 

On the other hand, if every each vector adds another dimension to the span, it's linear independent.