Vectors

1.1 Vectors

A vector is a one-dimensional array of numbers. In physics and engineering, vectors often represent quantities that have both magnitude and direction, such as force or velocity. In linear algebra, vectors are elements of vector spaces, which are sets equipped with the operations of vector addition and scalar multiplication.

Representation:

For the sake of these notes, we’ll represent vectors as column matrices.

where each \(v_i\) is a component of the vector. The following are some properties of vector addition and scalar multiplication for vectors \(\mathbf{u}\) and \(\mathbf{v}\) and scalars \(c\) and \(d\):

Vector Addition:

To add two vectors, simply add their corresponding components.

Given vectors:

and

Their sum is:

Two vectors in \(\mathbb R^n\) (of the same size) can be added together by adding the corresponding elements, to form another vector in \(\mathbb R^n\), called the sum of the vectors. For example:

Python Vector Addition

Here is where things get tricky in Python. If you try to add a list or tuple, Python does not do the vector addition as we defined above. In the following examples, notice that the two lists concatenate instead of adding by element:

## THIS IS WRONG
a = [1, 20]
b = [22,-3]
c = a+b
c

[1, 20, 22, -3]

## THIS IS ALSO WRONG
a = (1, 20)
b = (22,-3)
c = a+b
c

(1, 20, 22, -3)

To do proper vector math you need either use a special function (we will learn these) or loop over the list. Here is a very simplistic example:

a = (1, 20)
b = (22,-3)
c = []
for i in range(len(a)):
    c.append(a[i] + b[i])
c

[23, 17]

For fun, we can define this operation as a function in case we want to use it later:

def vecadd(a,b):
    """Function to add two equal size vectors."""
    if (len(a) != len(b)):
        raise Exception('Error - vector lengths do not match')
    c = []
    for i in range(len(a)):
        c.append(a[i] + b[i])
    return c

#Lets test it

vecadd(a,b)

[23, 17]

Scalar-Vector Multiplication:

To multiply a vector by a scalar, multiply each component of the vector by that scalar.

Given scalar \(k\) and vector:

The product is:

You can also multiply a scalar by a vector, which is done by multiplying every element of the vector by the scalar.

Scalar-Vector Multiplication in Python

Again, this can be tricky in Python because Python lists do not do what we want. Consider the following example that just concatenates three copies of the vector.

##THIS IS WRONG## 
z = 3
a = [3,-7,10]
c = z*a
c

[3, -7, 10, 3, -7, 10, 3, -7, 10]

Again, in order to do proper vector math in Python you need either use a special function (we will learn these) or loop over the list.

def sv_multiply(z,a):
    c = []
    for i in range(len(a)):
        c.append(z*a[i])
    return c

z = 3
a = [3,-7,10]
sv_multiply(z,a)

[9, -21, 30]

Numpy

Numpy is a common way to represent vectors, and you are suggested to use numpy unless otherwise specified. The benefit of numpy is that it can perform the linear algebra operations listed in the previous section.

For example, the following code uses numpy.array to define a vector of four elements.

import numpy as np
x = np.array([-1, 0, 2, 3.1])
y = np.array([4, 3, 2, 1])
print(x+y)
print(3*x)

[3.  3.  4.  4.1]
[-3.   0.   6.   9.3]

1.2 Vector Spaces

A Vector Space is a set \(V\) of elements called vectors, having operations of addition and scalar multiplication defined on it that satisfy the following conditions (\(\mathbf{u}\), \(\mathbf{v}\), and \(\mathbf{w}\) are arbitrary elements of \(V\), and c and d are scalars.)

• Closed under addition
• Closed under scalar multiplication

The following are some properties of vector addition and scalar multiplication for vectors \(\mathbf{u}\) and \(\mathbf{v}\) and scalars \(c\) and \(d\):

Closure Axioms

1. The sum \(\mathbf{u} + \mathbf{v}\) exists and is an element of \(V\). (\(V\) is closed under addition.)
2. \(c\mathbf{u}\) is an element of \(V\). (\(V\) is closed under scalar multiplication.)

Addition Axioms

3. \(\mathbf{u}+\mathbf{v} = \mathbf{v}+\mathbf{u}\) commutative property
4. \((\mathbf{u}+\mathbf{v}) + \mathbf{w} = \mathbf{u} + (\mathbf{v}+\mathbf{w})\) associative property
5. There exists an element of \(V\), called a zero vector, denoted \(0\), such that \(\mathbf{u}+0 = 0 + \mathbf{u} = \mathbf{u}\)
6. For every element \(\mathbf{u}\) of \(V\), there exists an element called a negative of \(\mathbf{u}\), denoted \(-\mathbf{u}\), such that \(\mathbf{u} + (-\mathbf{u}) = 0\).

Scalar Multiplication Axioms

7. \(c(\mathbf{u}+\mathbf{v}) = c\mathbf{u} + c\mathbf{v}\) distributed properties
8. \((c + d)\mathbf{u} = c\mathbf{u} + d\mathbf{u}\) distributed properties
9. \(c(d\mathbf{u}) = (cd)\mathbf{u}\) distributed properties
10. \(1\mathbf{u} = \mathbf{u}\) scalar multiplication by 1

1.2.1 \(\mathbb R^n\) as a Vector Space

Note that by \(\mathbb R^n\) we denote the \(n\)-tuples of real numbers. For example, \((10,20,3.2)\in \mathbb R^3\), \((0,-0.7)\in\mathbb R^2\), \((1,2,3,500)\in\mathbb R^4\).

There are much more general vectors spaces than \(\mathbb R^n\) - vectos can be almost any type of object as long as it maintains the two above properties. We will get into this concept later in the semester. In the case of \(\mathbb R^n\), the above concepts can be described as follows:

• Closed under addition means that if we add any two real vectors vectors (i.e. \(\mathbf{u}, \mathbf{v} \in \mathbb R^n\)) then the result is also in \(\mathbb R^n\)). Addition in \(\mathbb R^n\) is defined by \((u_1, u_2, \dots, u_n)+(v_1, v_2, \dots, v_n) = (u_1+v_1, u_2+v_2, \dots, u_n+v_n)\). Closure under addition is easy to understand - adding any two real n-vectors there is no way to get a result that is not also a real n-vector. A way to say this mathematically is as follows: if \(\mathbf{u}, \mathbf{v} \in \mathbb R^n\), then \(\mathbf{u}+\mathbf{v} \in \mathbb R^n\)
• Closed under scalar multiplication means that if we have any scalar number (\(s \in \mathbb R\)) and we multiply it by a vector (\(\mathbf{v} \in \mathbb R^n\)) then the result is also a vector in \(\mathbb R^n\). Since scalar multiplication is defined by \(s(v_1, v_2, \dots, v_n) = (sv_1, sv_2, \dots, sv_n)\) and multiplying a real number by a real number results in a real number this one is also true. Or we can say it as follows: if \(s \in \mathbb R\) and \(\mathbf{v} \in \mathbb R^n\), then \(s\mathbf{v} \in \mathbb R^n\).

1.3 Linear Combination

A linear combination is an expression constructed from a set of terms by multiplying each term by a constant and adding the results.

Definition:

Given vectors \(\mathbf{v}_1, \mathbf{v}_2, \ldots, \mathbf{v}_k\) in a vector space \(V\) and scalars \(c_1, c_2, \ldots, c_k\), a linear combination of the vectors with these scalars as coefficients is:

The scalars \(c_1, c_2, \ldots, c_k\) are often referred to as the coefficients of the linear combination.

1. Span: The set of all possible linear combinations of a given set of vectors is called the span of those vectors. It describes the subspace of $ V $ that can be reached using the given vectors.

2. Basis: A set of vectors that spans a space and is linearly independent (no vector in the set can be written as a linear combination of the others) is called a basis for that space. The concept of a linear combination is crucial in defining a basis.

3. Representation: In many problems, especially in physics and engineering, we often represent a vector as a linear combination of some basic, fundamental vectors.

Example:

Consider the vectors:

and

A linear combination of \(\mathbf{u}\) and \(\mathbf{v}\) might be:

Here, the coefficients are 2 and -3, respectively.

Linear combination with Numpy

In the following example, we are given two vectors (x_np and y_np), and two scalars (alpha and beta), we obtain the linear combination alpha*x_np + beta*y_np.

x_np = np.array([1,2])
y_np = np.array([3,4])
alpha = 0.5
beta = -0.8
c = alpha*x_np + beta*y_np
print(c)

[-1.9 -2.2]

1.4 Linear Independence

A set of vectors \(\{ \mathbf{v}_1, \mathbf{v}_2, \ldots, \mathbf{v}_k \}\) in a vector space \(V\) is said to be linearly independent if the only linear combination that produces the zero vector is the one where all coefficients \(c_i\) are zero:

implies that \(c_1 = c_2 = \ldots = c_k = 0\)

If a set of vectors is not linearly independent, then it is linearly dependent. This means that one of the vectors can be expressed as a linear combination of the others. In many contexts, especially when trying to form a basis for a space, linear dependence is undesired because it means there’s redundancy in our set of vectors.

Definition of a basis of a vector space

A finite set of vectors \({v_1,\dots, v_n}\) is called a basis of a vector space \(V\) if the set spans \(V\) and is linearly independent. i.e. each vector in \(V\) can be expressed uniquely as a linear combination of the vectors in a basis.

Span:

As touched upon earlier, the span of a set of vectors is the set of all possible linear combinations of those vectors. Mathematically, if \(S\) is a subset of a vector space \(V\), the span of \(S\) is given by:

where \(\mathbb{F}\) is the field over which the vector space is defined (often the real numbers \(\mathbb{R}\)).

Basis:

A basis of a vector space \(V\) is a set of linearly independent vectors that span \(V\). This means that every vector in \(V\) can be written as a unique linear combination of the basis vectors. The number of vectors in a basis of \(V\) is called the dimension of \(V\).

Canonical Basis:

The canonical basis (or standard basis) for \(\mathbb{R}^n\) is the set of vectors where one component is 1 and all other components are 0. For \(\mathbb{R}^3\), the canonical basis is:

For any vector space that is isomorphic to \(\mathbb{R}^n\), there exists a canonical basis, though the specific representation might differ based on the context.