Vector algebra

In physics, vectors describe quantities that have a magnitude and a spatial direction, such as a position, a velocity, and a temperature gradient. We will later see that these vectors are, in fact, basically first order tensors. But for now, let's cover the basic rules of vector algebra as an introduction.

Consider the vectors x‾\underline{\boldsymbol{ x}}, u‾\underline{\boldsymbol{ u}}, v‾\underline{\boldsymbol{ v}}, and w‾\underline{\boldsymbol{ w}} in the figure below.

The figure illustrates a the two first important properties of vector addition:

  • Addition is associative, i.e. u‾+v‾=v‾+u‾\underline{\boldsymbol{ u}}+\underline{\boldsymbol{ v}}=\underline{\boldsymbol{ v}}+\underline{\boldsymbol{ u}}

  • Addition is commutative, i.e. (u‾+v‾)+w‾=u‾+(v‾+w‾)(\underline{\boldsymbol{ u}}+\underline{\boldsymbol{ v}}) + \underline{\boldsymbol{ w}} = \underline{\boldsymbol{ u}} + (\underline{\boldsymbol{ v}} + \underline{\boldsymbol{ w}})

More formally, we can introduce a vector space V\mathbb{ V}. This can for example be the real (3-dimensional) world, R3\mathbb{ R}^3. We can then define the full set of properties required for the vectors in V\mathbb{ V} to be just vectors:

Vector summation

  • Associative addition (see above)

  • Commutative addition (see above)

  • There exists a unique zero vector, 0‾\underline{\boldsymbol{ 0}}, such that 0‾+v‾=v‾, ∀v‾∈V\underline{\boldsymbol{ 0}}+\underline{\boldsymbol{ v}}=\underline{\boldsymbol{ v}},\; \forall \underline{\boldsymbol{ v}}\in\mathbb{ V}

  • There exists a unique vector −v‾-\underline{\boldsymbol{ v}}, ∀v‾∈V\forall\underline{\boldsymbol{ v}}\in\mathbb{ V}, such that v‾+(−v‾)=0‾\underline{\boldsymbol{ v}}+(-\underline{\boldsymbol{ v}})=\underline{\boldsymbol{ 0}}.

Multiplication by scalar Consider scalars a,b∈Ra, b\in\mathbb{ R}, then

  • Associative multiplication by scalar, a(bv‾)=(ab)v‾a(b\underline{\boldsymbol{ v}})=(ab)\underline{\boldsymbol{ v}}

  • Commutative multiplication by scalar, av‾=v‾⋅aa\underline{\boldsymbol{ v}}=\underline{\boldsymbol{ v}}\cdot a

  • Multiplication by zero and unity: 1v‾=v‾1\underline{\boldsymbol{ v}}=\underline{\boldsymbol{ v}}, 0v‾=0‾0\underline{\boldsymbol{ v}}=\underline{\boldsymbol{ 0}}

  • Distributive multiplication by scalar wrt. vector addition:
    a(v‾+w‾)=av‾+aw‾a(\underline{\boldsymbol{ v}}+\underline{\boldsymbol{ w}})=a\underline{\boldsymbol{ v}}+a\underline{\boldsymbol{ w}}

  • Distributive multiplication by scalar wrt. scalar addition:
    (a+b)v‾=av‾+bv‾(a+b)\underline{\boldsymbol{ v}}=a\underline{\boldsymbol{ v}}+b\underline{\boldsymbol{ v}}

Linear dependence

Assume that you have two different vectors in 2d and you want to construct a 3rd vector by scaling your vectors and adding them together. Since there are 2 dimensions, and you have two vectors, this should be possible, or?

However, what if your two vectors are pointing in the same direction (but have different length)? Then, you can only construct another vector in that direction. So even if you have two vectors, you are only spanning a line in your 2-dimensional space. I.e. you are only spanning a 1-dimensional space with your vectors. The reason is that these vectors are linearly dependent. Generally, we can formulate the definition of linear dependence as

Definition:
With i=1,2,⋯ ,Ni=1,2,\cdots,N, the vectors v‾i∈V\underline{\boldsymbol{ v}}_i\in\mathbb{ V} are linearly dependent if there exists scalars ai∈R: aiai>0a_i\in\mathbb{ R}:\,a_i a_i > 0 (i.e. at least one ai≠0a_i\neq0) such that aiv‾i=0‾a_i \underline{\boldsymbol{ v}}_i = \underline{\boldsymbol{ 0}}). If no such scalars exists, the vectors v‾i\underline{\boldsymbol{ v}}_i are linearly independent.

v‾=aiv‾i\underline{\boldsymbol{ v}}=a_i\underline{\boldsymbol{ v}}_i, which we just used, is called a linear combination of the vectors v‾i\underline{\boldsymbol{ v}}_i. (Recall the Einstein summation convention)

Scalar product

The scalar product, u‾⋅v‾\underline{\boldsymbol{ u}}\cdot\underline{\boldsymbol{ v}}, between two vectors, u‾\underline{\boldsymbol{ u}} and v‾\underline{\boldsymbol{ v}}, is defined as

u‾⋅v‾=∣∣u‾∣∣∣∣v‾∣∣cos⁡(θ)\begin{aligned} \underline{\boldsymbol{ u}}\cdot\underline{\boldsymbol{ v}} = \left\vert\left\vert \underline{\boldsymbol{ u}}\right\vert\right\vert\left\vert\left\vert \underline{\boldsymbol{ v}}\right\vert\right\vert\cos(\theta) \end{aligned}

where ∣∣u‾∣∣\left\vert\left\vert \underline{\boldsymbol{ u}}\right\vert\right\vert is the length of the vector u‾\underline{\boldsymbol{ u}}, and θ\theta is the angle between vectors u‾\underline{\boldsymbol{ u}} and v‾\underline{\boldsymbol{ v}}. Note that we do not require a coordinate system to define this operation!

From this definition, it follows that ∣∣u‾∣∣=u‾⋅u‾\left\vert\left\vert \underline{\boldsymbol{ u}}\right\vert\right\vert = \sqrt{\underline{\boldsymbol{ u}}\cdot\underline{\boldsymbol{ u}}} as in this case, θ=0\theta=0. Hence, u‾⋅u‾=0⇔u‾=0‾\underline{\boldsymbol{ u}}\cdot\underline{\boldsymbol{ u}}=0 \Leftrightarrow \underline{\boldsymbol{ u}}=\underline{\boldsymbol{ 0}}. Furthermore, if u‾\underline{\boldsymbol{ u}} and v‾\underline{\boldsymbol{ v}} are perpendicular vectors, then θ=π/2\theta=\pi/2 and thus u‾⋅v‾=0\underline{\boldsymbol{ u}}\cdot\underline{\boldsymbol{ v}}=0.

When working with scalar products, the following algebraic rules apply

  • Distributive multiplication, u‾⋅(v‾+w‾)=u‾⋅v‾+u‾⋅w‾\underline{\boldsymbol{ u}}\cdot(\underline{\boldsymbol{ v}}+\underline{\boldsymbol{ w}}) = \underline{\boldsymbol{ u}}\cdot\underline{\boldsymbol{ v}} + \underline{\boldsymbol{ u}}\cdot\underline{\boldsymbol{ w}}.

  • Commutative multiplication, u‾⋅v‾=v‾⋅u‾\underline{\boldsymbol{ u}}\cdot\underline{\boldsymbol{ v}}=\underline{\boldsymbol{ v}}\cdot\underline{\boldsymbol{ u}}

  • Associative wrt. multiplication with scalar, a(u‾⋅v‾)=(au‾)⋅v‾a(\underline{\boldsymbol{ u}}\cdot\underline{\boldsymbol{ v}}) = (a\underline{\boldsymbol{ u}})\cdot\underline{\boldsymbol{ v}}

Cross product

The cross (or vector) product, u‾×v‾\underline{\boldsymbol{ u}}\times\underline{\boldsymbol{ v}}, between two vectors, u‾\underline{\boldsymbol{ u}} and v‾\underline{\boldsymbol{ v}}, is defined as

u‾×v‾=∣∣u‾∣∣∣∣v‾∣∣sin⁡(θ)n‾\begin{aligned} \underline{\boldsymbol{ u}}\times\underline{\boldsymbol{ v}} = \left\vert\left\vert \underline{\boldsymbol{ u}}\right\vert\right\vert\left\vert\left\vert \underline{\boldsymbol{ v}}\right\vert\right\vert\sin(\theta) \underline{\boldsymbol{ n}} \end{aligned}

with the same quantities as in Equation (1). The new vector, n‾\underline{\boldsymbol{ n}}, is the normal vector (with unit length) to the plane spanned by u‾\underline{\boldsymbol{ u}} and v‾\underline{\boldsymbol{ v}}, following the right-hand-rule. The following animation from Wikipedia illustrates this rule well

Cross product.gif

A couple of other properties, following from the definition in Equation (2), are also shown by this animation:

  • u‾×u‾=0‾\underline{\boldsymbol{ u}}\times\underline{\boldsymbol{ u}}=\underline{\boldsymbol{ 0}}

  • u‾×(−u‾)=0‾\underline{\boldsymbol{ u}}\times(-\underline{\boldsymbol{ u}})=\underline{\boldsymbol{ 0}}

  • ∣∣u‾×v‾∣∣=∣∣u‾∣∣∣∣v‾∣∣\left\vert\left\vert \underline{\boldsymbol{ u}}\times\underline{\boldsymbol{ v}}\right\vert\right\vert = \left\vert\left\vert \underline{\boldsymbol{ u}}\right\vert\right\vert\left\vert\left\vert \underline{\boldsymbol{ v}}\right\vert\right\vert if u‾⊥v‾\underline{\boldsymbol{ u}}\perp\underline{\boldsymbol{ v}}
    (i.e. u‾\underline{\boldsymbol{ u}} and v‾\underline{\boldsymbol{ v}} are perpendicular such that u‾⋅v‾=0\underline{\boldsymbol{ u}}\cdot\underline{\boldsymbol{ v}}=0)

When working with cross products, the following algebraic rules apply

  • Distributive multiplication, u‾×(v‾+w‾)=u‾×v‾+u‾×w‾\underline{\boldsymbol{ u}}\times(\underline{\boldsymbol{ v}}+\underline{\boldsymbol{ w}}) = \underline{\boldsymbol{ u}}\times\underline{\boldsymbol{ v}} + \underline{\boldsymbol{ u}}\times\underline{\boldsymbol{ w}}

  • Anti-commutative multiplication, u‾×v‾=−v‾×u‾\underline{\boldsymbol{ u}}\times\underline{\boldsymbol{ v}} = - \underline{\boldsymbol{ v}}\times\underline{\boldsymbol{ u}}

  • Associative wrt. multiplication with scalar, a(u‾×v‾)=(au‾)×v‾a(\underline{\boldsymbol{ u}}\times\underline{\boldsymbol{ v}}) = (a\underline{\boldsymbol{ u}})\times\underline{\boldsymbol{ v}}

Here it is important to note the difference to scalar products, which are commutative, while cross products are anti-commutative (i.e. the order of multiplication matters!)

Basis system

Above, we have seen that many properties of vectors can be defined without a basis system, simply by considering a vector as an object with a length and a direction. For these definitions, there is no need to introduce a coordinate system. However, if we want to perform actual calculations, everyting becomes easier if we define a base system from which we measure all quantities. Using a set of predefined so-called base vectors, we can express other vector quantities by using a linear combination of these base vectors. To ensure that this description is unique, we require that the base vectors are linearly independent. Otherwise, we can have the same vector described by different linear combinations.

Let's consider a basis system described by the linear independent vectors e‾i\underline{\boldsymbol{ e}}_i. A vector v‾\underline{\boldsymbol{ v}}, can then be described by the unique coefficients viv_i (considering the e‾i\underline{\boldsymbol{ e}}_i basis system) by v‾=vie‾i\underline{\boldsymbol{ v}}=v_i\underline{\boldsymbol{ e}}_i. If we take the scalar product u‾⋅v‾\underline{\boldsymbol{ u}}\cdot\underline{\boldsymbol{ v}}, we get

u‾⋅v‾=uivje‾i⋅e‾j\begin{aligned} \underline{\boldsymbol{ u}}\cdot\underline{\boldsymbol{ v}} = u_i v_j \underline{\boldsymbol{ e}}_i \cdot \underline{\boldsymbol{ e}}_j \end{aligned}

resulting in the metric coefficients eij=e‾i⋅e‾je_{ij}=\underline{\boldsymbol{ e}}_i \cdot \underline{\boldsymbol{ e}}_j for the coordinate system. This coefficient complicates our calculations, but by choosing our basis vectors cleverly, we can make it become δij\delta_{ij}, resulting in u‾⋅v‾=uivi\underline{\boldsymbol{ u}}\cdot\underline{\boldsymbol{ v}} = u_i v_i. But how can we choose e‾i\underline{\boldsymbol{ e}}_i, such that e‾i⋅e‾j=δij\underline{\boldsymbol{ e}}_i \cdot \underline{\boldsymbol{ e}}_j=\delta_{ij}?

The solution is an orthonormal coordinate system (Cartesian coordinate system). The ortho comes from orthogonal, implying perpendicularity between the base vectors. If we consider the scalar product, this implies that e‾i⋅e‾j=0\underline{\boldsymbol{ e}}_i\cdot\underline{\boldsymbol{ e}}_j=0 if i≠ji\neq j. The normal comes from that the base vectors are normalized, i.e. have unit length (∣∣e‾i∣∣=1\left\vert\left\vert \underline{\boldsymbol{ e}}_i\right\vert\right\vert=1). This implies that e‾i⋅e‾j=1\underline{\boldsymbol{ e}}_i\cdot\underline{\boldsymbol{ e}}_j=1 if i=ji=j. So with these two choices, we have exactly the definition of δij\delta_{ij}, for our orthonormal coordinate system with basis vectors e‾i\underline{\boldsymbol{ e}}_i:

e‾i⋅e‾j=δij={1i=j0i≠j\begin{aligned} \underline{\boldsymbol{ e}}_i\cdot\underline{\boldsymbol{ e}}_j = \delta_{ij} = \left\lbrace\begin{matrix} 1 & i=j \\ 0 & i\neq j\end{matrix}\right. \end{aligned}

Scalar product in orthonormal basis system

As mentioned above, the scalar product between two vectors, u‾=uie‾i\underline{\boldsymbol{ u}}=u_i\underline{\boldsymbol{ e}}_i and v‾=vie‾i\underline{\boldsymbol{ v}}=v_i\underline{\boldsymbol{ e}}_i, in an orthonormal coordinate system, e‾i\underline{\boldsymbol{ e}}_i, becomes

u‾⋅v‾=uivi\begin{aligned} \underline{\boldsymbol{ u}}\cdot\underline{\boldsymbol{ v}} = u_i v_i \end{aligned}

Cross product in orthonormal basis system

As seen from the definition, the cross product is more complicated. Again, we consider two vectors, u‾=uie‾i\underline{\boldsymbol{ u}}=u_i\underline{\boldsymbol{ e}}_i and v‾=vie‾i\underline{\boldsymbol{ v}}=v_i\underline{\boldsymbol{ e}}_i, in an orthonormal coordinate system, e‾i\underline{\boldsymbol{ e}}_i. The cross-product then becomes

u‾×v‾=(uie‾i)×(vje‾j)=uivje‾i×e‾j\begin{aligned} \underline{\boldsymbol{ u}}\times\underline{\boldsymbol{ v}} = (u_i\underline{\boldsymbol{ e}}_i) \times (v_j\underline{\boldsymbol{ e}}_j) = u_i v_j \underline{\boldsymbol{ e}}_i\times\underline{\boldsymbol{ e}}_j \end{aligned}

Based on the definition of the cross product and the orthonormal coordinate system, we know the following in 3 dimensions

e‾i×e‾j={±e‾ki≠jk≠i,j0‾i=j\begin{aligned} \underline{\boldsymbol{ e}}_i\times\underline{\boldsymbol{ e}}_j = \left\lbrace\begin{matrix} \pm\underline{\boldsymbol{ e}}_k & i\neq j & k\neq i,j\\ \underline{\boldsymbol{ 0}} & i=j & \end{matrix}\right. \end{aligned}

(In three dimensions, for i≠ji\neq j, only one choice remains for k≠i,jk\neq i,j) However, the direction of ±e‾k\pm\underline{\boldsymbol{ e}}_k in the case that i≠ji\neq j is not properly defined. This depends on the order of our basis vectors. We therefore need to introduce this order in our definition of our coordinate system and we will choose a right-handed coordinate system (see the illustration used to define the cross product). For a coordinate system, this implies that e‾3=e‾1×e‾2\underline{\boldsymbol{ e}}_3=\underline{\boldsymbol{ e}}_1\times\underline{\boldsymbol{ e}}_2. Using the Levi-Civita symbol, εijk\varepsilon_{ijk}, we then have

e‾i×e‾j=εijke‾k\begin{aligned} \underline{\boldsymbol{ e}}_i\times\underline{\boldsymbol{ e}}_j = \varepsilon_{ijk} \underline{\boldsymbol{ e}}_{k} \end{aligned}

in a right-handed orthonormal coordinate system. The cross product for two general vectors is then

u‾×v‾=uivjεijke‾k\begin{aligned} \underline{\boldsymbol{ u}}\times\underline{\boldsymbol{ v}} = u_i v_j \varepsilon_{ijk} \underline{\boldsymbol{ e}}_k \end{aligned}

Unless otherwise specified, coordinate systems discussed on this webpage are right-handed and orthonormal.

© Knut Andreas Meyer.
Last modified: April 22, 2024.
Website built with Franklin.jl and Julia.