Intuition
#- Think of dot product as directional multiplication.
- So, we are tracking the direction and multiplying the magnitudes along that direction.
- We are talking about finding what two vectors have in common. So, if there are 2
vectors (2,0), (3,0) we can see both have the common direction, which makes its dot
product 6. If they are perpendicular, like (0,2) and (3,0), they share no common direction,
and their dot product is 0. If they point in opposite directions, like (2,0) and (-3,0),
the dot product is -6.
- Another way to think about it is by dividing a vector into individual bits and
multiplying the overlapping parts:
$$
\vec{a} \cdot \vec{b} = a_x b_x + a_y b_y
$$ - Yet another way, the physics way would be to think about in term of rotation:
$$
\vec{a} \cdot \vec{b} = |\vec{a}| |\vec{b}| \cos(\theta)
$$
Geometric Interpretation and Projection
#- Dot product can be thought of as projecting one vector onto another and measuring
how much of one vector goes in the direction of another.
- If the vector projected onto other vector and the other vector are in the same
direction, the dot product is positive. If they are in opposite directions, the
dot product is negative. And if they are perpendicular, the dot product is zero.
\(
\vec{v} \cdot \vec{w} > 0 \text{ when, } \theta < 90^\circ \\
\vec{v} \cdot \vec{w} = 0 \text{ when, } \theta = 90^\circ \\
\vec{v} \cdot \vec{w} < 0 \text{ when, } \theta > 90^\circ
\) - The order does not matter, you can project \(\vec{v}\) onto \(\vec{w}\) or \(\vec{w}\)
onto \(\vec{v}\), the result will be the same.
- Why does the order not matter?
- If the two vectors happen to have the same length, we can leverage symmetry to
see that projecting \(\vec{v}\) onto \(\vec{w}\) and \(\vec{w}\) onto \(\vec{v}\) will
yield the same result.
Both are mirror images of each other. - If the two vectors have different lengths, let’s say we do \(2.\vec{v}\) and
project \(\vec{w}\) onto it, we can see the length of the projection does not change
Or, if we project \(\vec{v}\) onto \(\vec{w}\), we can see the length of the projection
gets scaled by the same factor of 2.
But, in both cases the dot product remains the same.
Linear transformations and Duality
#- There is a deeper connection between dot product and linear transformations.
- Think of a linear transformation that takes a vector from 2D space and maps it
to a single dimension (number line).
Example: Let’s say we have a linear transformation of a vector \(\vec{v}\) in 2D space,
\(\vec{v} = \begin{bmatrix} 4 \\ 3 \end{bmatrix}\) that maps it to a number line. So,
both the basis vectors \(\hat{\imath}\) and \(\hat{\jmath}\) get mapped to some
numbers on the number line. The transformation matrix, \( T = \begin{bmatrix} 1 & -2 \end{bmatrix} \)
means that \(\hat{\imath}\) gets mapped to 1 and \(\hat{\jmath}\) gets mapped to -2 on
the number line. Here is a visualization of this linear transformation:
This linear transformation can be computed as:
$$
T \cdot \vec{v} = \begin{bmatrix} 1 & -2 \end{bmatrix} \cdot \begin{bmatrix} 4 \\ 3 \end{bmatrix} = 1*4 + (-2)*3 = 4 - 6 = -2
$$ - Notice it looks like similar to if we just did dot product of two vectors:
$$
\begin{bmatrix} 1 \\ -2 \end{bmatrix} \cdot
\begin{bmatrix} 4 \\ 3 \end{bmatrix} =
1*4 + (-2)*3 = 4 - 6 = -2
$$
- We can see that there is a nice association between 1x2 matrices and 2D vectors. The transformation
just looks like the vector tilted on its side. \(1\times 2 \ matrices \leftrightarrow 2d \ vectors\)
- So, there seems to be some connection between linear transformations that takes vectors to numbers
and vectors themselves. This is called duality.
- Formally, duality refers to situation where you have a natural but surprising correspondence between
two types of mathematical objects. In this case, the duality is between vectors in a vector space
and linear functionals (linear transformations that map vectors to scalars) defined on that vector space.
References
#