↓ Skip to main content
  1. Notes/
  2. Mathematics/
  3. Linear Algebra/

Vectors

Notes Math

Intro
#

  • Vectors are fundamental building block of linear algebra.

What are vectors?
#

3 perspectives:

  • Physics student:

    • Vectors are arrows pointing in space.
    • What defines a vector is its length and the direction it is pointing.
    • Vectors living in the flat plane are 2D and that sitting in broader space are 3D.

  • CS student:

    • Vectors are ordered list of numbers.
    • Example: For a house we are modelling using a vector like this:
    1. Square footage = 2,600 ft2
    2. Price = $300,000 This can be represented using a 2D vector like: $$ \vec{v} = \begin{bmatrix} 2,600 ft^2 \\ \$300,000 \end{bmatrix} \neq \begin{bmatrix} \$300,000 \\ 2,600 ft^2 \end{bmatrix} $$
    3. Vector here is just a fancy way of saying list. And notice the order matters here. Also what makes the vector 2D is the fact that the length is 2.

  • Mathematician:

    • Seeks to generalize both of the views.
    • Vectors can be anything where there is a sensible notion of adding 2 vectors and multiplying a vector by a number.

How to think about vectors?
#

  • Always, when any one says vector think of the vector as an arrow with its tail on the origin(0,0), because in linear algebra it will almost always be the case that your vector will be rooted at the origin.
  • Now, this can be translated to list of number point of view by considering the coordinates of a vector.
  • Vector coordinates basically gives instructions on how to get from the tail of the vector(origin) to its tip. First number telling how far along to walk across x axis, 2nd y axis and 3rd(if any) z axis.
  • To distinguish from coordinates write them using square brackets.

Vector addition
#

  • To sum 2 vectors move the 2nd one so that its tail sits next to first ones tip then connect the 2 with a third vector which will be the sum. vector addition
  • But why? Think of it like this: Vectors have direction so if I walk through vector A, and then from the tip of vector A I walk in the same direction as B but from tip of A, the point I will reach will be the same as directly going from tail of vector A to tip of vector B. This is very similar to how we think of adding simple numbers using a number line but in 2D.
  • Now, if you use the coordinates and add them as vectors you can see this work. Example: $$ \begin{bmatrix} 1 \\ 2 \end{bmatrix} + \begin{bmatrix} 3 \\ -1 \end{bmatrix} = \begin{bmatrix} 1 + 3 \\ 2 + (-1) \end{bmatrix} = \begin{bmatrix} 4 \\ 1 \end{bmatrix} $$

Scalar multiplication
#

  • If I 2x a vector I am doubling its size. If I 0.1x it I am shrinking the vector to 10% of its original size. This is called scaling the vector.
  • Example 1: $$ 2 \times \begin{bmatrix} 1 \\ 2 \end{bmatrix} = \begin{bmatrix} 2 \times 1 \\ 2 \times 2 \end{bmatrix} = \begin{bmatrix} 2 \\ 4 \end{bmatrix} $$
  • Example 2: $$ 0.5 \times \begin{bmatrix} 1 \\ 2 \end{bmatrix} = \begin{bmatrix} 0.5 \times 1 \\ 0.5 \times 2 \end{bmatrix} = \begin{bmatrix} 0.5 \\ 1 \end{bmatrix} $$
    • Because of scaling, the word scalar is interchangeably used with the word number.

Basis vectors
#

  • 2 basis vectors of the xy coordinate system:

    1. Unit vector in the x direction: \( \hat{\imath} \)
    2. Unit vector in the y direction: \( \hat{\jmath} \)

  • Think of every vector like sum of 2 scaled vectors using some scalars.

  • Example: sum of 2 scaled vectors So, here: the final vector is the sum of unit vectors scaled by 2 scalars 3 and -2.

  • Using these 2 basis vectors we can reach any possible vector in the 2D space.

  • Note: We could have chosen different basis vectors other than the unit vectors and still we would be able to reach every possible 2D vector.

Linear Combinations
#

  • By scaling two vectors and adding them we can get linear combinations. $$ a \vec{v} + b \vec{w} $$

Span
#

  • If we take any 2 vectors in 2D space, we can reach every point in the space, except there is a catch:
    • If both the vectors lie on the same line we won’t be able to reach the whole 2D space.
    • Or, if both vectors are 0, then we are stuck at the origin.
  • The span is the set of all vectors we can reach using linear combinations of a given pair of vectors.

Vectors in 3D space
#

  • Think of 2 vectors in a 3D space. 2 vectors in 3d space
  • What will be the span? It will be a flat sheet. Really think about it or refer to the 2nd video in References span of 2 vectors in 3d space
  • If there are 3 vectors instead of 2 the linear combination will defined by pretty much the same way as 2: $$ a \vec{v} + b \vec{w} + c \vec{u} $$

Linear Dependence and Independence of vectors
#

  • If at least one of the vector is redundant, i.e., not adding anything to our span, we call the vectors linearly dependent.
  • Another way of phrasing would be: One of the vector can be expressed as a linear combination of the others since its already on the span of others.
  • Example: In 2D space think of both vectors on a single line. Even if we remove one of the vector the span does not change so it makes them linearly dependent.
  • The opposite of this is linearly independent vectors, i.e., they add another dimension to the span.
  • In 2D space any 2 vectors not on the same line or origin are linearly independent.

Another definition of basis of a vector space
#

The basis of a vector space is a set of linearly independent vectors that span the entire space. “Think about it!”

Conclusion
#

  • More useful than the 2 views of vectors(physics and CS) in linear algebra is the ability to translate between the 2 views.
  • It gives data analysts a way to think about data points as arrows in space.
  • It gives physicists and graphics programmer a language to describe space and manipulate objects in space.
  • Think about vectors as points when there are multiple vectors. And as arrows when they are alone.

References
#