3. Maths4ML: Dot Product

🗓 Dec 21, 2025 | #Maths for ML | View Source

#Drawbacks of Rulers (Euclidean Distance)

#Example :

There is data of 2 people listening to music :

	Person A	Person B
Time per day	5 hours (300 mins)	10 mins
jazz	270 mins (90%)	9 mins (90%)
pop	30 mins (10%)	1 min (10%)

When plotted on a graph where axes are min. of jazz & min. of pop, then person A & B are miles apart.

The Euclidean distance between them will be huge.

It tells how much physical space separates these two points?

Thus according to Euclidean distance, A & B have completely different taste. But logically, their tastes are same only the magnitude is different.

To solve this problem of looking at magnitude or looking at directions, Shadows are used.

#Shadow (Projection)

Let there be 2 vectors A and B. Let a light source is shining down from above on A, perpendicular to B and casts shadow of A on B.

The length of shadow tells how much of vector A goes in the direction of vector B.

Or it can also be conveyed as drawing a straight line from the tip of the top vector down so that it hits the bottom vector perpendicularly.

The angle between the vectors/arrow tells the cosine similarity.

Vectors pointing is same directions will have angle of $0\degree$ : perfectly aligned.
Vectors perpendicular to each other will have angle of $90\degree$ : unrelated.

#Dot Product

To understand projections dot product and its geometric definition is helpful.

\mathbf{a} \cdot \mathbf{b} = \sum a_i b_i = a_1b_1 + a_2b_2+\cdots+a_nb_n

Multiply matching coordinates and sum them up.

#Geometric definition

\mathbf{a} \cdot \mathbf{b} = |a||b| \cos(\theta)

So, dot product is basically Length of A x Length of B x Percentage of alignment ( $\cos\theta$ ).

From trigonometry,

\text{Projection Length} = |a| \cos(\theta)

It is saying, how much of a exists in the direction of b.

\mathbf{a} \cdot \mathbf{b} = \underbrace{|\mathbf{a}| \cos(\theta)}_{\text{Shadow}} \times |\mathbf{b}|

\mathbf{a} \cdot \mathbf{b} = (\text{Length of Shadow}) \times (\text{Length of Ground})

So, Dot Product is designed to measure the total impact of one vector moving along another.

#Cosine Similarity

\text{Cosine Similarity} = \cos(\theta) = \frac{\mathbf{a}\cdot \mathbf{b}}{|a||b|}

This comes directly from the geometric definition above.

The vectors get normalized by dividing by lengths. This turns vectors a & b into unit vectors. And now only their alignment is left to compare.

Result +1.0: Perfect Match ( $0\degree$ ).
Result 0.0: Orthogonal / No Relation ( $90°$ ).
- Because $\cos(90\degree)=0$ .
Result -1.0: Exact Opposites ( $180\degree$ ).

#Shadow Caster

Drag the white arrow (vector A) or adjust the sliders to change its length and direction.

The yellow line is the projection / shadow cast upon the green / ground vector.
When the white arrow is straight up, shodow is gone and cosine similarity is 0, signifying orthogonality or independence.
An obtuse angle or ( $\gt 90\degree$ ) represents negative correlation

With this, dot product and its geometric meaning is covered.