1. Core Concept: Projection Geometry & Inner Product
The Dot Product (Inner Product) evaluates the degree of alignment between two vectors while weighting the outcome by their respective lengths:
$$\mathbf{a} \cdot \mathbf{b} = \sum_{i=1}^n a_i b_i = \|\mathbf{a}\| \|\mathbf{b}\| \cos(\alpha)$$
Geometric Meaning: Projection onto Direction
Geometrically, the dot product is calculated by projecting vector $\mathbf{a}$ directly onto the direction of vector $\mathbf{b}$. The length of this projection is $\|\mathbf{a}\|\cos(\alpha)$. Multiplying this projected length by the magnitude of $\mathbf{b}$ yields $\mathbf{a} \cdot \mathbf{b}$.
Similarity vs. Distance Inversion
- Similarity Metric: Dot product is naturally a similarity metric where higher positive numbers indicate closer semantic alignment or higher affinity.
- Distance Conversion in Chroma DB: Vector indices like HNSW minimize distance. To convert dot product to a distance metric, Chroma DB uses the negative dot product:
$$\text{Distance}_{IP}(\mathbf{a}, \mathbf{b}) = -(\mathbf{a} \cdot \mathbf{b})$$
A larger negative value (or higher positive distance value) represents greater divergence.
- Chroma DB Parameter: Set
"space": "ip" in the HNSW configuration dictionary.
| Dimension |
Dot Product (Inner Product) |
Cosine Distance |
L2 Distance |
| Chroma DB Space Key |
"ip" |
"cosine" |
"l2" |
| Sensitive to Magnitude? |
Yes (encodes popularity/confidence) |
No (direction/angle only) |
Yes (penalizes length variance) |
| Distance Output |
Negative Dot Product ($- \mathbf{a} \cdot \mathbf{b}$) |
$1 - \cos(\alpha) \in [0, 2]$ |
$\sum (a_i - b_i)^2 \ge 0$ |
| Primary Real-World Fit |
Recommender systems, Matrix Factorization, Pre-normalized embeddings |
General text retrieval, NLP semantics, variable document lengths |
Spatial coordinates, image feature mapping, clustering |
2. Daily Targeted Drills (15–20 Mins)
Drill 4.1: Manual Dot Product (2D Calculation)
Given vectors $\mathbf{a} = [4.0, 8.0]$ and $\mathbf{b} = [11.5, 5.0]$ (from our Day 3 example), calculate the raw dot product $\mathbf{a} \cdot \mathbf{b}$ and the corresponding Chroma DB distance.
Solution:
$\mathbf{a} \cdot \mathbf{b} = (4.0 \times 11.5) + (8.0 \times 5.0) = 46.0 + 40.0 = 86.0$
$\text{Chroma Distance}_{IP} = -(\mathbf{a} \cdot \mathbf{b}) = -86.0$
Drill 4.2: 3D High-Dimensional Inner Product
Compute the dot product between query vector $\mathbf{q} = [1.0, 2.0, 3.0]$ and candidate item vector $\mathbf{r} = [4.0, 5.0, 6.0]$.
Solution:
$\mathbf{q} \cdot \mathbf{r} = (1.0 \times 4.0) + (2.0 \times 5.0) + (3.0 \times 6.0) = 4.0 + 10.0 + 18.0 = 32.0$
Corresponding IP distance $= -32.0$.
Drill 4.3: Why Recommender Systems Leverage Dot Product
In movie recommendation systems, why is raw Inner Product preferred over Cosine Similarity when evaluating user-item embeddings?
Answer: In collaborative filtering and matrix factorization embeddings, the vector direction captures genre or topic alignment (e.g., action vs. comedy), while the magnitude captures popularity, rating volume, or interaction frequency. Dot product allows the system to recommend items that are both topical and popular.
Drill 4.4: Pre-Normalized Equivalence
If two vectors $\mathbf{u}$ and $\mathbf{v}$ are unit-normalized such that $\|\mathbf{u}\| = 1.0$ and $\|\mathbf{v}\| = 1.0$, what is the relationship between their Dot Product and their Cosine Similarity?
Answer: They are mathematically identical: $\mathbf{u} \cdot \mathbf{v} = \|\mathbf{u}\| \|\mathbf{v}\| \cos(\alpha) = 1.0 \times 1.0 \times \cos(\alpha) = \cos(\alpha)$. Because dot product bypasses division by norms during search iterations, querying pre-normalized vectors with "space": "ip" yields maximal search throughput.
Drill 4.5: Negative vs. Positive Inner Product Interpretation
If two item vectors produce a dot product score of $-15.0$, what does this indicate about their geometric relationship and similarity?
Answer: An angle $\alpha > 90^\circ$ ($\cos(\alpha) < 0$). They point in opposing directions in vector space, indicating negative correlation or opposing semantic attributes.
3. Hands-On Verification Snippet
Save and run this script to test Inner Product calculation in NumPy and query a Chroma DB collection configured with "space": "ip":
import numpy as np
import chromadb
# 1. NumPy verification of Dot Product and Projection
a = np.array([4.0, 8.0], dtype=np.float32)
b = np.array([11.5, 5.0], dtype=np.float32)
dot_product = np.dot(a, b)
norm_a = np.linalg.norm(a)
norm_b = np.linalg.norm(b)
cos_alpha = dot_product / (norm_a * norm_b)
print(f"NumPy Dot Product: {dot_product:.4f}") # 86.0000
print(f"Norm ||a||: {norm_a:.4f}") # 8.9443
print(f"Norm ||b||: {norm_b:.4f}") # 12.5399
print(f"Cos(alpha): {cos_alpha:.4f}") # 0.7668
print(f"Projected ||c||: {(norm_a * cos_alpha):.4f}") # 6.8581
# 2. Chroma DB Collection with Inner Product ('ip') Space
client = chromadb.Client()
collection = client.create_collection(
name="recommender_ip_demo",
configuration={"hnsw": {"space": "ip"}} # Chroma computes negative dot product
)
# Insert items with topic and popularity encoded in coordinates
collection.add(
ids=["movie_popular_action", "movie_niche_action", "movie_comedy"],
embeddings=[
[10.0, 10.0], # Strong action alignment + high magnitude (popular)
[2.0, 2.0], # Strong action alignment + low magnitude (indie/niche)
[-5.0, 1.0] # Distinct direction (different genre)
],
documents=[
"Blockbuster Action Movie (High engagement)",
"Indie Action Film (Low views)",
"Romantic Comedy Classic"
]
)
# User query: prefers action [3.0, 3.0]
results = collection.query(
query_embeddings=[[3.0, 3.0]],
n_results=3
)
print("\nChroma Recommender Query Output (space='ip'):")
for doc_id, dist, doc in zip(results["ids"][0], results["distances"][0], results["documents"][0]):
# Note: Chroma returns negative dot product as distance
actual_dot = -dist
print(f"ID: {doc_id:<22} | Chroma Dist: {dist:>8.2f} | Raw Dot Product: {actual_dot:>6.2f} | {doc}")