Cosine Similarity, Distance & Normalization Mathematics

1. Core Concept: Angular Similarity & Normalization

Cosine Similarity measures the cosine of the angle $\alpha$ between two vectors, isolating their directional orientation regardless of vector length:

$$\text{cosine\_similarity}(\mathbf{a}, \mathbf{b}) = \frac{\mathbf{a} \cdot \mathbf{b}}{\|\mathbf{a}\| \|\mathbf{b}\|} = \frac{\sum_{i=1}^n a_i b_i}{\sqrt{\sum_{i=1}^n a_i^2}\sqrt{\sum_{i=1}^n b_i^2}}$$

Distance Conversion

Because vector databases like Chroma DB minimize distance values when ranking nearest neighbors, similarity is converted into Cosine Distance:

$$\text{cosine\_distance}(\mathbf{a}, \mathbf{b}) = 1 - \text{cosine\_similarity}(\mathbf{a}, \mathbf{b})$$

• Distance Range: $\in [0, 2]$. A distance of $0.0$ means identical direction ($\alpha = 0^\circ$), $1.0$ means orthogonal ($\alpha = 90^\circ$), and $2.0$ means complete opposite direction ($\alpha = 180^\circ$).
• Chroma DB Parameter: Set "space": "cosine" in the HNSW collection configuration.

Unit Normalization & Computational Shortcut

To normalize a vector to unit length ($L2$ norm = 1.0), divide each component by its magnitude:

$$\text{norm}(\mathbf{a}) = \frac{\mathbf{a}}{\|\mathbf{a}\|} \implies \sum_{i=1}^n \left(\text{norm}(\mathbf{a})_i\right)^2 = 1.0$$

When vectors are pre-normalized, their Cosine Similarity simplifies directly to the Dot Product:

$$\text{cosine\_similarity}(\mathbf{a}_{\text{norm}}, \mathbf{b}_{\text{norm}}) = \mathbf{a}_{\text{norm}} \cdot \mathbf{b}_{\text{norm}}$$

2. Daily Targeted Drills (15–20 Mins)

3. Hands-On Verification Snippet

Save and run this script to verify Cosine Similarity vs. Cosine Distance in NumPy and observe query rankings in Chroma DB with "space": "cosine":

import numpy as np
import chromadb

# 1. NumPy verification of Cosine Similarity, Distance, and Normalization
a = np.array([4.0, 8.0], dtype=np.float32)
b = np.array([11.5, 5.0], dtype=np.float32)

# Manual Cosine Similarity
norm_a = np.linalg.norm(a)
norm_b = np.linalg.norm(b)
cos_sim = np.dot(a, b) / (norm_a * norm_b)
cos_dist = 1.0 - cos_sim

print(f"Norm ||a||:          {norm_a:.4f}")
print(f"Norm ||b||:          {norm_b:.4f}")
print(f"Cosine Similarity:   {cos_sim:.4f}")   # 0.7668
print(f"Cosine Distance:     {cos_dist:.4f}")  # 0.2332

# Unit Normalization Verification
a_unit = a / norm_a
b_unit = b / norm_b
dot_normalized = np.dot(a_unit, b_unit)
print(f"Normalized Dot Prod: {dot_normalized:.4f}")  # 0.7668 (identical to cos_sim)

# 2. Chroma DB Collection with Cosine Space
client = chromadb.Client()
collection = client.create_collection(
    name="cosine_metric_demo",
    configuration={"hnsw": {"space": "cosine"}}  # Chroma computes 1 - cos(theta)
)

collection.add(
    ids=["doc_similar", "doc_orthogonal", "doc_divergent"],
    embeddings=[
        [4.0, 8.0],   # Angle ~ 39.9° relative to b
        [-5.0, 11.5], # Nearly orthogonal
        [-11.5, -5.0] # Complete opposite direction (180°)
    ],
    documents=[
        "Document with closely aligned topic",
        "Document with unrelated context",
        "Document with opposing context"
    ]
)

# Query using reference point b = [11.5, 5.0]
results = collection.query(
    query_embeddings=[[11.5, 5.0]],
    n_results=3
)

print("\nChroma Query Output (space='cosine'):")
for doc_id, dist, doc in zip(results["ids"][0], results["distances"][0], results["documents"][0]):
    sim = 1.0 - dist
    print(f"ID: {doc_id:<16} | Distance: {dist:.4f} | Cosine Sim: {sim:>6.4f} | {doc}")