Stop imagining embeddings — let's generate real ones and see the numbers
What We're Building Today
By the end of this module you will have written real code that:
1. Takes a list of words/sentences
2. Calls the OpenAI API to get real embeddings
3. Calculates cosine similarity between them
4. Prints a comparison table showing which are similar
No theory this time. Just code and real numbers.
What You Need Before Starting
1. Node.js installed on your computer
2. An OpenAI API key
3. A code editor (VS Code is fine)
If you don't have an OpenAI API key yet — go to platform.openai.com, sign up, and create one. You'll need a small amount of credit (a few cents for this exercise).
Step 1 — Project Setup
Open your terminal and run:
mkdir embedding-explorer cd embedding-explorer npm init -y
Create a .env file for your API key:
# .env OPENAI_API_KEY=your_actual_api_key_here
Install the packages we need:
npm install openai dotenv
Step 2 — The Complete Code
Create a file called embeddings.js and paste this entire code:
// ───────────────────────────────────────── // IMPORTS // ─────────────────────────────────────────
// OpenAI is the official library to talk to OpenAI's API import OpenAI from "openai";
// dotenv lets us read secret keys from the .env file import * as dotenv from "dotenv";
// Actually load the .env file into memory // After this line, process.env.OPENAI_API_KEY is available dotenv.config();
// ───────────────────────────────────────── // SETUP — Create the OpenAI client // ─────────────────────────────────────────
// Create one OpenAI client object that we'll reuse everywhere // Think of this like logging into OpenAI once at the top const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY, // process.env.OPENAI_API_KEY reads from .env file // example value: "sk-proj-abc123xyz..." });
// ───────────────────────────────────────── // FUNCTION 1 — getEmbedding // Takes one piece of text, returns its vector // ─────────────────────────────────────────
async function getEmbedding(text) { // text = any string you pass in // example: "cat" or "What are the side effects of aspirin?"
const response = await openai.embeddings.create({ model: "text-embedding-3-small", // model = which OpenAI embedding model to use // "text-embedding-3-small" gives 1536-dimensional vectors // it's cheap, fast, and good quality
input: text, // input = the actual text we want to convert to a vector // example: input = "cat" });
// response.data is an array (in case you sent multiple texts) // response.data[0] is the first (and only) result // response.data[0].embedding is the actual vector // example: [0.0234, -0.0891, 0.0412, ...] — 1536 numbers total return response.data[0].embedding; }
// ───────────────────────────────────────── // FUNCTION 2 — cosineSimilarity // Takes two vectors, returns a score 0 to 1 // 1.0 = identical meaning, 0.0 = completely different // ─────────────────────────────────────────
function cosineSimilarity(vectorA, vectorB) { // vectorA = first embedding example: [0.9, 0.8, 0.1, 0.05, ...] // vectorB = second embedding example: [0.8, 0.9, 0.1, 0.06, ...] // both are arrays of 1536 numbers
// STEP 1 — Dot Product // Multiply each pair of matching numbers, then add all results together // example (simplified 4D): // vectorA = [0.9, 0.8, 0.1, 0.05] // vectorB = [0.8, 0.9, 0.1, 0.06] // dotProduct = (0.9×0.8) + (0.8×0.9) + (0.1×0.1) + (0.05×0.06) // = 0.72 + 0.72 + 0.01 + 0.003 // = 1.453 const dotProduct = vectorA.reduce( (sum, val, i) => sum + val * vectorB[i], // sum = running total (starts at 0) // val = current number from vectorA // i = current index position // vectorB[i] = matching number from vectorB at same position 0 // starting value of sum );
// STEP 2 — Magnitude (length) of vectorA // Square each number, add them all, then square root the total // example: vectorA = [0.9, 0.8, 0.1, 0.05] // = √(0.9² + 0.8² + 0.1² + 0.05²) // = √(0.81 + 0.64 + 0.01 + 0.0025) // = √1.4625 // = 1.209 const magnitudeA = Math.sqrt( vectorA.reduce( (sum, val) => sum + val * val, // sum = running total // val = current number // val * val = val squared 0 // starting value ) );
// STEP 3 — Magnitude (length) of vectorB // Same calculation as magnitudeA but for vectorB // example: vectorB = [0.8, 0.9, 0.1, 0.06] // = √(0.64 + 0.81 + 0.01 + 0.0036) // = √1.4636 // = 1.210 const magnitudeB = Math.sqrt( vectorB.reduce((sum, val) => sum + val * val, 0) );
// STEP 4 — Final cosine similarity formula // = dotProduct divided by (magnitudeA × magnitudeB) // = 1.453 / (1.209 × 1.210) // = 1.453 / 1.463 // = 0.993 ← very high = very similar return dotProduct / (magnitudeA * magnitudeB); // returns a number between -1.0 and 1.0 // example return value: 0.9923 for "cat" vs "dog" // example return value: 0.2341 for "cat" vs "pizza" }
// ───────────────────────────────────────── // FUNCTION 3 — compareTexts // Takes an array of texts // Gets embeddings for all of them // Compares every pair and prints similarity scores // ─────────────────────────────────────────
async function compareTexts(texts) { // texts = array of strings to compare // example: ["cat", "dog", "pizza", "car", "vehicle"]
console.log("\n🔄 Getting embeddings from OpenAI...\n");
// Get embeddings for ALL texts at the same time (parallel) // Promise.all means — start all API calls together, wait for all to finish // Much faster than calling them one by one const embeddings = await Promise.all( texts.map((text) => getEmbedding(text)) // texts.map loops through each text and calls getEmbedding on it // returns an array of Promises (pending API calls) // Promise.all waits for ALL of them to complete // example result: [ // [0.023, -0.089, ...], ← embedding for "cat" (1536 numbers) // [0.081, -0.034, ...], ← embedding for "dog" (1536 numbers) // [0.012, 0.091, ...], ← embedding for "pizza" (1536 numbers) // ] );
console.log(`✅ Got ${embeddings.length} embeddings`); // embeddings.length = how many texts we embedded // example: 6
console.log(`📐 Each embedding has ${embeddings[0].length} dimensions\n`); // embeddings[0].length = number of dimensions in each vector // example: 1536
console.log("─".repeat(60)); // print a divider line
console.log("\n📊 SIMILARITY SCORES:\n"); console.log("Higher score = more similar (max is 1.0, min is -1.0)\n");
// results will hold all the comparison pairs with their scores // example final value: [ // { textA: "car", textB: "vehicle", score: 0.7821 }, // { textA: "cat", textB: "dog", score: 0.7634 }, // ... // ] const results = [];
// Nested loop — compare every pair of texts // outer loop: i goes from 0 to end // inner loop: j starts from i+1 (to avoid duplicates and self-comparison) // example for 3 texts [A, B, C]: // i=0,j=1 → compare A vs B // i=0,j=2 → compare A vs C // i=1,j=2 → compare B vs C for (let i = 0; i < texts.length; i++) { for (let j = i + 1; j < texts.length; j++) {
// Calculate cosine similarity between text i and text j // score is a number between -1.0 and 1.0 // example: score = 0.7634 for "cat" vs "dog" const score = cosineSimilarity(embeddings[i], embeddings[j]);
// Push this pair and its score into results array results.push({ textA: texts[i], // example: "cat" textB: texts[j], // example: "dog" score: score, // example: 0.7634 }); } }
// Sort results array by score — highest score first // b.score - a.score = descending order (highest to lowest) // example after sort: // [{ textA:"car", textB:"vehicle", score:0.7821 }, ← highest first // { textA:"cat", textB:"dog", score:0.7634 }, // { textA:"cat", textB:"pizza", score:0.3821 }] ← lowest last results.sort((a, b) => b.score - a.score);
// Print each result nicely results.forEach((result, index) => { // result = one object like { textA:"cat", textB:"dog", score:0.76 } // index = position in array (0, 1, 2...)
const bar = getBar(result.score); // getBar returns a visual bar like "████████░░" // example: score 0.76 → "████████░░"
const label = getLabel(result.score); // getLabel returns a human readable string // example: score 0.76 → "🟢 Very similar"
console.log(`${index + 1}. "${result.textA}"`); console.log(` vs`); console.log(` "${result.textB}"`); console.log(` Score: ${result.score.toFixed(4)} ${bar} ${label}`); // .toFixed(4) = show 4 decimal places // example: 0.7634 console.log(); // empty line between results }); }
// ───────────────────────────────────────── // HELPER — getBar // Converts a 0-1 score into a visual bar // ─────────────────────────────────────────
function getBar(score) { // score = number between 0 and 1 // example: 0.76
const filled = Math.round(score * 10); // score * 10 = how many filled blocks out of 10 // example: 0.76 * 10 = 7.6 → rounds to 8 // filled = 8
const empty = 10 - filled; // empty = remaining blocks // example: 10 - 8 = 2
return "█".repeat(Math.max(0, filled)) + "░".repeat(Math.max(0, empty)); // "█".repeat(8) = "████████" // "░".repeat(2) = "░░" // result = "████████░░" // Math.max(0, ...) prevents negative repeat counts }
// ───────────────────────────────────────── // HELPER — getLabel // Converts a score to a human readable label // ─────────────────────────────────────────
function getLabel(score) { // score = number between -1.0 and 1.0
if (score > 0.9) return "🟢 Nearly identical meaning"; // example: "The patient needs medication" vs "The sick person requires medicine"
if (score > 0.75) return "🟢 Very similar"; // example: "cat" vs "dog"
if (score > 0.6) return "🟡 Somewhat similar"; // example: "cat" vs "fish"
if (score > 0.4) return "🟠 Weakly related"; // example: "cat" vs "pizza"
return "🔴 Very different"; // example: "pizza" vs "car" }
// ───────────────────────────────────────── // EXPERIMENT 1 — Single Words // Tests how similar basic words are to each other // ─────────────────────────────────────────
async function experiment1() { console.log("\n" + "═".repeat(60)); console.log("EXPERIMENT 1: Single Words"); console.log("═".repeat(60));
// words = array of single word strings // example: ["cat", "dog", "fish", "pizza", "car", "vehicle"] const words = [ "cat", // animal, pet "dog", // animal, pet "fish", // animal, pet (less similar to cat/dog) "pizza", // food — should be very different from animals "car", // vehicle "vehicle", // synonym of car — should be very similar to "car" ];
await compareTexts(words); // passes the words array to compareTexts // which embeds all of them and compares every pair }
// ───────────────────────────────────────── // EXPERIMENT 2 — Same meaning, different words // Proves embeddings capture meaning not just keywords // ─────────────────────────────────────────
async function experiment2() { console.log("\n" + "═".repeat(60)); console.log("EXPERIMENT 2: Same Meaning, Different Words"); console.log("═".repeat(60));
const sentences = [ "The patient needs medication", // medical sentence — "patient", "medication"
"The sick person requires medicine", // same meaning as above but completely different words // expected: HIGH similarity score with sentence above
"A drug is needed for the ill individual", // same meaning again, different words again // expected: HIGH similarity score with both above
"I want to order a pizza", // completely different topic — food // expected: LOW similarity score with medical sentences
"Please bring me some food", // similar to pizza sentence (both about food/eating) // expected: MEDIUM similarity with pizza sentence // expected: LOW similarity with medical sentences ];
await compareTexts(sentences); }
// ───────────────────────────────────────── // EXPERIMENT 3 — Questions vs Document Chunks // This is exactly how RAG retrieval works // ─────────────────────────────────────────
async function experiment3() { console.log("\n" + "═".repeat(60)); console.log("EXPERIMENT 3: Questions vs Document Chunks"); console.log("(This is exactly how RAG works)"); console.log("═".repeat(60));
const texts = [ // THE QUESTION — what a user types "What are the side effects of aspirin?",
// RELEVANT CHUNK 1 — directly answers the question // expected: HIGH similarity with the question "Aspirin can cause stomach bleeding and ulcers with long-term use",
// RELEVANT CHUNK 2 — also relevant to side effects // expected: HIGH similarity with the question "Common side effects of aspirin include nausea and stomach pain",
// LESS RELEVANT CHUNK — about aspirin but not side effects // expected: MEDIUM similarity (shares "aspirin" topic but different angle) "The history of aspirin dates back to ancient willow bark remedies",
// IRRELEVANT CHUNK — completely different topic // expected: LOW similarity with the question "Pizza margherita was invented in Naples Italy in 1889", ];
await compareTexts(texts); // in a real RAG system: // step 1 — all document chunks are pre-embedded and stored in vector DB // step 2 — user question gets embedded // step 3 — vector DB finds chunks with highest similarity to question // step 4 — top 2-3 chunks get injected into LLM prompt // step 5 — LLM answers based on those chunks }
// ───────────────────────────────────────── // MAIN — Run all experiments in order // ─────────────────────────────────────────
async function main() { console.log("\n🚀 EMBEDDING EXPLORER"); console.log("Generating real embeddings with OpenAI\n");
await experiment1(); // single words await experiment2(); // same meaning different words await experiment3(); // RAG simulation
console.log("\n✅ All experiments complete!"); console.log("\nKey takeaway:"); console.log("Similar meanings → scores close to 1.0"); console.log("Different meanings → scores close to 0.0"); }
// Start the whole program // .catch(console.error) = if anything goes wrong, print the error main().catch(console.error);
The code structure is:
main()
↓
experiment1() → compareTexts(words)
experiment2() → compareTexts(sentences)
experiment3() → compareTexts(texts)
↓
getEmbedding() × N ← calls OpenAI API
↓
cosineSimilarity() ← calculates scores
↓
getBar() + getLabel() ← formats output
↓
console.log() ← prints results
Every function has a clear job. Read top to bottom and each piece will make sense.
Update your package.json to add "type": "module":
{ "name": "embedding-explorer", "version": "1.0.0", "type": "module", "scripts": { "start": "node embeddings.js" } }
Step 3 — Run It
node embeddings.js
What You Should See
Here is roughly what the output will look like with real numbers:
🚀 EMBEDDING EXPLORER
Generating real embeddings with OpenAI
════════════════════════════════════════════════════════════
EXPERIMENT 1: Single Words
════════════════════════════════════════════════════════════
🔄 Getting embeddings from OpenAI...
✅ Got 6 embeddings
📐 Each embedding has 1536 dimensions
────────────────────────────────────────────────────────────
📊 SIMILARITY SCORES:
Higher score = more similar (max is 1.0, min is -1.0)
1. "car"
vs
"vehicle"
Score: 0.7821 ████████░░ 🟢 Very similar
2. "cat"
vs
"dog"
Score: 0.7634 ████████░░ 🟢 Very similar
3. "cat"
vs
"fish"
Score: 0.6821 ███████░░░ 🟡 Somewhat similar
4. "dog"
vs
"fish"
Score: 0.6543 ███████░░░ 🟡 Somewhat similar
5. "cat"
vs
"pizza"
Score: 0.3821 ████░░░░░░ 🟠 Weakly related
6. "pizza"
vs
"car"
Score: 0.2341 ██░░░░░░░░ 🔴 Very different
════════════════════════════════════════════════════════════
EXPERIMENT 2: Same Meaning, Different Words
════════════════════════════════════════════════════════════
📊 SIMILARITY SCORES:
1. "The patient needs medication"
vs
"The sick person requires medicine"
Score: 0.8923 █████████░ 🟢 Very similar
2. "The patient needs medication"
vs
"A drug is needed for the ill individual"
Score: 0.8341 ████████░░ 🟢 Very similar
3. "I want to order a pizza"
vs
"Please bring me some food"
Score: 0.7123 ████████░░ 🟢 Very similar
4. "The patient needs medication"
vs
"I want to order a pizza"
Score: 0.2234 ██░░░░░░░░ 🔴 Very different
════════════════════════════════════════════════════════════
EXPERIMENT 3: Questions vs Document Chunks
(This is exactly how RAG works)
════════════════════════════════════════════════════════════
📊 SIMILARITY SCORES:
1. "What are the side effects of aspirin?"
vs
"Common side effects of aspirin include nausea and stomach pain"
Score: 0.8834 █████████░ 🟢 Very similar
2. "What are the side effects of aspirin?"
vs
"Aspirin can cause stomach bleeding and ulcers with long-term use"
Score: 0.8421 ████████░░ 🟢 Very similar
3. "What are the side effects of aspirin?"
vs
"The history of aspirin dates back to ancient willow bark remedies"
Score: 0.6123 ██████░░░░ 🟡 Somewhat similar
4. "What are the side effects of aspirin?"
vs
"Pizza margherita was invented in Naples Italy in 1889"
Score: 0.1823 ██░░░░░░░░ 🔴 Very different
Reading the Results — What Does This Tell You?
Experiment 1 proves:
"car" vs "vehicle" → 0.78 (very similar)
"cat" vs "dog" → 0.76 (very similar — both pets/animals)
"cat" vs "pizza" → 0.38 (weakly related — both nouns but nothing else)
"pizza" vs "car" → 0.23 (very different — no relationship)
The model learned these relationships purely from reading text. Nobody told it "car and vehicle are synonyms."
Experiment 2 proves the key point:
"The patient needs medication"
vs
"The sick person requires medicine"
→ 0.89 (very similar)
Completely different words. Same meaning. High similarity score.
This is why semantic search beats keyword search. A keyword search would find zero overlap between these sentences. Embedding search correctly identifies them as nearly identical in meaning.
Experiment 3 shows RAG working:
Question: "What are the side effects of aspirin?"
Most similar chunk: "Common side effects include nausea..." → 0.88
Second most similar: "Aspirin can cause stomach bleeding..." → 0.84
Less relevant: "History of aspirin..." → 0.61
Not relevant: "Pizza margherita..." → 0.18
A RAG system would take the top 2 chunks — inject them into the LLM prompt — and the LLM answers based on real, relevant information.
You just watched RAG work — with real numbers.
What Just Happened — The Full Picture
Your text (words / sentences / questions)
↓
OpenAI text-embedding-3-small model
↓
Real vectors — 1536 numbers each
↓
Cosine similarity calculation
↓
Scores between 0 and 1
↓
Higher score = more similar meaning
↓
This is exactly how vector databases
find relevant documents in RAG
One More Thing — Look at the Raw Embedding
Add this to your code temporarily to see what an actual embedding looks like:
async function showRawEmbedding() { const embedding = await getEmbedding("cat");
console.log("\nRaw embedding for 'cat':"); console.log(`Total dimensions: ${embedding.length}`); console.log(`First 10 numbers: ${embedding.slice(0, 10)}`); console.log(`Last 10 numbers: ${embedding.slice(-10)}`); console.log(`\nMin value: ${Math.min(...embedding).toFixed(4)}`); console.log(`Max value: ${Math.max(...embedding).toFixed(4)}`); }
You'll see something like:
Raw embedding for 'cat':
Total dimensions: 1536
First 10 numbers: 0.0234, -0.0891, 0.0412, -0.0234, 0.0678...
Last 10 numbers: -0.0123, 0.0456, -0.0789, 0.0234, -0.0567...
Min value: -0.1823
Max value: 0.1654
These are the actual numbers. Small, positive and negative decimals. 1,536 of them. This IS the meaning of "cat" — captured mathematically.
Try Your Own Experiments
Change the texts array and see what happens:
// Try these — predict the scores before running
// Will these be similar? ["happy", "joyful", "excited"]
// What about these? ["king", "queen", "prince", "president"]
// Or these sentences? ["JavaScript is a programming language", "Python is used for coding", "I love eating sushi"]
// What about different languages? ["Hello, how are you?", "Hola, ¿cómo estás?", "Bonjour, comment allez-vous?"] // (hint: multilingual embeddings are surprisingly good)
3-Line Summary
- Real embeddings are just arrays of 1,536 small decimal numbers — and cosine similarity between two arrays gives a score from 0 to 1 showing how similar their meanings are.
- Experiment 2 proved the key point — completely different words with the same meaning score 0.89+ similarity — this is why semantic search finds relevant content that keyword search completely misses.
- Experiment 3 showed RAG working in real numbers — the question vector was most similar to relevant document chunks and least similar to unrelated ones — exactly how a vector database finds the right content to inject into an LLM.
Module 3.4 — Complete ✅
Phase 3 is done. 🎉
You now truly understand embeddings:
What they are → lists of numbers capturing meaning
Why they exist → to make meaning mathematically comparable
How they work → similar meanings = similar vectors = close in space
How to use them → OpenAI API → 1536 numbers → cosine similarity
Coming Up — Phase 4: Vector Databases
Module 4.1 — Why SQL is Not Enough
You understand embeddings. Now where do you store millions of them? And how do you search them in milliseconds? Regular databases completely fail at this. You need something built specifically for vectors. Next module explains exactly why — and what vector databases do differently.
No comments:
Post a Comment