Rebuilding the PDF chatbot in LangChain — see how much simpler it gets
What We're Doing
In Phase 5 you built a PDF chatbot manually — 7 files, ~370 lines.
Now we rebuild the exact same thing using LangChain components. Same functionality. Much less code. And because you built it manually first — you'll understand every line.
By the end of this module you'll have:
LangChain PDF Chatbot
├── .env
├── package.json
└── src/
├── 01_loaders.js ← Document Loaders
├── 02_splitters.js ← Text Splitters
├── 03_vectorstores.js ← Vector Stores
└── 04_rag_chain.js ← Complete RAG in ~40 lines
Project Setup
mkdir langchain-rag cd langchain-rag npm init -y
Update package.json:
{ "name": "langchain-rag", "version": "1.0.0", "type": "module", "scripts": { "start": "node src/04_rag_chain.js" } }
Install packages:
npm install langchain @langchain/core @langchain/openai @langchain/classic dotenv
Create .env:
OPENAI_API_KEY=sk-proj-your-key-here
Put any PDF in the project root — name it sample.pdf.
Part 1 — Document Loaders
What they do
Document loaders load content from different sources and return it as Document objects — the standard format LangChain uses everywhere.
// Every loader returns this same structure: [ { pageContent: "actual text from the source...", metadata: { source: "file.pdf", page: 1, // other source-specific info } } ]
Create src/01_loaders.js:
import fs from "fs";
import { PDFLoader } from "@langchain/community/document_loaders/fs/pdf"; // PDFLoader = loads PDF files // reads each page as a separate Document object
import { TextLoader } from "@langchain/classic/document_loaders/fs/text"; // TextLoader = loads plain .txt files
import { CSVLoader } from "@langchain/community/document_loaders/fs/csv"; // CSVLoader = loads CSV files // each row becomes a Document
import { CheerioWebBaseLoader } from "@langchain/community/document_loaders/web/cheerio"; // CheerioWebBaseLoader = loads web pages // scrapes HTML and extracts text
import * as dotenv from "dotenv"; dotenv.config();
// ───────────────────────────────────────── // LOADER 1 — PDF Loader // Loads a PDF file — each page = one Document // ─────────────────────────────────────────
async function loader1() { console.log("─".repeat(55)); console.log("LOADER 1: PDFLoader"); console.log("─".repeat(55));
const loader = new PDFLoader("./sample.pdf", { // splitPages: true = each page becomes separate Document (default) // splitPages: false = entire PDF becomes one Document splitPages: true, });
const docs = await loader.load(); // load() = reads the PDF and returns array of Documents // one Document per page
console.log(`Pages loaded: ${docs.length}`); // example: 5 (one per page)
// Show first document console.log("\nFirst page Document:"); console.log("pageContent preview:", docs[0].pageContent.substring(0, 150) + "..."); console.log("metadata:", docs[0].metadata); // metadata example: // { source: './sample.pdf', pdf: { version: '1.7', pages: 5 }, page: 0 }
console.log("\nAll pages:"); docs.forEach((doc, i) => { console.log(` Page ${i + 1}: ${doc.pageContent.length} characters`); });
console.log(); return docs; }
// ───────────────────────────────────────── // LOADER 2 — Text Loader // Loads a plain text file // ─────────────────────────────────────────
async function loader2() { console.log("─".repeat(55)); console.log("LOADER 2: TextLoader"); console.log("─".repeat(55));
// Create a sample text file first fs.writeFileSync("./sample.txt", `This is a sample text file. It contains information about AI engineering.
LangChain makes it easy to build AI applications. You can load documents from many different sources.` );
const loader = new TextLoader("./sample.txt"); const docs = await loader.load();
console.log(`Documents loaded: ${docs.length}`); // 1 — whole file as one Document
console.log("Content:", docs[0].pageContent); console.log("Metadata:", docs[0].metadata); // { source: './sample.txt' }
console.log(); return docs; }
// ───────────────────────────────────────── // LOADER 3 — Web Loader // Loads content from a web URL // ─────────────────────────────────────────
async function loader3() { console.log("─".repeat(55)); console.log("LOADER 3: CheerioWebBaseLoader"); console.log("─".repeat(55));
const loader = new CheerioWebBaseLoader( "https://en.wikipedia.org/wiki/Retrieval-augmented_generation", { selector: "p", // CSS selector — only extract <p> paragraph tags // ignores navigation, headers, footers // gives us clean article content } );
try { const docs = await loader.load();
console.log(`Documents loaded: ${docs.length}`); // Usually 1 — whole page as one Document
console.log("Content preview:", docs[0].pageContent.substring(0, 200) + "..."); console.log("Content length:", docs[0].pageContent.length, "characters"); console.log("Metadata:", docs[0].metadata);
} catch (error) { console.log("Web loader error (might be network issue):", error.message); }
console.log(); }
// ───────────────────────────────────────── // COMPARE — What you built vs LangChain // ─────────────────────────────────────────
function compareWithManual() { console.log("─".repeat(55)); console.log("COMPARISON: Manual vs LangChain"); console.log("─".repeat(55));
console.log(` Your manual pdfLoader.js: LangChain PDFLoader: ──────────────────────────── ──────────────────── import fs from "fs" import { PDFLoader } import pdfParse from "pdf-parse" from "@langchain/community/..." import { createRequire }... const loader = new PDFLoader(path) export async function loadPDF(p) { const docs = await loader.load() check file exists... read buffer... parse PDF... clean text... return { text, pageCount, ... } // Done — 2 lines } // ~40 lines // 2 lines `); }
async function main() { console.log("\n📄 DOCUMENT LOADERS DEMO\n"); await loader1(); await loader2(); await loader3(); compareWithManual(); console.log("✅ Loaders demo complete!"); }
main().catch(console.error);
Install the community package (has PDF and web loaders):
npm install @langchain/community pdf-parse cheerio
Before run your package.json look like this:
{
"name": "langchain-rag",
"version": "1.0.0",
"main": "index.js",
"type": "module",
"scripts": {
"start": "node src/04_rag_chain.js",
"test": "echo \"Error: no test specified\" && exit 1"
},
"keywords": [],
"author": "",
"license": "ISC",
"description": "",
"dependencies": {
"@langchain/classic": "^1.0.40",
"@langchain/community": "^1.1.29",
"@langchain/core": "^1.2.3",
"@langchain/openai": "^1.5.5",
"cheerio": "^1.2.0",
"dotenv": "^16.4.5",
"langchain": "^1.5.3",
"pdf-parse": "^2.4.5"
}
}
Run:
node src/01_loaders.js
Part 2 — Text Splitters
What they do
The same job as your chunker.js — split large documents into smaller chunks.
LangChain's RecursiveCharacterTextSplitter is smarter than most manual implementations — it tries multiple separators in order.
Create src/02_splitters.js:
import { RecursiveCharacterTextSplitter, CharacterTextSplitter, } from "@langchain/textsplitters"; // RecursiveCharacterTextSplitter = tries multiple separators in order // → first tries to split on \n\n (paragraphs) // → then \n (lines) // → then ". " (sentences) // → then " " (words) // → last resort: characters // This gives much cleaner chunks than splitting blindly
// CharacterTextSplitter = splits on one specific character only
import { PDFLoader } from "@langchain/community/document_loaders/fs/pdf"; import * as dotenv from "dotenv"; dotenv.config();
// ───────────────────────────────────────── // SPLITTER 1 — RecursiveCharacterTextSplitter // The most commonly used splitter in LangChain // This is what replaces your chunker.js // ─────────────────────────────────────────
async function splitter1() { console.log("─".repeat(55)); console.log("SPLITTER 1: RecursiveCharacterTextSplitter"); console.log("─".repeat(55));
const splitter = new RecursiveCharacterTextSplitter({ chunkSize: 800, // target size in characters per chunk // same as your chunker.js chunkSize
chunkOverlap: 150, // characters repeated between consecutive chunks // same as your chunker.js overlap
separators: ["\n\n", "\n", ". ", " ", ""], // tries these separators in ORDER // first tries paragraph breaks (\n\n) — cleanest split // then line breaks (\n) // then sentence ends (". ") // then word boundaries (" ") // last resort: split anywhere ("") // much smarter than your manual implementation });
// Sample text to split const text = ` Chapter 1: Introduction to RAG
Retrieval Augmented Generation is a technique that combines the power of large language models with the ability to retrieve relevant information from a knowledge base.
This approach solves two major problems with pure LLMs: knowledge cutoffs and hallucination. By retrieving real documents and injecting them into the prompt, RAG grounds the LLM's responses in actual information.
Chapter 2: How RAG Works
The RAG pipeline has two phases. The indexing phase processes documents: loading them, splitting them into chunks, converting to embeddings, and storing in a vector database.
The query phase handles user questions: embedding the question, searching for similar chunks, injecting them into a prompt, and generating a grounded answer.
Chapter 3: Implementation
Building a RAG system requires choosing the right components: a document loader for your data source, a text splitter for chunking, an embedding model for vectorization, a vector store for storage and retrieval, and an LLM for generating answers. `;
// Split plain text const chunks = await splitter.splitText(text); // splitText() = takes a string, returns array of strings
console.log(`Original text: ${text.length} characters`); console.log(`Chunks created: ${chunks.length}`);
chunks.forEach((chunk, i) => { console.log(`\nChunk ${i + 1} (${chunk.length} chars):`); console.log(chunk.substring(0, 100) + "..."); });
console.log(); }
// ───────────────────────────────────────── // SPLITTER 2 — Split Documents (not just text) // Works directly with Document objects from loaders // ─────────────────────────────────────────
async function splitter2() { console.log("─".repeat(55)); console.log("SPLITTER 2: splitDocuments — works with Document objects"); console.log("─".repeat(55));
// Load PDF const loader = new PDFLoader("./sample.pdf"); const docs = await loader.load(); // docs = array of Document objects (one per page)
console.log(`PDF pages loaded: ${docs.length}`);
const splitter = new RecursiveCharacterTextSplitter({ chunkSize: 600, chunkOverlap: 100, });
// Split Document objects — NOT just text strings const chunks = await splitter.splitDocuments(docs); // splitDocuments() = takes array of Documents // returns array of smaller Documents // AUTOMATICALLY preserves and updates metadata!
console.log(`Chunks after splitting: ${chunks.length}`);
// Show metadata preservation console.log("\nFirst chunk metadata:"); console.log(chunks[0].metadata); // metadata still has source, page number — automatically preserved // { // source: './sample.pdf', // page: 0, ← which page this chunk came from // loc: { ... } ← character location in original // }
console.log("\nSample chunks:"); chunks.slice(0, 3).forEach((chunk, i) => { console.log(`\nChunk ${i + 1}:`); console.log(` Content: ${chunk.pageContent.substring(0, 80)}...`); console.log(` Metadata: page=${chunk.metadata.page}, source=${chunk.metadata.source}`); });
console.log(); return chunks; }
// ───────────────────────────────────────── // SPLITTER 3 — Comparing chunk sizes // See how different settings affect output // ─────────────────────────────────────────
async function splitter3() { console.log("─".repeat(55)); console.log("SPLITTER 3: Comparing different chunk sizes"); console.log("─".repeat(55));
const loader = new PDFLoader("./sample.pdf"); const docs = await loader.load();
const settings = [ { chunkSize: 200, chunkOverlap: 50, label: "Small chunks" }, { chunkSize: 600, chunkOverlap: 100, label: "Medium chunks (recommended)" }, { chunkSize: 1200, chunkOverlap: 200, label: "Large chunks" }, ];
for (const setting of settings) { const splitter = new RecursiveCharacterTextSplitter({ chunkSize: setting.chunkSize, chunkOverlap: setting.chunkOverlap, });
const chunks = await splitter.splitDocuments(docs);
console.log(`\n${setting.label} (size=${setting.chunkSize}, overlap=${setting.chunkOverlap}):`); console.log(` Total chunks: ${chunks.length}`); console.log(` Avg chunk size: ${Math.round( chunks.reduce((sum, c) => sum + c.pageContent.length, 0) / chunks.length )} chars`); }
console.log("\nKey insight:"); console.log("More chunks = more precise retrieval but more API calls to embed"); console.log("Fewer chunks = cheaper but might miss specific details"); console.log(); }
// ───────────────────────────────────────── // COMPARISON with your manual chunker // ─────────────────────────────────────────
function compareWithManual() { console.log("─".repeat(55)); console.log("COMPARISON: Manual vs LangChain"); console.log("─".repeat(55));
console.log(` Your manual chunker.js: LangChain: ──────────────────────────── ────────────────────────── 80 lines of while loop logic const splitter = new handling boundaries, overlap, RecursiveCharacterTextSplitter({ edge cases, infinite loop bugs... chunkSize: 800, chunkOverlap: 150 }) const chunks = await splitter.splitDocuments(docs) // 4 lines. Battle tested. // Handles all edge cases. // Never gets infinite loops. `); }
async function main() { console.log("\n✂️ TEXT SPLITTERS DEMO\n"); await splitter1(); await splitter2(); await splitter3(); compareWithManual(); console.log("✅ Splitters demo complete!"); }
main().catch(console.error);
Run:
node src/02_splitters.js
Part 3 — Vector Stores
What they do
Store embeddings and search for similar content — same as your vectorStore.js but pre-built.
Create src/03_vectorstores.js:
import { OpenAIEmbeddings } from "@langchain/openai"; // OpenAIEmbeddings = generates embeddings using OpenAI API // replaces your getEmbedding() function
import { MemoryVectorStore } from "@langchain/classic/vectorstores/memory"; // MemoryVectorStore = in-memory vector store // same as your store[] array — but built-in // no server needed — perfect for development
import { PDFLoader } from "@langchain/community/document_loaders/fs/pdf"; import { RecursiveCharacterTextSplitter } from "@langchain/textsplitters"; import * as dotenv from "dotenv"; dotenv.config();
// ───────────────────────────────────────── // SETUP — Embeddings model // ─────────────────────────────────────────
const embeddings = new OpenAIEmbeddings({ model: "text-embedding-3-small", // same model you used manually // 1536 dimensions });
// ───────────────────────────────────────── // EXAMPLE 1 — Create vector store from documents // ─────────────────────────────────────────
async function example1() { console.log("─".repeat(55)); console.log("EXAMPLE 1: Create vector store from documents"); console.log("─".repeat(55));
// Load and split PDF const loader = new PDFLoader("./sample.pdf"); const docs = await loader.load();
const splitter = new RecursiveCharacterTextSplitter({ chunkSize: 800, chunkOverlap: 150, }); const chunks = await splitter.splitDocuments(docs);
console.log(`Chunks to embed: ${chunks.length}`); console.log("Embedding and storing... (calls OpenAI API)");
// Create vector store from chunks const vectorStore = await MemoryVectorStore.fromDocuments( chunks, // documents to embed and store
embeddings, // embedding model to use // LangChain calls OpenAI for each chunk automatically ); // fromDocuments() does everything: // 1. Calls OpenAI to embed each chunk // 2. Stores embedding + text + metadata in memory // 3. Returns ready-to-search vector store
console.log("✅ Vector store created!\n"); return vectorStore; }
// ───────────────────────────────────────── // EXAMPLE 2 — Similarity search // ─────────────────────────────────────────
async function example2(vectorStore) { console.log("─".repeat(55)); console.log("EXAMPLE 2: Similarity search"); console.log("─".repeat(55));
// Basic similarity search const results = await vectorStore.similaritySearch( "What is this document about?", // query string — LangChain embeds this automatically
4, // top K results to return ); // returns array of Document objects — most similar first
console.log(`Found ${results.length} similar chunks\n`);
results.forEach((doc, i) => { console.log(`Result ${i + 1}:`); console.log(` Content: ${doc.pageContent.substring(0, 100)}...`); console.log(` Source: ${doc.metadata.source}`); console.log(` Page: ${doc.metadata.page}`); console.log(); }); }
// ───────────────────────────────────────── // EXAMPLE 3 — Similarity search with scores // See the actual similarity scores // ─────────────────────────────────────────
async function example3(vectorStore) { console.log("─".repeat(55)); console.log("EXAMPLE 3: Similarity search with scores"); console.log("─".repeat(55));
const resultsWithScores = await vectorStore.similaritySearchWithScore( "main topic", // query
5, // top 5 results ); // returns array of [Document, score] pairs // score = similarity score (higher = more similar in MemoryVectorStore)
console.log("Results with similarity scores:\n");
resultsWithScores.forEach(([doc, score], i) => { // destructure [Document, score] pair const bar = "█".repeat(Math.round(score * 10)); console.log(`${i + 1}. Score: ${score.toFixed(4)} ${bar}`); console.log(` ${doc.pageContent.substring(0, 80)}...`); console.log(); }); }
// ───────────────────────────────────────── // EXAMPLE 4 — Retriever interface // Cleaner way to use vector store in chains // ─────────────────────────────────────────
async function example4(vectorStore) { console.log("─".repeat(55)); console.log("EXAMPLE 4: Retriever interface"); console.log("─".repeat(55));
// Convert vector store to a retriever const retriever = vectorStore.asRetriever({ k: 4, // return top 4 results
searchType: "similarity", // "similarity" = cosine similarity search (default) // "mmr" = Maximum Marginal Relevance // returns diverse results — avoids repetitive chunks });
// Use retriever directly const docs = await retriever.invoke("What topics are covered?"); // invoke() = same as similaritySearch() but cleaner interface
console.log(`Retrieved ${docs.length} documents\n`); docs.forEach((doc, i) => { console.log(`Doc ${i + 1}: ${doc.pageContent.substring(0, 80)}...`); });
console.log(); return retriever; }
// ───────────────────────────────────────── // EXAMPLE 5 — MMR search // Maximum Marginal Relevance // Returns diverse results — avoids duplicate content // ─────────────────────────────────────────
async function example5(vectorStore) { console.log("─".repeat(55)); console.log("EXAMPLE 5: MMR search (Maximum Marginal Relevance)"); console.log("─".repeat(55));
console.log("Regular similarity search — might return similar/duplicate chunks:"); const regular = await vectorStore.similaritySearch("main topic", 3); regular.forEach((doc, i) => { console.log(` ${i + 1}. ${doc.pageContent.substring(0, 60)}...`); });
console.log("\nMMR search — returns diverse, non-redundant chunks:"); const diverse = await vectorStore.maxMarginalRelevanceSearch( "main topic", // query
{ k: 3, // return 3 results
fetchK: 10, // fetch top 10 candidates first // then pick 3 most diverse from those 10 // more fetchK = more diverse but slower
lambda: 0.5, // balance between relevance and diversity // 0 = pure diversity (ignore relevance) // 1 = pure relevance (same as regular search) // 0.5 = balanced (good default) } );
diverse.forEach((doc, i) => { console.log(` ${i + 1}. ${doc.pageContent.substring(0, 60)}...`); });
console.log("\nMMR is better for RAG — avoids injecting the same"); console.log("information multiple times into the LLM prompt."); console.log(); }
async function main() { console.log("\n🗄️ VECTOR STORES DEMO\n");
const vectorStore = await example1(); await example2(vectorStore); await example3(vectorStore); await example4(vectorStore); await example5(vectorStore);
console.log("✅ Vector stores demo complete!"); }
main().catch(console.error);
Run:
node src/03_vectorstores.js
Part 4 — Complete RAG Chain
Now everything together — the full PDF chatbot in ~50 lines.
Create src/04_rag_chain.js:
import { PDFLoader } from "@langchain/community/document_loaders/fs/pdf"; import { RecursiveCharacterTextSplitter } from "@langchain/textsplitters"; import { OpenAIEmbeddings, ChatOpenAI } from "@langchain/openai"; import { MemoryVectorStore } from "@langchain/classic/vectorstores/memory"; import { ChatPromptTemplate, MessagesPlaceholder } from "@langchain/core/prompts"; import { StringOutputParser } from "@langchain/core/output_parsers"; import { RunnablePassthrough, RunnableSequence } from "@langchain/core/runnables"; import * as readline from "readline"; import * as dotenv from "dotenv"; dotenv.config();
// ───────────────────────────────────────── // STEP 1 — Load PDF // ─────────────────────────────────────────
async function loadAndSplitPDF(filePath) { console.log(`\n📄 Loading PDF: ${filePath}`);
const loader = new PDFLoader(filePath); const docs = await loader.load(); // docs = array of Documents (one per page)
console.log(` Loaded ${docs.length} pages`);
const splitter = new RecursiveCharacterTextSplitter({ chunkSize: 800, chunkOverlap: 150, });
const chunks = await splitter.splitDocuments(docs); // chunks = array of smaller Documents
console.log(` Split into ${chunks.length} chunks`); return chunks; }
// ───────────────────────────────────────── // STEP 2 — Create Vector Store // ─────────────────────────────────────────
async function createVectorStore(chunks) { console.log("\n💾 Creating vector store..."); console.log(` Embedding ${chunks.length} chunks with OpenAI...`);
const vectorStore = await MemoryVectorStore.fromDocuments( chunks, new OpenAIEmbeddings({ model: "text-embedding-3-small" }) );
console.log(" ✅ Vector store ready\n"); return vectorStore; }
// ───────────────────────────────────────── // STEP 3 — Build RAG Chain // This is the magic — all pieces connected // ─────────────────────────────────────────
function buildRAGChain(vectorStore) {
const retriever = vectorStore.asRetriever({ k: 5, // retrieve top 5 most relevant chunks searchType: "mmr", // use MMR for diverse results });
const llm = new ChatOpenAI({ model: "gpt-4o", temperature: 0.1, // low temp = factual, consistent answers });
// Format retrieved documents into a string function formatDocs(docs) { // docs = array of Document objects from retriever return docs .map((doc, index) => `[Source ${index + 1} — Page ${doc.metadata.page + 1}]\n${doc.pageContent}` ) .join("\n\n" + "─".repeat(40) + "\n\n"); // formats each chunk with source number and page // joins with clear separator // example output: // [Source 1 — Page 3] // text content here... // ──────────────────── // [Source 2 — Page 5] // more content here... }
// RAG prompt — forces grounded answers const ragPrompt = ChatPromptTemplate.fromMessages([ [ "system", `You are a helpful assistant that answers questions about documents.
STRICT RULES: 1. Answer ONLY using information from the context below 2. If answer is not in context — say "This is not covered in the document" 3. Always mention which Source number your answer comes from 4. Be concise and direct
Context: {context}` ], ["human", "{question}"], ]);
// Build the complete RAG chain using LCEL const ragChain = RunnableSequence.from([ { context: retriever.pipe( (docs) => formatDocs(docs) ), // retriever gets relevant docs → formatDocs converts to string // result stored as "context" variable for the prompt
question: new RunnablePassthrough(), // RunnablePassthrough = pass the question through unchanged // result stored as "question" variable for the prompt }, ragPrompt, // fills {context} and {question} in the template
llm, // calls GPT-4o with the filled prompt
new StringOutputParser(), // extracts string from AIMessage response ]);
return ragChain; }
// ───────────────────────────────────────── // STEP 4 — Interactive Chat // ─────────────────────────────────────────
async function startChat(ragChain) { const rl = readline.createInterface({ input: process.stdin, output: process.stdout, });
function prompt(question) { return new Promise(resolve => rl.question(question, resolve)); }
console.log("=".repeat(55)); console.log("✅ PDF Chatbot Ready!"); console.log(' Type your question or "quit" to exit'); console.log("=".repeat(55) + "\n");
while (true) { const question = await prompt("You: "); const trimmed = question.trim();
if (!trimmed) continue; // skip empty input
if (trimmed.toLowerCase() === "quit") { console.log("\n👋 Goodbye!\n"); rl.close(); break; }
try { console.log("\n🤖 Thinking...\n");
// Stream the response for better UX const stream = await ragChain.stream(trimmed); // stream() = get response token by token
process.stdout.write("Assistant: "); for await (const chunk of stream) { process.stdout.write(chunk); // print each token as it arrives } console.log("\n");
} catch (error) { console.log(`Error: ${error.message}\n`); } }
process.exit(0); }
// ───────────────────────────────────────── // MAIN — Tie everything together // ─────────────────────────────────────────
async function main() { console.log("\n" + "=".repeat(55)); console.log("🤖 LANGCHAIN PDF CHATBOT"); console.log("=".repeat(55));
const pdfPath = process.argv[2] || "./sample.pdf"; // process.argv[2] = first command line argument // example: node src/04_rag_chain.js ./my-document.pdf // if not provided — defaults to ./sample.pdf
try { // Index the PDF const chunks = await loadAndSplitPDF(pdfPath); const vectorStore = await createVectorStore(chunks); const ragChain = buildRAGChain(vectorStore);
// Start chatting await startChat(ragChain);
} catch (error) { if (error.message.includes("not found")) { console.log(`\n❌ PDF not found: ${pdfPath}`); console.log("Usage: node src/04_rag_chain.js <path-to-pdf>"); } else { console.log(`\n❌ Error: ${error.message}`); } process.exit(1); } }
main().catch(console.error);
Run it:
node src/04_rag_chain.js ./sample.pdf
Or with a different PDF:
node src/04_rag_chain.js ./any-document.pdf
Manual vs LangChain — Final Comparison
YOUR MANUAL PDF CHATBOT:
────────────────────────
pdfLoader.js → 40 lines
chunker.js → 85 lines
vectorStore.js → 95 lines
retriever.js → 45 lines
generator.js → 65 lines
chatbot.js → 70 lines
index.js → 65 lines
────────────────────────────
Total: → 465 lines
7 files
LANGCHAIN PDF CHATBOT:
──────────────────────
04_rag_chain.js → ~150 lines
1 file
────────────────────────────
Same functionality. 3x less code.
Battle-tested components.
No infinite loop bugs.
Streaming built in.
MMR search built in.
3-Line Summary
- LangChain document loaders all return the same
Documentformat —{ pageContent, metadata }— so you can swap a PDFLoader for a WebLoader or CSVLoader without changing any downstream code. RecursiveCharacterTextSplitterreplaces your manual chunker — it tries multiple separators in order (paragraphs → lines → sentences → words) giving cleaner chunks without infinite loop risks, andsplitDocuments()automatically preserves metadata.MemoryVectorStore.fromDocuments()replaces your entire vectorStore.js — one line embeds all chunks, stores them, and returns a searchable store that supports similarity search, MMR search, and the retriever interface used in LCEL chains.
Module 6.4 — Complete ✅
Coming up — Module 6.5 — Retrievers & Agents in LangChain
The final module of Phase 6. We go deep on advanced retriever patterns — contextual compression, multi-query retrieval — and build a LangChain agent with tools that can reason and take actions autonomously.
No comments:
Post a Comment