Now that you've built everything manually — LangChain will make complete sense
Start With What You Just Did
In Phase 5 you built a PDF chatbot from scratch. Think about how many files you created:
pdfLoader.js → load and extract PDF text
chunker.js → split text into chunks
vectorStore.js → embed and store chunks, search them
retriever.js → find relevant chunks for a question
generator.js → build prompt and call LLM
chatbot.js → tie everything together
index.js → run the whole thing
7 files. Hundreds of lines of code. Just for one RAG application.
Now imagine you need to build a different AI application — say a web scraper chatbot that answers questions about websites instead of PDFs.
You'd need to:
Replace pdfLoader.js → with a web scraper loader
Keep chunker.js → same chunking logic
Keep vectorStore.js → same embedding and search
Keep retriever.js → same retrieval logic
Keep generator.js → mostly same prompt logic
Rewrite chatbot.js → new pipeline
And for a CSV chatbot? Replace loader again. For a YouTube transcript chatbot? Replace loader again.
You're rewriting the same patterns over and over.
This is exactly the problem LangChain solves.
What is LangChain?
LangChain is a framework — a collection of pre-built components for building AI applications.
Instead of writing everything from scratch every time — you plug together pre-built pieces.
Without LangChain:
You write → PDF loader, chunker, embedder,
vector store, retriever, prompt builder,
LLM caller, output parser...
every single time for every project
With LangChain:
You import → PDFLoader, TextSplitter, OpenAIEmbeddings,
MemoryVectorStore, RetrievalQAChain...
plug them together → done
Think of it like Express.js for AI applications.
Web development without Express:
→ Write your own HTTP server
→ Write your own routing
→ Write your own middleware
→ Write your own request parsing
→ Hundreds of lines for a simple API
Web development with Express:
→ app.get('/route', handler)
→ Done in 10 lines
AI development without LangChain:
→ Write your own loader
→ Write your own chunker
→ Write your own vector store
→ Write your own retriever
→ Hundreds of lines for a simple RAG app
AI development with LangChain:
→ new PDFLoader() | new RecursiveCharacterTextSplitter()
→ Done in 20 lines
What LangChain Actually Provides
LangChain is organized into several categories of components:
Category 1 — Document Loaders
Load data from any source:
// PDF filesimport { PDFLoader } from "langchain/document_loaders/fs/pdf";// Web pagesimport { CheerioWebBaseLoader } from "langchain/document_loaders/web/cheerio";// CSV filesimport { CSVLoader } from "langchain/document_loaders/fs/csv";// YouTube transcriptsimport { YoutubeLoader } from "langchain/document_loaders/web/youtube";// Notion pagesimport { NotionLoader } from "langchain/document_loaders/fs/notion";// GitHub reposimport { GithubRepoLoader } from "langchain/document_loaders/web/github";All loaders return the same format — array of Document objects:// Every loader returns this same structure[{pageContent: "actual text content here...",metadata: {source: "document.pdf",page: 1,// other source-specific metadata}},// more documents...]
Same interface — swap the loader — same code works for any source.
Category 2 — Text Splitters
Split documents into chunks:
// Most commonly used — splits intelligently on paragraphs, sentences, words import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";
const splitter = new RecursiveCharacterTextSplitter({ chunkSize: 800, // target chunk size in characters
chunkOverlap: 150, // overlap between chunks
separators: ["\n\n", "\n", ". ", " ", ""], // try to split on these — in order of preference // first tries paragraph breaks, then newlines, then sentences... });
const chunks = await splitter.splitDocuments(documents); // takes array of Document objects // returns array of smaller Document objects
This is exactly what your chunker.js did — but pre-built and battle-tested.
Category 3 — Embeddings
Convert text to vectors:
import { OpenAIEmbeddings } from "@langchain/openai";
const embeddings = new OpenAIEmbeddings({ model: "text-embedding-3-small", // which OpenAI embedding model to use });
// Embed a single text const vector = await embeddings.embedQuery("What is aspirin?"); // returns [0.23, -0.87, ...] — 1536 numbers
// Embed multiple texts at once const vectors = await embeddings.embedDocuments(["text1", "text2"]); // returns array of vectors
Category 4 — Vector Stores
Store and search embeddings:
// In-memory (no server needed — like your vectorStore.js) import { MemoryVectorStore } from "langchain/vectorstores/memory";
// Chroma (local server) import { Chroma } from "@langchain/community/vectorstores/chroma";
// Pinecone (cloud) import { PineconeStore } from "@langchain/pinecone";
// All vector stores have the same interface: const vectorStore = await MemoryVectorStore.fromDocuments( chunks, // array of Document objects to store
embeddings, // embeddings model to use );
// Search const results = await vectorStore.similaritySearch( "What are aspirin side effects?", // query text
5, // top K results );
Same code works with any vector store — swap MemoryVectorStore for Chroma or PineconeStore — nothing else changes.
Category 5 — LLMs and Chat Models
Call language models:
import { ChatOpenAI } from "@langchain/openai";
const llm = new ChatOpenAI({ model: "gpt-4o", temperature: 0.1, maxTokens: 1000, });
// Simple call const response = await llm.invoke("What is RAG?"); console.log(response.content); // "RAG stands for Retrieval Augmented Generation..."
// With messages import { HumanMessage, SystemMessage } from "@langchain/core/messages";
const response = await llm.invoke([ new SystemMessage("You are a helpful assistant"), new HumanMessage("What is RAG?"), ]);
Category 6 — Prompt Templates
Build prompts dynamically:
import { ChatPromptTemplate } from "@langchain/core/prompts";
const prompt = ChatPromptTemplate.fromMessages([ ["system", `You are a helpful assistant. Answer ONLY from the provided context. Context: {context}`], // {context} = placeholder — filled at runtime
["human", "{question}"], // {question} = placeholder — filled at runtime ]);
// Fill the template const filledPrompt = await prompt.formatMessages({ context: "Aspirin reduces fever and pain...", question: "What does aspirin do?", }); // returns array of formatted messages ready for LLM
Category 7 — Chains
Connect components into pipelines:
import { RetrievalQAChain } from "langchain/chains";
// This one chain replaces your entire: // retriever.js + generator.js + chatbot.js combined
const chain = RetrievalQAChain.fromLLM( llm, // the language model to use
vectorStore.asRetriever(), // the vector store to search );
const result = await chain.call({ query: "What are aspirin side effects?" });
console.log(result.text); // Answer based on retrieved documents
One chain. Replaces hundreds of lines of manual code.
Category 8 — Output Parsers
Parse LLM output into structured formats:
import { StructuredOutputParser } from "langchain/output_parsers"; import { z } from "zod";
const parser = StructuredOutputParser.fromZodSchema( z.object({ answer: z.string().describe("the answer to the question"), confidence: z.enum(["high", "medium", "low"]), sources: z.array(z.string()).describe("sources used"), }) );
// LLM output automatically parsed into a JavaScript object const result = await parser.parse(llmOutput); // result = { answer: "...", confidence: "high", sources: [...] }
LangChain's Core Concept — LCEL
LangChain Expression Language (LCEL) is how you chain components together using the pipe operator |:
import { RunnableSequence } from "@langchain/core/runnables";
// The pipe operator chains components: // output of one becomes input of next
const chain = prompt | llm | outputParser; // prompt → formats input // llm → generates response // parser → structures output
const result = await chain.invoke({ context: retrievedChunks, question: userQuestion, });
This is like Unix pipes but for AI:
Unix: cat file.txt | grep "error" | sort | uniq
LCEL: prompt | llm | parser
Clean. Readable. Composable.
LangChain JS vs LangChain Python
LangChain exists in both Python and JavaScript, and we use JavaScript.
Python LangChain: langchain (pip)
JavaScript LangChain: langchain + @langchain/core + @langchain/openai
Most tutorials online use Python.
We use JavaScript — same concepts, JS syntax.
Package structure in JS:
@langchain/core → base interfaces (prompts, messages, runnables)
@langchain/openai → OpenAI specific (ChatOpenAI, OpenAIEmbeddings)
@langchain/community → community integrations (Chroma, Pinecone, etc.)
langchain → higher level chains and utilities
What LangChain is NOT
Important to understand what LangChain doesn't do:
LangChain is NOT:
→ A database (it uses your existing DBs)
→ An LLM (it calls OpenAI, Anthropic, etc.)
→ Magic (it's just well-organized code)
→ Required (you can build without it — you just did)
→ Always the right choice (sometimes simpler is better)
LangChain IS:
→ A collection of pre-built components
→ A standard interface for AI building blocks
→ A way to avoid reinventing the wheel
→ Useful for complex multi-step AI applications
Manual vs LangChain — Side by Side
Here's the same PDF chatbot — your manual version vs LangChain version:
Your manual version (Phase 5):
// pdfLoader.js — 40 lines
// chunker.js — 80 lines
// vectorStore.js — 90 lines
// retriever.js — 40 lines
// generator.js — 60 lines
// chatbot.js — 60 lines
// Total: ~370 lines across 6 files
LangChain version (what we'll build in Module 6.4):
import { PDFLoader } from "langchain/document_loaders/fs/pdf"; import { RecursiveCharacterTextSplitter } from "langchain/text_splitter"; import { OpenAIEmbeddings, ChatOpenAI } from "@langchain/openai"; import { MemoryVectorStore } from "langchain/vectorstores/memory"; import { RetrievalQAChain } from "langchain/chains";
// Load PDF const loader = new PDFLoader("./sample.pdf"); const docs = await loader.load();
// Split into chunks const splitter = new RecursiveCharacterTextSplitter({ chunkSize: 800, chunkOverlap: 150, }); const chunks = await splitter.splitDocuments(docs);
// Embed and store const vectorStore = await MemoryVectorStore.fromDocuments( chunks, new OpenAIEmbeddings({ model: "text-embedding-3-small" }) );
// Create RAG chain const chain = RetrievalQAChain.fromLLM( new ChatOpenAI({ model: "gpt-4o", temperature: 0.1 }), vectorStore.asRetriever({ k: 5 }) );
// Ask question const result = await chain.call({ query: "What are the main topics in this PDF?" });
console.log(result.text); // Total: ~30 lines. One file.
Same functionality. 370 lines → 30 lines.
But — and this is important — you understand every single line of those 30 lines because you built the full version first. Most developers who start with LangChain have no idea what's happening underneath. You do.
3-Line Summary
- LangChain is a framework of pre-built AI components — loaders, splitters, embeddings, vector stores, LLMs, chains — so you don't rewrite the same patterns for every project.
- Every LangChain component has a standard interface — swap a PDF loader for a web loader, swap MemoryVectorStore for Pinecone — the rest of your code stays the same.
- You understand LangChain better than most developers because you built everything manually first — LangChain is just a cleaner version of what you already know how to do.
Module 6.1 — Complete ✅
Coming up — Module 6.2 — Models, Prompts & Chains
We write real LangChain code. You'll see how ChatOpenAI, PromptTemplate, and LCEL chains work together — and build your first LangChain pipeline from scratch.
No comments:
Post a Comment