Making your RAG system production-ready — responses appear word by word, errors handled gracefully, and a clean API for your frontend
What This Module Covers
Part 1 → Streaming — response appears word by word like ChatGPT
Part 2 → Error Handling — every failure point handled gracefully
Part 3 → Express API — wrap everything in an API your Next.js can call
Install New Packages
npm install express cors @langchain/langgraph
express → HTTP server for the API
cors → allows frontend to call the API
@langchain/langgraph → MemorySaver for conversation checkpointing
Final package.json look like this
{ "name": "langchain-production-rag", "version": "1.0.0", "type": "module", "main": "index.js", "scripts": { "start": "node src/rag.js", "test": "echo \"Error: no test specified\" && exit 1" }, "keywords": [], "author": "", "license": "ISC", "description": "", "dependencies": { "@langchain/community": "^1.1.29", "@langchain/core": "^1.2.3", "@langchain/langgraph": "^1.4.8", "@langchain/openai": "^1.5.5", "@langchain/textsplitters": "^1.0.1", "cors": "^2.8.6", "dotenv": "^16.4.5", "express": "^5.2.1", "langchain": "^1.5.3", "pdf-parse": "^2.4.5" } }
Part 1 — Streaming
What Streaming Is
Without streaming:
User asks question
↓
Wait 5-10 seconds (nothing on screen)
↓
Full answer appears all at once
With streaming:
User asks question
↓
Words start appearing immediately
↓
Answer builds up word by word (like ChatGPT)
↓
User sees progress — feels much faster
How LangChain Streaming Works
LangChain's latest API has five streaming modes:
streamMode: "messages" → stream LLM tokens word by word
streamMode: "updates" → stream agent step updates
streamMode: "custom" → stream custom data from tools
streamMode: "values" → stream complete state after each step
streamMode: "debug" → stream detailed execution events for tracing & debuggingFor a RAG chatbot — "messages" is what you want. It gives you each token as the LLM generates it.
Create src/streaming.js:
// ─────────────────────────────────────────
// IMPORTS
// ─────────────────────────────────────────
import { createAgent, tool } from "langchain";
// createAgent = builds an agent with LLM + tools + memory
// tool = wraps any JS function into a tool the agent can call
import { MemorySaver } from "@langchain/langgraph";
// MemorySaver = stores conversation state in memory
// Required when using thread_id for persistent memory across turns
import { ChatOpenAI } from "@langchain/openai";
// ChatOpenAI = OpenAI's chat model wrapper
import { z } from "zod";
// zod = schema validation — defines what inputs each tool accepts
import * as dotenv from "dotenv";
dotenv.config();
// reads OPENAI_API_KEY from .env file
// ─────────────────────────────────────────
// SIMPLE TOOL
// ─────────────────────────────────────────
const getInfoTool = tool(
async ({ topic }) => {
// This function runs when the agent decides to use this tool
// topic = the string the agent passes in
// example: topic = "RAG" or "embeddings" or "LangChain"
const info = {
"RAG": "RAG stands for Retrieval Augmented Generation. It combines LLMs with document retrieval to give grounded answers.",
"embeddings": "Embeddings convert text into vectors — lists of numbers that capture semantic meaning.",
"LangChain": "LangChain is a framework for building AI applications with pre-built components.",
};
return info[topic] || `No specific information found about "${topic}".`;
// return value goes back to the agent as the tool result
// agent then uses this to compose its final answer
},
{
name: "get_info",
description: "Gets information about AI topics. Use when asked about RAG, embeddings, or LangChain.",
schema: z.object({
topic: z.string().describe("the AI topic to look up"),
}),
}
);
// ─────────────────────────────────────────
// CREATE AGENT
// ─────────────────────────────────────────
const checkpointer = new MemorySaver();
// MemorySaver = stores conversation checkpoints in memory
// Each thread_id gets its own saved state
// When the same thread_id is used again → agent remembers previous messages
const agent = createAgent({
model: new ChatOpenAI({ model: "gpt-4o", temperature: 0.1 }),
// temperature: 0.1 = mostly deterministic, slight natural variation
tools: [getInfoTool],
// list of tools this agent can call
checkpointer,
// enables memory — without this thread_id does nothing
systemPrompt: "You are a helpful AI assistant. Use your tools to answer questions accurately.",
// system prompt = hidden instructions that shape agent behavior
});
// ─────────────────────────────────────────
// EXAMPLE 1 — Basic token streaming
// ─────────────────────────────────────────
async function example1() {
console.log("─".repeat(55));
console.log("EXAMPLE 1: Basic token streaming");
console.log("─".repeat(55));
const threadId = crypto.randomUUID();
// generates a unique ID like "a1b2-c3d4-e5f6-..."
// used to identify this specific conversation thread
const config = {
configurable: { thread_id: threadId },
// thread_id = which conversation to load/save from MemorySaver
};
console.log("\nQuestion: What is RAG?\n");
process.stdout.write("Answer: ");
// process.stdout.write = print without adding a newline
// so all tokens appear on the same line as "Answer: "
for await (const [token, metadata] of await agent.stream(
// agent.stream() = run the agent but yield pieces as they generate
// instead of waiting for the full response
{ messages: [{ role: "user", content: "What is RAG?" }] },
// messages = the conversation so far (just the current question here)
// role: "user" = this message is from the human
{ ...config, streamMode: "messages" }
// streamMode: "messages" = yield one [token, metadata] per LLM token
// token = AIMessageChunk — a small piece of the response
// metadata = info about where this token came from in the agent graph
)) {
// token = AIMessageChunk object
// example: AIMessageChunk { content: "RAG", id: "abc" }
// example: AIMessageChunk { content: " stands", id: "abc" }
// example: AIMessageChunk { content: " for", id: "abc" }
// example: AIMessageChunk { content: [{"type":"tool_use",...}] }
// ↑ this is a tool call — content is an array, not a string
//
// metadata = { langgraph_node: "???" }
// NOTE: the exact node name depends on createAgent internals
// We do NOT filter by node name — just check if content is a string
const content = token.content;
// token.content = the actual payload of this chunk
// could be a string (text token) or array (tool call) or empty string ""
if (typeof content === "string" && content.length > 0) {
// typeof content === "string" → it's a text token, not a tool call array
// content.length > 0 → it has actual characters (not empty "")
//
// example: content = "RAG" → print it ✓
// example: content = " stands" → print it ✓
// example: content = "" → skip (empty chunk) ✗
// example: content = [{...}] → typeof is "object", not "string" → skip ✗
process.stdout.write(content);
// print this token immediately without newline
// builds up the full response word by word on screen
}
}
console.log("\n");
// newline after all tokens have been printed
}
// ─────────────────────────────────────────
// EXAMPLE 2 — Streaming with memory
// Second question remembers the first
// ─────────────────────────────────────────
async function example2() {
console.log("─".repeat(55));
console.log("EXAMPLE 2: Streaming with conversation memory");
console.log("─".repeat(55));
const threadId = crypto.randomUUID();
// new unique thread for this 2-turn conversation
const config = { configurable: { thread_id: threadId } };
// ── Turn 1 ────────────────────────────────────────────────
console.log("\nTurn 1 — Question: Tell me about embeddings");
process.stdout.write("Answer: ");
for await (const [token, metadata] of await agent.stream(
{ messages: [{ role: "user", content: "Tell me about embeddings" }] },
{ ...config, streamMode: "messages" }
)) {
const content = token.content;
// same check as example1 — just print string tokens
if (typeof content === "string" && content.length > 0) {
process.stdout.write(content);
}
}
// After Turn 1, MemorySaver saves:
// thread state for threadId = {
// messages: [
// HumanMessage { content: "Tell me about embeddings" },
// AIMessage { content: "Embeddings convert text into vectors..." }
// ]
// }
console.log("\n");
// ── Turn 2 ────────────────────────────────────────────────
console.log("Turn 2 — Question: Can you give me an example of what you just explained?");
process.stdout.write("Answer: ");
for await (const [token, metadata] of await agent.stream(
{ messages: [{ role: "user", content: "Can you give me an example of what you just explained?" }] },
{ ...config, streamMode: "messages" }
// SAME threadId → MemorySaver loads Turn 1 messages
// Agent now has context of both turns
// "what you just explained" → agent knows this refers to embeddings
)) {
const content = token.content;
if (typeof content === "string" && content.length > 0) {
process.stdout.write(content);
}
}
console.log("\n");
}
// ─────────────────────────────────────────
// EXAMPLE 3 — Multiple stream modes at once
// See tool calls happening AND tokens streaming
// ─────────────────────────────────────────
async function example3() {
console.log("─".repeat(55));
console.log("EXAMPLE 3: Streaming agent progress (updates + messages)");
console.log("─".repeat(55));
const threadId = crypto.randomUUID();
const config = { configurable: { thread_id: threadId } };
console.log("\nQuestion: What is LangChain?\n");
for await (const [mode, chunk] of await agent.stream(
{ messages: [{ role: "user", content: "What is LangChain?" }] },
{
...config,
streamMode: ["updates", "messages"],
// passing an ARRAY = stream MULTIPLE modes at once
// each loop iteration yields [mode, chunk] tuple
//
// mode = "updates" → chunk = what changed after this agent step
// mode = "messages" → chunk = [token, metadata] tuple (LLM token)
}
)) {
if (mode === "updates") {
// "updates" fires once after each agent step completes
// chunk = what the agent state changed to after this step
// example chunk when tool ran: { tools: { messages: [ToolMessage {...}] } }
// example chunk when LLM ran: { agent: { messages: [AIMessage {...}] } }
if (chunk.tools) {
// chunk.tools exists → a tool was just called and returned
console.log("🔧 Tool called and returned a result");
// In a real UI you might show: "Searching document..." here
}
}
if (mode === "messages") {
// "messages" fires for each individual LLM token
// chunk = [token, metadata] when mode is "messages"
const [token, metadata] = chunk;
// destructure chunk into token and metadata
// token = AIMessageChunk with .content
// metadata = { langgraph_node: "???" }
const content = token.content;
if (typeof content === "string" && content.length > 0) {
// same fix: just check if content is a non-empty string
process.stdout.write(content);
// print each token as it arrives
}
}
}
console.log("\n");
}
// ─────────────────────────────────────────
// MAIN
// ─────────────────────────────────────────
async function main() {
console.log("\n🌊 STREAMING DEMO\n");
await example1();
await example2();
await example3();
console.log("✅ Streaming demo complete!");
}
main().catch(console.error);
Run:
node src/streaming.js
Part 2 — Error Handling
Why Error Handling Matters in Production
Things that WILL fail in production:
OpenAI API → rate limits, timeouts, server errors
PDF loading → file not found, corrupted PDF, too large
Embedding → token limit exceeded, API down
Vector search → empty store, corrupted index
User input → empty question, malicious input
Memory → session not found, storage full
Without proper error handling — one failure crashes everything. With it — the app degrades gracefully and tells the user what happened.
Create src/errorHandling.js:
// ───────────────────────────────────────── // IMPORTS // ─────────────────────────────────────────
import { ChatOpenAI } from "@langchain/openai"; import { ChatPromptTemplate } from "@langchain/core/prompts"; import { StringOutputParser } from "@langchain/core/output_parsers"; import * as dotenv from "dotenv"; dotenv.config();
// ───────────────────────────────────────── // CUSTOM ERROR CLASSES // Makes it easy to identify what type of error occurred // and handle each type differently // ─────────────────────────────────────────
class RAGError extends Error { // Base class for all RAG-specific errors // Extends built-in Error class constructor(message, code, details = {}) { super(message); // super() = call Error constructor with message
this.name = "RAGError"; // name = type of error (shows in stack traces)
this.code = code; // code = machine-readable error identifier // example: "PDF_NOT_FOUND", "API_RATE_LIMIT", "EMPTY_QUERY"
this.details = details; // details = any extra info about what went wrong // example: { filePath: "./missing.pdf", userId: "user_123" } } }
class PDFLoadError extends RAGError { constructor(filePath, originalError) { super(`Failed to load PDF: ${filePath}`, "PDF_LOAD_ERROR", { filePath, originalError: originalError.message, }); this.name = "PDFLoadError"; } }
class EmbeddingError extends RAGError { constructor(chunkCount, originalError) { super(`Failed to embed ${chunkCount} chunks`, "EMBEDDING_ERROR", { chunkCount, originalError: originalError.message, }); this.name = "EmbeddingError"; } }
class EmptyQueryError extends RAGError { constructor() { super("Question cannot be empty", "EMPTY_QUERY", {}); this.name = "EmptyQueryError"; } }
class APIRateLimitError extends RAGError { constructor(retryAfter) { super("OpenAI API rate limit hit — please wait", "API_RATE_LIMIT", { retryAfter }); this.name = "APIRateLimitError"; } }
// ───────────────────────────────────────── // RETRY HELPER // Automatically retries failed operations // Essential for OpenAI API calls that sometimes fail // ─────────────────────────────────────────
async function withRetry(fn, options = {}) { const { maxRetries = 3, // maximum number of retry attempts // default: try 3 times before giving up
initialDelay = 1000, // milliseconds to wait before first retry // default: 1 second
backoffFactor = 2, // multiply delay by this factor each retry // default: 2 (exponential backoff) // attempt 1: wait 1000ms // attempt 2: wait 2000ms // attempt 3: wait 4000ms
shouldRetry = (error) => true, // function that decides if this error is retryable // default: retry everything // you can customize: don't retry auth errors } = options;
let lastError; // stores the error from the most recent failed attempt
for (let attempt = 1; attempt <= maxRetries; attempt++) { // attempt = current try number (1, 2, 3, ...)
try { return await fn(); // try calling the function // if it succeeds → return immediately // if it throws → caught by catch block below
} catch (error) { lastError = error; // save this error in case we run out of retries
const isLastAttempt = attempt === maxRetries; // true if this was the final allowed attempt
const isRetryable = shouldRetry(error); // check if this type of error can be retried
if (isLastAttempt || !isRetryable) { throw error; // don't retry — throw the error to the caller }
const delay = initialDelay * Math.pow(backoffFactor, attempt - 1); // calculate how long to wait before next attempt // attempt 1: 1000 * 2^0 = 1000ms // attempt 2: 1000 * 2^1 = 2000ms // attempt 3: 1000 * 2^2 = 4000ms
console.log(` Attempt ${attempt} failed. Retrying in ${delay}ms... (${error.message})`);
await new Promise(resolve => setTimeout(resolve, delay)); // wait before retrying // setTimeout wrapped in Promise so we can await it } }
throw lastError; // shouldn't reach here but TypeScript needs this }
// ───────────────────────────────────────── // INPUT VALIDATION // Validate user input BEFORE sending to API // Saves money and prevents confusing errors // ─────────────────────────────────────────
function validateQuestion(question) { // question = user's raw input string
if (!question || typeof question !== "string") { throw new EmptyQueryError(); // null, undefined, or non-string }
const trimmed = question.trim(); // remove leading/trailing whitespace // " hello " → "hello"
if (trimmed.length === 0) { throw new EmptyQueryError(); // user just pressed Enter with no text }
if (trimmed.length < 3) { throw new RAGError( "Question too short — please ask a complete question", "QUERY_TOO_SHORT", { length: trimmed.length } ); // "hi" or "ok" — not useful questions }
if (trimmed.length > 2000) { throw new RAGError( "Question too long — please keep it under 2000 characters", "QUERY_TOO_LONG", { length: trimmed.length } ); // prevents prompt injection and excessive token usage }
return trimmed; // return cleaned question if all validation passes }
// ───────────────────────────────────────── // SAFE LLM CALL // Wraps an LLM call with retry + error classification // ─────────────────────────────────────────
async function safeLLMCall(chain, input) { return withRetry( async () => { try { return await chain.invoke(input); // attempt the LLM call
} catch (error) { // Classify the error so we handle it correctly
if (error.status === 429) { // HTTP 429 = Too Many Requests = rate limit hit const retryAfter = error.headers?.["retry-after"] || 60; // retry-after header tells us when we can try again throw new APIRateLimitError(retryAfter); }
if (error.status === 401) { // HTTP 401 = Unauthorized = bad API key throw new RAGError( "Invalid OpenAI API key — check your .env file", "INVALID_API_KEY", {} ); // don't retry auth errors — they won't fix themselves }
if (error.status === 500 || error.status === 503) { // HTTP 500/503 = OpenAI server error throw new RAGError( "OpenAI API server error — will retry", "API_SERVER_ERROR", { status: error.status } ); // these ARE retryable — server might recover }
if (error.code === "ECONNRESET" || error.code === "ETIMEDOUT") { // network errors — connection dropped or timed out throw new RAGError( "Network error connecting to OpenAI — will retry", "NETWORK_ERROR", { code: error.code } ); }
throw error; // unknown error — rethrow as-is } }, { maxRetries: 3, initialDelay: 1000, shouldRetry: (error) => { // only retry certain types of errors const retryableCodes = [ "API_SERVER_ERROR", "NETWORK_ERROR", "API_RATE_LIMIT", ]; return retryableCodes.includes(error.code); // auth errors, validation errors → don't retry // server errors, network errors → do retry }, } ); }
// ───────────────────────────────────────── // ERROR RESPONSE FORMATTER // Converts errors into user-friendly messages // Users should never see raw error stack traces // ─────────────────────────────────────────
function formatErrorResponse(error) {
if (error instanceof EmptyQueryError) { return { success: false, error: "Please type a question before submitting.", code: error.code, }; }
// ADD THIS — was missing: if (error.code === "QUERY_TOO_SHORT") { return { success: false, error: "Question too short — please ask a complete question.", code: error.code, }; }
if (error.code === "QUERY_TOO_LONG") { return { success: false, error: "Question too long — please keep it under 2000 characters.", code: error.code, }; }
if (error instanceof PDFLoadError) { return { success: false, error: "Could not read the document. Please check the file and try again.", code: error.code, }; }
if (error instanceof APIRateLimitError) { return { success: false, error: `Too many requests. Please wait ${error.details.retryAfter} seconds and try again.`, code: error.code, retryAfter: error.details.retryAfter, }; }
if (error.code === "INVALID_API_KEY") { return { success: false, error: "Configuration error. Please contact support.", code: error.code, }; }
console.error("Unexpected error:", error); return { success: false, error: "Something went wrong. Please try again in a moment.", code: "UNKNOWN_ERROR", }; }
// ───────────────────────────────────────── // DEMO — Show error handling in action // ─────────────────────────────────────────
async function demo() { console.log("\n🛡️ ERROR HANDLING DEMO\n");
const llm = new ChatOpenAI({ model: "gpt-4o", temperature: 0 }); const prompt = ChatPromptTemplate.fromMessages([ ["human", "{question}"] ]); const chain = prompt.pipe(llm).pipe(new StringOutputParser());
// Test 1 — Empty question console.log("─".repeat(50)); console.log("Test 1: Empty question"); try { const validated = validateQuestion(""); const result = await safeLLMCall(chain, { question: validated }); console.log("Result:", result); } catch (error) { const response = formatErrorResponse(error); console.log("Error caught:", response); }
// Test 2 — Valid question console.log("\n─".repeat(50)); console.log("Test 2: Valid question"); try { const validated = validateQuestion("What is 2 + 2?"); const result = await safeLLMCall(chain, { question: validated }); console.log("Result:", result); } catch (error) { const response = formatErrorResponse(error); console.log("Error:", response); }
// Test 3 — Very short question console.log("\n─".repeat(50)); console.log("Test 3: Too short question"); try { const validated = validateQuestion("hi"); console.log("Validated:", validated); } catch (error) { const response = formatErrorResponse(error); console.log("Error caught:", response); } }
if (process.argv[1].includes("errorHandling")) { // only run demo when this file is run directly // node src/errorHandling.js → runs demo // imported by api.js → does NOT run demo demo().catch(console.error); }
export { validateQuestion, safeLLMCall, formatErrorResponse, withRetry, RAGError, PDFLoadError };
Run:
node src/errorHandling.js
Part 3 — Express API
Now wrap everything in an Express HTTP server so your Next.js frontend can call it.
Create src/api.js:
import express from "express"; // express = Node.js web framework for building HTTP servers // provides methods like app.get(), app.post() to define API routes // when a request comes in → express routes it to the correct handler function
import cors from "cors"; // cors = Cross-Origin Resource Sharing middleware // browsers block requests between different origins by default // example: Next.js on localhost:3000 calling Express on localhost:3001 → blocked without cors // cors middleware adds the right headers so browser allows the request
import { createAgent, tool } from "langchain"; // createAgent = builds a complete LangGraph agent (LLM + tools + memory + system prompt) // tool = wraps any JavaScript function into a tool the agent can call
import { MemorySaver } from "@langchain/langgraph"; // MemorySaver = stores agent conversation state as checkpoints in memory // each unique thread_id gets its own saved state // when same thread_id is used again → agent loads that state and remembers previous messages
import { ChatOpenAI } from "@langchain/openai"; // ChatOpenAI = LangChain wrapper around OpenAI's chat completion API // internally calls: POST https://api.openai.com/v1/chat/completions
import { OpenAIEmbeddings } from "@langchain/openai"; // OpenAIEmbeddings = converts text strings into vectors (arrays of numbers) // uses OpenAI's text-embedding-3-small model // internally calls: POST https://api.openai.com/v1/embeddings
import { MemoryVectorStore } from "@langchain/classic/vectorstores/memory"; // MemoryVectorStore = stores text chunks + their embedding vectors in a JavaScript array // no external database needed — everything lives in RAM // provides .similaritySearch() to find the most relevant chunks for a given query
import { PDFLoader } from "@langchain/community/document_loaders/fs/pdf"; // PDFLoader = reads a PDF file from disk and extracts its text content // returns an array of Document objects — one Document per page // each Document has: { pageContent: "text...", metadata: { source, page } }
import { RecursiveCharacterTextSplitter } from "@langchain/textsplitters"; // RecursiveCharacterTextSplitter = splits large Document text into smaller chunks // tries these separators in order: "\n\n" → "\n" → ". " → " " → "" // this means it prefers clean paragraph splits over mid-sentence splits // preserves metadata from the original Document in every chunk it creates
import { z } from "zod"; // zod = schema validation library // used to define exactly what inputs each tool accepts // example: z.string() means the input must be a string // if LLM passes wrong type → zod throws an error before the tool runs
import { validateQuestion, formatErrorResponse } from "./errorHandling.js"; // validateQuestion = checks user input before sending to AI // throws custom errors if question is empty / too short / too long // formatErrorResponse = converts any error into a clean JSON object for the frontend // example: { success: false, error: "Please type a question", code: "EMPTY_QUERY" }
import * as dotenv from "dotenv"; dotenv.config(); // dotenv.config() reads the .env file and loads its values into process.env // after this line: process.env.OPENAI_API_KEY = "sk-proj-..." // LangChain and OpenAI clients automatically read from process.env
// ───────────────────────────────────────── // EXPRESS APP SETUP // ─────────────────────────────────────────
const app = express(); // app = the main Express application object // all routes and middleware are registered on this object
const PORT = process.env.PORT || 3001; // PORT = which TCP port to listen on // process.env.PORT = set this in production (e.g. Render, Railway auto-sets it) // fallback: 3001 so it doesn't conflict with Next.js running on 3000 // example value: 3001
app.use(cors({ origin: ["http://localhost:3000", "http://localhost:3001"], // origin = list of allowed frontend URLs that can call this API // browser checks this — if request comes from a different origin → blocked // in production replace with: "https://yourdomain.com" })); // cors() returns a middleware function // app.use() = register that middleware to run on every incoming request // it adds this header to every response: Access-Control-Allow-Origin: http://localhost:3000
app.use(express.json()); // express.json() = middleware that parses the request body as JSON // without this: req.body = undefined // with this: req.body = { question: "what is this doc about?", sessionId: "session_123" }
// ───────────────────────────────────────── // GLOBAL STATE // These variables are shared across all HTTP requests // They live in memory as long as the server is running // ─────────────────────────────────────────
let vectorStore = null; // vectorStore = the MemoryVectorStore holding all embedded PDF chunks // null = no PDF has been loaded yet // after /api/load-pdf succeeds: // vectorStore = MemoryVectorStore { // memoryVectors: [ // { content: "chunk text...", embedding: [0.023, -0.089, ...], metadata: { page: 0 } }, // { content: "chunk text...", embedding: [0.071, 0.042, ...], metadata: { page: 1 } }, // ... // ] // }
let isIndexing = false; // isIndexing = flag to prevent two PDF loads from running at the same time // false = server is idle, ready to accept a load request // true = server is currently loading and embedding a PDF // if a second /api/load-pdf request arrives while isIndexing is true → returns 409 Conflict
// ───────────────────────────────────────── // AGENT FACTORY FUNCTION // Called once after a PDF is loaded // Returns a ready-to-use RAG agent // ─────────────────────────────────────────
function createRAGAgent(retriever) { // retriever = vectorStore.asRetriever({ k: 5 }) // calling retriever.invoke("some query") returns the 5 most similar Document chunks // example return value of retriever.invoke("pension benefits"): // [ // Document { pageContent: "subscribers get pension of 1000...", metadata: { page: 0 } }, // Document { pageContent: "pension continues to spouse...", metadata: { page: 1 } }, // ...3 more // ]
// ── TOOL DEFINITION ────────────────────────────────────────────────────── // A tool = a JavaScript function the LLM can decide to call // The agent reads the tool's name + description to decide when to use it // The schema defines what inputs the LLM must provide when calling it
const searchTool = tool( async ({ query }) => { // This function body runs when the LLM calls this tool // query = the search string the LLM decides to search for // example: query = "pension benefits for subscriber" // query = "how to file a complaint" // query = "employee responsibilities"
const docs = await retriever.invoke(query); // retriever.invoke(query) does three things internally: // 1. calls OpenAI embeddings API to convert query → vector // 2. computes cosine similarity between query vector and all stored chunk vectors // 3. returns top 5 most similar Document chunks // // example docs value: // [ // Document { pageContent: "Subscriber gets Rs.1000-5000 pension...", metadata: { page: 0 } }, // Document { pageContent: "Pension continues to spouse after...", metadata: { page: 1 } }, // Document { pageContent: "Nomination details must be provided...", metadata: { page: 0 } }, // ... // ]
if (docs.length === 0) { return "No relevant information found in the document."; // returned string goes back to the LLM as the tool result // LLM then uses this to tell the user nothing was found }
return docs .map((doc, i) => `Source ${i + 1} (Page ${(doc.metadata.page || 0) + 1}):\n${doc.pageContent.trim()}` ) // .map() loops through each Document and formats it as a labeled source // i = index (0, 1, 2, 3, 4) // doc.metadata.page = 0-indexed page number from PDF → add 1 for human-readable // doc.pageContent.trim() = actual text with whitespace removed from edges // // example single formatted source: // "Source 1 (Page 1):\nSubscriber gets Rs.1000-5000 guaranteed pension..." .join("\n\n"); // join all 5 formatted sources into one string separated by blank lines // // example final return value (what LLM receives as tool result): // "Source 1 (Page 1):\nSubscriber gets Rs.1000-5000...\n\n // Source 2 (Page 2):\nPension continues to spouse...\n\n // ..." },
{ name: "search_document", // name = identifier the LLM uses when it decides to call this tool // example LLM output when calling tool: { "tool": "search_document", "args": { "query": "..." } }
description: `Search the indexed document for information relevant to a query. ALWAYS use this tool before answering any question about the document. The tool returns relevant text sections — READ them and write your answer in your own words. Do not copy-paste the sections — summarize and explain.`, // description = what the LLM reads to decide WHEN to call this tool // clear, specific description = LLM calls it at the right time // "ALWAYS use this tool" = prevents LLM from answering from its own knowledge
schema: z.object({ query: z.string().describe("search query to find relevant information"), // z.string() = the query input must be a text string // .describe() = tells LLM what this field is for // LLM must provide a value for this field when calling the tool // example: { query: "what are the pension benefits?" } }), } ); // searchTool = a LangChain Tool object // it wraps the function above with its name, description, and schema // the agent holds a reference to this and can call it during reasoning
// ── MEMORY SETUP ─────────────────────────────────────────────────────────
const checkpointer = new MemorySaver(); // MemorySaver = in-memory checkpoint store // internally it's a Map: thread_id → serialized agent state // example internal state after two conversation turns: // { // "session_123": { // messages: [ // HumanMessage { content: "what is this doc about?" }, // AIMessage { content: "This document is about..." }, // HumanMessage { content: "what are the benefits?" }, // AIMessage { content: "The benefits include..." } // ] // } // } // when same thread_id is used next time → agent loads this state and has full context
// ── CREATE AND RETURN THE AGENT ───────────────────────────────────────────
return createAgent({ model: new ChatOpenAI({ model: "gpt-4o", // "gpt-4o" = GPT-4 Omni — supports tool calling, fast, high quality temperature: 0.1, // temperature = how deterministic the output is // 0.0 = always picks highest probability token → same answer every time // 0.1 = nearly deterministic with very slight variation → good for factual Q&A // 1.0 = highly random → good for creative writing, bad for document Q&A }),
tools: [searchTool], // array of tools this agent is allowed to call // agent reads each tool's name + description to know what's available // during reasoning, agent decides which tool to call based on the question
checkpointer, // connects the MemorySaver to this agent // every time agent.invoke() or agent.stream() runs: // BEFORE: loads conversation history for this thread_id from checkpointer // AFTER: saves updated conversation history back to checkpointer
systemPrompt: `You are a helpful document assistant.
STRICT RULES: 1. ALWAYS call the search_document tool first to find relevant information 2. After getting tool results — write a clear, natural answer IN YOUR OWN WORDS 3. Do NOT output the raw source text — only write your own summary 4. Write like ChatGPT — conversational, clear, well-structured 5. At the very end mention sources used: (Source 1, Source 2, etc.) 6. If document doesn't contain the answer — say so clearly 7. Keep answers concise — 3 to 5 sentences maximum for simple questions
IMPORTANT: Your response should ONLY be your written answer. Never output the raw retrieved text chunks. That is internal data only.`, // systemPrompt = hidden instruction sent to the LLM before every conversation turn // the user never sees this — it shapes the agent's behavior and output style // Rule 1 ensures agent always retrieves before answering (prevents hallucination) // Rule 3 prevents raw chunk copy-paste (happened before this was added) // Rule 7 keeps answers concise (avoids overwhelming the user) }); // return value = a compiled LangGraph agent object // it has two main methods: // .invoke(input, config) = run agent, wait for full response, return it // .stream(input, config) = run agent, yield tokens as they generate }
let ragAgent = null; // ragAgent = the agent created by createRAGAgent() // null = no PDF loaded yet, cannot answer questions // after /api/load-pdf succeeds: // ragAgent = CompiledStateGraph { ... } (LangGraph agent object) // reused for all subsequent /api/chat and /api/chat/stream requests
// ───────────────────────────────────────── // ROUTE 1 — Health Check // GET /api/health // Frontend calls this to check if server is running and PDF is loaded // ─────────────────────────────────────────
app.get("/api/health", (req, res) => { res.json({ status: "ok", // always "ok" if the server is running and responded // if server is down this endpoint won't respond at all
documentLoaded: vectorStore !== null, // true = PDF has been loaded and vector store is ready // false = no PDF yet, user needs to call /api/load-pdf first // frontend uses this to enable/disable the chat input
isIndexing, // true = PDF is currently being processed (loading + embedding) // false = idle // frontend can poll this endpoint and show a loading spinner while true
timestamp: new Date().toISOString(), // current server time in ISO format // example: "2025-07-16T14:32:00.000Z" // useful for debugging: confirms server responded at this exact time }); // example full response body: // { "status": "ok", "documentLoaded": true, "isIndexing": false, "timestamp": "2025-07-16T..." } });
// ───────────────────────────────────────── // ROUTE 2 — Load PDF // POST /api/load-pdf // Body: { pdfPath: "./sample.pdf" } // Loads, chunks, embeds, and indexes the PDF // Must be called before any chat requests // ─────────────────────────────────────────
app.post("/api/load-pdf", async (req, res) => {
const { pdfPath } = req.body; // pdfPath = file path string from the request body // example req.body: { "pdfPath": "./sample.pdf" } // example pdfPath value: "./sample.pdf"
if (!pdfPath) { return res.status(400).json({ success: false, error: "pdfPath is required in request body", }); // 400 = HTTP Bad Request — client sent incomplete data // return stops the function — nothing below this runs }
if (isIndexing) { return res.status(409).json({ success: false, error: "Already indexing a document. Please wait.", }); // 409 = HTTP Conflict — another indexing operation is already in progress // prevents race condition: two users loading PDFs simultaneously }
isIndexing = true; // set flag to true BEFORE starting the async work // any request that arrives now will see isIndexing = true and get 409
try { console.log(`\n📄 Loading PDF: ${pdfPath}`);
const loader = new PDFLoader(pdfPath, { splitPages: true }); // PDFLoader opens the file at pdfPath // splitPages: true = each page becomes a separate Document object // splitPages: false = entire PDF is one Document (harder to chunk well)
const docs = await loader.load(); // reads the PDF file from disk and extracts text from each page // example docs value for a 3-page PDF: // [ // Document { pageContent: "page 1 text content...", metadata: { source: "./sample.pdf", page: 0 } }, // Document { pageContent: "page 2 text content...", metadata: { source: "./sample.pdf", page: 1 } }, // Document { pageContent: "page 3 text content...", metadata: { source: "./sample.pdf", page: 2 } }, // ]
const splitter = new RecursiveCharacterTextSplitter({ chunkSize: 800, // target size of each chunk in characters // 800 chars ≈ 150-200 words ≈ 1-2 paragraphs // smaller = more precise retrieval, more chunks, more embedding API calls // larger = less precise, fewer chunks, fewer API calls
chunkOverlap: 150, // number of characters repeated between consecutive chunks // prevents losing context at chunk boundaries // example: end of chunk 1 = start of chunk 2 (150 characters overlap) });
const chunks = await splitter.splitDocuments(docs); // takes the full-page Documents and splits each into smaller chunks // automatically copies metadata from parent Document to each chunk // example chunks for one page (before full list): // [ // Document { pageContent: "first 800 chars of page 1...", metadata: { source: "./sample.pdf", page: 0 } }, // Document { pageContent: "overlap + next 800 chars...", metadata: { source: "./sample.pdf", page: 0 } }, // Document { pageContent: "start of page 2...", metadata: { source: "./sample.pdf", page: 1 } }, // ... // ]
vectorStore = await MemoryVectorStore.fromDocuments( chunks, // array of Document chunks to embed and store
new OpenAIEmbeddings({ model: "text-embedding-3-small" }) // embedding model: converts each chunk's text into a 1536-dimensional vector // internally: calls OpenAI embeddings API once per chunk (or batched) ); // fromDocuments() does three things: // 1. extracts .pageContent from each chunk // 2. calls OpenAI API to get a vector (1536 numbers) for each chunk // 3. stores { content, embedding, metadata } for every chunk in memory // // example vectorStore internal state after 25 chunks: // MemoryVectorStore { // memoryVectors: [ // { content: "APY khata kholte samay...", embedding: [0.023, -0.089, ...1536 numbers] }, // { content: "bachat bank khate dwara...", embedding: [0.071, 0.042, ...1536 numbers] }, // ... 23 more // ] // }
const retriever = vectorStore.asRetriever({ k: 5 }); // .asRetriever() wraps the vector store in a Retriever interface // k: 5 = return the 5 most similar chunks per search query // retriever.invoke("some query") → top 5 Documents sorted by similarity
ragAgent = createRAGAgent(retriever); // creates the complete RAG agent with this retriever as its search tool // ragAgent is now ready to answer questions about this PDF
console.log(`✅ Indexed ${chunks.length} chunks from ${docs.length} pages`); // example log: "✅ Indexed 25 chunks from 7 pages"
res.json({ success: true, message: "PDF loaded and indexed successfully", stats: { pages: docs.length, // total pages in the PDF // example: 7
chunks: chunks.length, // total chunks created after splitting // example: 25
fileName: pdfPath.split("/").pop(), // extract just the filename from the full path // example: "./sample.pdf".split("/").pop() = "sample.pdf" }, }); // example full response body: // { "success": true, "message": "PDF loaded...", "stats": { "pages": 7, "chunks": 25, "fileName": "sample.pdf" } }
} catch (error) { console.error("PDF load error:", error); vectorStore = null; ragAgent = null; // reset both to null so server is in a clean state // next /api/load-pdf request can try again
const response = formatErrorResponse(error); res.status(500).json(response); // 500 = HTTP Internal Server Error // example response: { "success": false, "error": "Something went wrong...", "code": "UNKNOWN_ERROR" }
} finally { isIndexing = false; // always reset this flag — whether load succeeded or failed // without this: server gets permanently stuck if an error occurs mid-indexing } });
// ───────────────────────────────────────── // ROUTE 3 — Chat (Non-Streaming) // POST /api/chat // Body: { question: "...", sessionId: "..." } // Waits for complete answer then returns it all at once // Use this for simple integrations that don't need streaming // ─────────────────────────────────────────
app.post("/api/chat", async (req, res) => {
const { question, sessionId } = req.body; // question = the text string the user typed // example: "what are the pension benefits?" // sessionId = unique conversation identifier from the frontend // example: "session_1752672000000" // used to load/save conversation history in MemorySaver
if (!ragAgent) { return res.status(400).json({ success: false, error: "No document loaded. Call /api/load-pdf first.", code: "NO_DOCUMENT", }); // 400 = Bad Request — user tried to chat before loading a PDF }
let validatedQuestion; try { validatedQuestion = validateQuestion(question); // validateQuestion checks: // null / undefined / non-string → throws EmptyQueryError // empty string "" → throws EmptyQueryError // shorter than 3 characters → throws RAGError (QUERY_TOO_SHORT) // longer than 2000 characters → throws RAGError (QUERY_TOO_LONG) // all checks pass → returns trimmed string // // example: validateQuestion(" what is APY? ") = "what is APY?" } catch (error) { return res.status(400).json(formatErrorResponse(error)); // example: { "success": false, "error": "Please type a question...", "code": "EMPTY_QUERY" } }
try { const config = { configurable: { thread_id: sessionId || "default", // thread_id = which conversation history to load from MemorySaver // example: "session_1752672000000" // if no sessionId → "default" (all users share one history — avoid in production) }, }; // example config value: // { configurable: { thread_id: "session_1752672000000" } }
const result = await ragAgent.invoke( { messages: [{ role: "user", content: validatedQuestion }] }, // input to the agent: // messages = array of message objects in OpenAI chat format // role: "user" = this message is from the human // content = the actual question text // example: { messages: [{ role: "user", content: "what are the pension benefits?" }] }
config // tells the agent which thread_id to use for memory ); // ragAgent.invoke() runs the full agent loop synchronously: // 1. loads conversation history from MemorySaver for this thread_id // 2. LLM reads system prompt + history + current question // 3. LLM decides to call search_document tool // 4. tool searches vector store, returns relevant chunks // 5. LLM reads chunks and writes a grounded answer // 6. saves updated conversation (all messages) back to MemorySaver // 7. returns { messages: [...all messages in this run...] } // // example result value: // { // messages: [ // HumanMessage { content: "what are the pension benefits?" }, // AIMessage { content: "", tool_calls: [{ name: "search_document", args: { query: "pension benefits" } }] }, // ToolMessage { content: "Source 1 (Page 1):\nSubscriber gets pension..." }, // AIMessage { content: "The APY scheme provides a guaranteed pension of Rs.1000 to Rs.5000..." } // ] // }
const lastMessage = result.messages[result.messages.length - 1]; // result.messages = array of all messages produced in this agent run // result.messages.length - 1 = index of the last message // last message = the LLM's final text response to the user // example lastMessage value: // AIMessage { content: "The APY scheme provides a guaranteed monthly pension..." }
res.json({ success: true, answer: lastMessage.content, // lastMessage.content = the actual answer text string // example: "The APY scheme provides a guaranteed monthly pension of Rs.1000..."
sessionId: sessionId || "default", // echo back the sessionId so frontend can store it for future requests }); // example full response body: // { // "success": true, // "answer": "The APY scheme provides a guaranteed monthly pension...", // "sessionId": "session_1752672000000" // }
} catch (error) { console.error("Chat error:", error); const response = formatErrorResponse(error); res.status(500).json(response); } });
// ───────────────────────────────────────── // ROUTE 4 — Chat with Streaming // POST /api/chat/stream // Body: { question: "...", sessionId: "..." } // Sends response tokens one by one using Server-Sent Events (SSE) // // HOW SSE WORKS: // Normal HTTP: client requests → server sends one response → connection closes // SSE: client requests → server keeps connection open // → server writes small data chunks one at a time // → client receives each chunk immediately as it arrives // → this creates the word-by-word streaming effect like ChatGPT // ─────────────────────────────────────────
app.post("/api/chat/stream", async (req, res) => {
const { question, sessionId } = req.body; // question = user's question string // example: "what is the document about?" // sessionId = unique conversation ID from the frontend // example: "session_1752672000000" // generated in useChat.js as: `session_${Date.now()}`
if (!ragAgent) { return res.status(400).json({ success: false, error: "No document loaded. Call /api/load-pdf first.", }); }
let validatedQuestion; try { validatedQuestion = validateQuestion(question); // same validation as /api/chat route above // example output: "what is the document about?" } catch (error) { return res.status(400).json(formatErrorResponse(error)); }
// ── SET SSE RESPONSE HEADERS ────────────────────────────────────────────── // These headers tell the browser: "this is a streaming connection, keep it open"
res.setHeader("Content-Type", "text/event-stream"); // "text/event-stream" = the MIME type that tells browser this is an SSE connection // browser will NOT close the connection after receiving data // browser will parse incoming data as SSE events
res.setHeader("Cache-Control", "no-cache"); // tells browser and proxies: do not cache this response // streaming responses must always be fresh — caching would break them
res.setHeader("Connection", "keep-alive"); // HTTP/1.1 normally closes connection after each response // keep-alive = maintain the TCP connection so we can keep writing data
res.setHeader("Access-Control-Allow-Origin", "*"); // CORS header for SSE specifically // allows frontend at any origin to read the streaming response
// ── SSE EVENT HELPER ──────────────────────────────────────────────────────
function sendEvent(eventType, data) { // eventType = label for what kind of event this is // examples: "start", "token", "done", "error" // // data = JavaScript object containing the event payload // examples: // { message: "Starting response..." } // { content: "This" } // { content: " document" } // { message: "Response complete" } // { success: false, error: "Something went wrong" }
res.write(`event: ${eventType}\n`); // res.write() sends data to the client WITHOUT closing the connection // (unlike res.send() / res.json() which close the connection) // // SSE protocol: the "event:" line sets the event type name // browser's EventSource API reads this and knows which handler to call // example written to stream: "event: token\n"
res.write(`data: ${JSON.stringify(data)}\n\n`); // JSON.stringify(data) converts JavaScript object → JSON string // example: JSON.stringify({ content: "This" }) = '{"content":"This"}' // // SSE protocol: the "data:" line contains the event payload // \n\n (double newline) = signals end of this complete SSE event // // example complete SSE event written to stream: // "event: token\n" // 'data: {"content":"This"}\n\n' // // browser receives these two lines and fires: { data: '{"content":"This"}' } // frontend parses that and appends "This" to the displayed message }
try { const config = { configurable: { thread_id: sessionId || "default" }, }; // same as /api/chat — tells agent which conversation history to use // example: { configurable: { thread_id: "session_1752672000000" } }
sendEvent("start", { message: "Starting response..." }); // first SSE event — tells frontend the agent has started processing // frontend can show a "thinking..." indicator // written to stream: // "event: start\n" // 'data: {"message":"Starting response..."}\n\n'
console.log("\n🔍 Starting stream for:", validatedQuestion);
for await (const [token, metadata] of await ragAgent.stream( { messages: [{ role: "user", content: validatedQuestion }] }, // same input format as ragAgent.invoke()
{ ...config, streamMode: "messages" } // streamMode: "messages" = yield one [token, metadata] tuple per LLM token // instead of waiting for the full response, yields as tokens generate // // each yielded item is a tuple: // token = AIMessageChunk — one small piece of the LLM output // metadata = { langgraph_node: "???" } — which internal node generated this )) { // token = AIMessageChunk object // possible token.content values per iteration: // "This" → string (text token — send to frontend) // " document" → string (text token — send to frontend) // "" → empty string (skip) // [{ type: "tool_use", ... }] → array (tool call — skip) // "Source 1 (Page 1):\ntext..." → string BUT from tool result — skip via getType() // // metadata example: { langgraph_node: "agent", langgraph_step: 3 }
console.log( "node:", metadata?.langgraph_node, "| type:", typeof token?.content, "| value:", JSON.stringify(token?.content)?.substring(0, 80) ); // debug log — remove in production // example output during agent run: // node: tools | type: string | value: "Source 1 (Page 1):\nSubscriber gets..." // node: agent | type: string | value: "The" // node: agent | type: string | value: " APY" // node: agent | type: string | value: " scheme"
const content = token?.content; // safe access with ?. → returns undefined if token is null // content = the text payload of this chunk
if (token?.getType && token.getType() === "tool") continue; // token.getType() = returns the message type: "human", "ai", "tool", "system" // "tool" = this is a ToolMessage (the result returned by search_document tool) // we do NOT want to send tool results to the frontend — they are raw chunks // skip this iteration and move to the next token
if (Array.isArray(content)) { // Array content = LLM is making a tool call // example: [{ type: "tool_use", name: "search_document", input: { query: "..." } }] // this is an internal action, not text for the user — skip it continue; }
if (typeof content === "string" && content.length > 0) { // typeof content === "string" → it's a text token (not a tool call array) // content.length > 0 → it has actual characters (not empty "") // // example content values that pass this check: // "The" → send ✓ " APY" → send ✓ " scheme" → send ✓ // // example values that fail and are skipped: // "" → length is 0 ✗ [] → Array, not string ✗
sendEvent("token", { content }); // sends this text token to the frontend immediately // written to HTTP stream: // "event: token\n" // 'data: {"content":"The"}\n\n' // // frontend's useChat.js receives this event // appends content to the current message being displayed // user sees "The" appear on screen, then " APY", then " scheme"... } } // for-await loop ends when the agent finishes generating its response // all tokens have been sent to the frontend at this point
sendEvent("done", { message: "Response complete" }); // final SSE event — tells frontend the full response has been sent // written to stream: // "event: done\n" // 'data: {"message":"Response complete"}\n\n' // // frontend receives this and: // sets isStreaming = false // removes the blinking cursor from the message // re-enables the input box
res.end(); // closes the HTTP connection // without res.end() the browser keeps the connection open waiting for more events // res.end() signals: "this SSE stream is finished, you can close the connection"
} catch (error) { console.error("Stream error:", error); // log full error details on the server for debugging
const errorResponse = formatErrorResponse(error); // converts error into user-friendly format // example: { success: false, error: "Something went wrong. Please try again.", code: "UNKNOWN_ERROR" }
sendEvent("error", errorResponse); // sends error event to frontend so it can show an error message to the user // written to stream: // "event: error\n" // 'data: {"success":false,"error":"Something went wrong..."}\n\n'
res.end(); // always close the connection on error too // otherwise browser hangs waiting for more events that will never come } });
// ───────────────────────────────────────── // START THE SERVER // ─────────────────────────────────────────
app.listen(PORT, () => { // app.listen() starts the HTTP server on the specified port // the callback function runs once when the server is ready to accept connections
console.log("\n" + "=".repeat(55)); console.log(`🚀 RAG API Server running on port ${PORT}`); console.log(` Health check: http://localhost:${PORT}/api/health`); console.log(` Load PDF: POST http://localhost:${PORT}/api/load-pdf`); console.log(` Chat: POST http://localhost:${PORT}/api/chat`); console.log(` Stream Chat: POST http://localhost:${PORT}/api/chat/stream`); console.log("=".repeat(55) + "\n"); // example terminal output when server starts: // ======================================================= // 🚀 RAG API Server running on port 3001 // Health check: http://localhost:3001/api/health // Load PDF: POST http://localhost:3001/api/load-pdf // Chat: POST http://localhost:3001/api/chat // Stream Chat: POST http://localhost:3001/api/chat/stream // ======================================================= });
Run the server:
node src/api.js
How to Call This API From Next.js
// In your Next.js app — call the non-streaming endpoint:
const response = await fetch("http://localhost:3001/api/chat", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ question: "What is this document about?", sessionId: "user_abc123", }), }); const data = await response.json(); console.log(data.answer);
// Call the streaming endpoint using EventSource:
const eventSource = new EventSource( "http://localhost:3001/api/chat/stream", // EventSource = browser API for SSE // automatically reconnects if connection drops );
// Note: EventSource only supports GET. // For POST streaming, use fetch with ReadableStream:
const res = await fetch("http://localhost:3001/api/chat/stream", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ question: "...", sessionId: "user_123" }), });
const reader = res.body.getReader(); // getReader() = get a ReadableStream reader // read() = get next chunk of streamed data
const decoder = new TextDecoder(); while (true) { const { done, value } = await reader.read(); if (done) break; const text = decoder.decode(value); // text = SSE event text // parse it to get the token content console.log(text); }
Expected API Output
# Terminal 1 — Start server
node src/api.js
=======================================================
🚀 RAG API Server running on port 3001
Health check: http://localhost:3001/api/health
Load PDF: POST http://localhost:3001/api/load-pdf
Chat: POST http://localhost:3001/api/chat
Stream Chat: POST http://localhost:3001/api/chat/stream
=======================================================
# Terminal 2 — Test with curl
# Load a PDF
curl -X POST http://localhost:3001/api/load-pdf \
-H "Content-Type: application/json" \
-d '{"pdfPath": "./sample.pdf"}'
# Response:
{
"success": true,
"message": "PDF loaded and indexed successfully",
"stats": { "pages": 8, "chunks": 32, "fileName": "sample.pdf" }
}
# Ask a question
curl -X POST http://localhost:3001/api/chat \
-H "Content-Type: application/json" \
-d '{"question": "What is this document about?", "sessionId": "user_123"}'
# Response:
{
"success": true,
"answer": "Based on Source 1 (Page 1), this document covers...",
"sessionId": "user_123"
}
3-Line Summary
- LangChain streaming uses
agent.stream()withstreamMode: "messages"which returns[token, metadata]tuples — filter formetadata.langgraph_node === "agent"to get only the final response tokens, not internal tool call tokens. - Production error handling needs custom error classes for different failure types, a retry helper with exponential backoff for transient API errors, input validation before any API call, and user-friendly error messages that never expose internal details.
- The Express API exposes three endpoints —
/api/load-pdfto index a document,/api/chatfor complete responses, and/api/chat/streamfor Server-Sent Events streaming — all sharing the same vector store and agent instance created at startup.
Module 7.2 — Complete ✅
Coming up — Module 7.3 — Next.js Frontend for the RAG API
We build the actual UI — a clean Next.js chat interface that calls the Express API, renders streaming responses word by word, shows source citations, and has a PDF upload feature. This is the complete end-to-end product.
No comments:
Post a Comment