CodeWithGagan | Programming Language and IT Lectures

Phase 1: Foundations — Part 1

The Problem Every App Faces

Every application that lets people log in has to answer three questions, constantly, for every single request:

Who is this person? (Authentication)
What are they allowed to do? (Authorization)
Are they still who they say they are, right now? (Session validity)

Building this yourself sounds easy at first — a users table, some hashed passwords, done. But real apps quickly need:

Password reset flows
Email verification
Multi-factor authentication (MFA)
Social logins (Google, GitHub, Microsoft)
Session expiry and refresh
Account lockout after failed login attempts
Audit logs for compliance

Building and maintaining all of this correctly is a full-time specialty. Getting it wrong is how companies get breached. This exact problem is what an Identity Provider (IdP) exists to solve.

What is IAM?

Identity and Access Management (IAM) is the general discipline of managing:

Identity — who a user is
Access — what that user is allowed to do

Okta is a company that provides IAM as a hosted, external service. Instead of you building login, MFA, password recovery, and user storage inside your own Next.js app, you outsource all of that to Okta.

What This Means for You as a Developer

Your job changes. You are no longer writing authentication logic yourself — you are integrating with an authentication provider using a standard protocol called OpenID Connect (OIDC), which sits on top of OAuth 2.0. We'll cover that protocol in full detail in Lecture 2.

Where Okta Actually Sits in Your App

Here's the mental model, in plain terms:

User clicks "Log in" on your Next.js app.
Your app redirects the user to Okta — a login page hosted on Okta's own servers, not yours.
The user enters their password (and does MFA if required) on Okta's page. Your app never sees the password.
Okta authenticates the user and redirects them back to your app, along with proof of who they are (a signed token).
Your app verifies that proof and starts a session for the user.

That's the entire authentication flow at a conceptual level. Every lecture from Module 2 onward is about implementing these five steps with real TypeScript code in Next.js.

One More Thing to Know

Okta actually has two product lines:

Workforce Identity Cloud — for logging employees into internal company tools.
Customer Identity Cloud — for logging your app's end users (customers) in.

This course focuses on the developer-facing OIDC/OAuth layer, which works identically for both — so nothing here changes based on which one you're using.

OAuth 2.0 and OpenID Connect, Explained Properly

You've heard these terms thrown around, and Okta's docs assume you already know them. You don't need to feel that way after this lecture. We're going to build up the concepts slowly, with real reasoning behind each piece, so nothing feels like magic later.

Start With the Actual Problem

Imagine you're building an app called "TaskFlow." A user wants to log in. Somewhere, a check needs to happen: "does this password match this email?" You have two options.

Option A: Build your own login system. Store passwords (hashed), build a login form, handle "forgot password," handle account lockouts, eventually add MFA. You own all of it, forever, including every security patch.

Option B: Let a specialist (Okta) hold the credentials and do the checking. Your app never touches a password. Okta just tells your app "yes, this is [email protected], and here's proof."

Almost every serious app today picks Option B. But Option B raises a new question: how does Okta prove to your app that a user is who they say they are, in a way your app can trust, without your app and Okta needing to secretly share passwords?

That question is exactly what OAuth 2.0 and OpenID Connect answer. They are not Okta-specific — they're open standards. Okta is just one company that implements them (Google, Auth0, Microsoft Entra ID, and others also do). Once you understand these protocols, you understand any modern login system, not just Okta's.

OAuth 2.0: Solving "Access," Not "Identity"

Here's a common point of confusion, so let's kill it immediately: OAuth 2.0 was never designed to answer "who is this user?" It was designed to answer a narrower question: "can this app access this specific resource, on this user's behalf, without knowing the user's password?"

Think of the classic example: a photo-printing website wants to print your Google Photos, without you typing your Google password into the printing website. OAuth 2.0 lets you approve, from within Google's own page, that the printing site can access just your photos — nothing else, and only for as long as you allow.

The output of OAuth 2.0 is an access token — a piece of proof that says "the bearer of this token is allowed to access X." Critically, an access token does not, by design, guarantee anything about who the user actually is. It's a permission slip, not an ID card.

So Where Does "Identity" Come From? OpenID Connect

This is where OpenID Connect (OIDC) comes in. OIDC is a thin, standardized layer built directly on top of OAuth 2.0. It adds exactly one crucial thing that OAuth 2.0 alone doesn't guarantee: a reliable, verifiable answer to "who is this user?"

OIDC does this by introducing a new token type: the ID token. Unlike the access token (a permission slip), the ID token is explicitly an identity document. It's a signed piece of data containing claims like the user's email, name, and a unique user ID, and your app can cryptographically verify it hasn't been tampered with.

So, in one sentence that will save you a lot of confusion going forward:

OAuth 2.0 grants access. OpenID Connect proves identity. When people say "Okta uses OAuth," what they really mean is "Okta uses OIDC," because OIDC is what actually logs a user in. You'll use both together, always.

The Three Tokens You'll See Constantly

Once login succeeds, Okta hands your app up to three tokens. You'll work with these directly starting in Phase 2, so get familiar with their purpose now:

ID Token — Proves who the user is. Contains identity claims (email, name, user ID). You use this to know who's logged in. Never send this to an API as proof of authorization — that's not its job.
Access Token — Proves what the bearer is allowed to do. You attach this to requests to protected APIs (including your own Next.js API routes later). It does not reliably contain identity information you should trust for display purposes.
Refresh Token — A long-lived token used to silently get new Access/ID tokens once the old ones expire, without forcing the user to log in again. We cover this properly in Phase 3.

The Authorization Code Flow, Step by Step

There are several ways ("flows" or "grant types") OAuth/OIDC define for getting these tokens. Some are outdated for browser apps and considered insecure today — if you find an old tutorial using the "Implicit Flow," skip it; it's deprecated and Okta actively discourages it. The flow we'll use throughout this course, and the one Okta recommends for basically everything now, is the Authorization Code Flow with PKCE. Here's exactly what happens, in order:

User clicks "Log in" on your Next.js app.
Your app redirects the browser to Okta's /authorize endpoint, including your app's Client ID, the permissions you're requesting (called scopes — e.g. openid profile email), and a redirect URI telling Okta where to send the user back afterward.
Before this redirect, your app generates a random secret called a code verifier, and sends a hashed version of it (the code challenge) along with the request. This is the "PKCE" part — Proof Key for Code Exchange. It exists to stop an attacker from intercepting the authorization code in step 5 and using it themselves.
The user authenticates on Okta's page — types their password, does MFA if required. Your app is never involved in this step; it's entirely on Okta's domain.
Okta redirects the browser back to your redirect URI, appending a short-lived, one-time-use authorization code as a query parameter.
Your app's backend takes that code, along with the original code verifier from step 3, and sends both directly to Okta's /token endpoint (a server-to-server request, not visible to the browser).
Okta checks that the code verifier matches the code challenge from step 3. If it matches, Okta responds with the ID token, access token, and (if requested) refresh token.
Your app stores these tokens (we'll cover exactly where, safely, in Phase 2) and the user is now logged in.

Why does PKCE matter so much that Okta requires it by default now? Because step 5's authorization code briefly travels through the browser's URL bar, which can potentially be intercepted (via browser history, referrer headers, or a badly behaved extension). Without PKCE, anyone who steals that code could redeem it themselves. With PKCE, the code is useless to a thief because they don't have the original code verifier, which never left your app's server.

Scopes and Claims — Two Terms You'll See Everywhere

Two words you'll run into constantly, so let's define them precisely now:

A scope is a request for a category of access — think of it as a checkbox you're asking the user to approve. openid is a mandatory scope for OIDC logins. profile and email request that the ID token includes the user's name and email.
A claim is an actual piece of data returned inside a token as a result of scopes you requested. Request the email scope, and you receive an email claim inside your ID token. Later, in Phase 5, you'll learn to create your own custom claims (like a user's role) to enforce authorization logic in your app.

Okta's Core Vocabulary

Now that you understand the protocol, here's the specific vocabulary Okta uses to implement it. Every one of these terms will reappear throughout the course, so it's worth being precise now.

Org — Your entire Okta account is called an "Org" (short for organization). It's the top-level container for everything: your users, your applications, your settings. When you sign up, you get a unique domain like dev-12345678.okta.com — this is your Org's base URL, and you'll use it constantly in configuration.

Application (or "App Integration") — A registration inside your Org representing one specific app that wants to use Okta for login — in our case, the Next.js dashboard we're building. Each Application gets its own Client ID, and (depending on type) a Client Secret. When you set one up, Okta asks what type of application it is — critically, whether it's a "Web Application" (has a secure backend, can keep a Client Secret private — this is what our Next.js app will be) versus a "Single-Page Application" (all code runs in the browser, so no secret can be kept safe, and PKCE alone protects it).

Authorization Server — The actual component that issues tokens and validates login requests, per the flow we just walked through. This part has some nuance worth knowing now: every Okta Org automatically comes with an Org Authorization Server, which handles basic SSO login but cannot be customized (no custom scopes or claims). Separately, Okta Orgs (including the free Integrator Free Plan we'll use) come with a default Custom Authorization Server, literally named default, which can be customized — this is the one we'll use starting in Phase 2, because Phase 5 requires custom claims that only a Custom Authorization Server supports. One detail worth flagging now so it doesn't surprise you later: on the free Integrator Free Plan, this default authorization server does not come with a basic access policy attached — we'll add one ourselves when we get to setup.

Users — Individual identity records inside your Org. Each has a profile (email, name, custom attributes you define) and credentials.

Groups — Collections of Users, used for organizing permissions. Instead of assigning access rules to individual users one by one, you assign a user to a Group (like "Admins" or "Editors") and write rules against the Group. This becomes central in Phase 5 when we build role-based access control.

Sign-On Policies / Authentication Policies — Rules attached to an Application or Authorization Server controlling how someone is allowed to log in — for example, "require MFA if logging in from a new device." We'll configure these hands-on in Phase 4.

What's Next

You now have the full theoretical foundation: why identity providers exist, exactly how the Authorization Code Flow with PKCE works step by step, what each token is for, and the specific Okta terms you'll be clicking through in the admin console. Nothing above required you to touch a keyboard yet — that changes in the next post.

Next we'll do the two hands-on steps that finish this phase: creating your free Okta Integrator Free Plan account and registering your first Application, followed by setting up the actual Next.js + TypeScript + Tailwind project we'll build on for the rest of the course.

The Complete Okta + Next.js Integration Course — Roadmap

Introduction

Welcome to this complete, from-scratch course on integrating Okta with Next.js. If you've ever tried to learn Okta by piecing together random blog posts and outdated GitHub repos, you know the pain — half the code uses the Pages Router, half uses packages that don't support Server Components, and none of it explains why things work the way they do. This course fixes that. Every lecture is researched fresh against the latest official Okta documentation and current package versions before it's written, and everything is explained in plain, beginner-friendly language with real, working TypeScript code.

By the end of this course, you won't just know how to "add login" to a Next.js app — you'll understand identity and access management deeply enough to handle MFA, social logins, role-based access, custom claims, backend user management, and production security, all inside a real project you build incrementally, lecture by lecture.

Who This Course Is For

You should already be comfortable with:

Basic React (components, hooks, state)
Basic Next.js (pages, routing, the general idea of the App Router)
Basic TypeScript syntax

You do not need any prior knowledge of authentication, OAuth, OIDC, or Okta itself. We start from zero on all of that.

The Tech Stack We'll Use

Next.js, latest version, App Router (Route Handlers, Middleware, Server Components)
TypeScript throughout — no plain JavaScript
Tailwind CSS for all UI
@okta/okta-auth-js (currently v8.x) — Okta's own core SDK, used directly rather than through a third-party wrapper, so you learn every Okta capability, not just a simplified subset
Okta Node Management SDK — for backend/admin operations later in the course
A free Okta Integrator Free Plan account (Okta's current free developer org type — the older "Developer Edition" orgs were retired in 2025, so we'll use the current one)

The Project We'll Build

Rather than scattering disconnected code snippets across lectures, we build one real application throughout the course: a small SaaS-style dashboard. It starts as a bare Next.js app with no auth at all, and by the end it has full login, MFA, social sign-in, role-based permissions, an admin user-management panel, and a production deployment. Each lecture adds one real feature to this same codebase, so you always see how pieces connect — nothing is a throwaway demo.

How the Course Is Structured

The course is organized into seven phases. Each phase is a logical stage of maturity for the app — starting with pure theory, then basic login, then real Okta features, then authorization, then backend/admin work, then production concerns. Lectures within a phase build directly on each other, so they should be followed in order the first time through.

Phase 1 — Foundations

This phase has zero code. It exists so you never write a line of authentication logic without understanding what it's actually doing. We cover:

What Identity and Access Management is, and why apps offload it to a provider like Okta.
OAuth 2.0 and OpenID Connect explained from first principles — what a token is, what "authorization code flow" and "PKCE" mean, and why these specific standards exist instead of everyone rolling their own login system.
Okta's own vocabulary: what an "Org" is, what an "Authorization Server" is, the difference between an "Application" and a "User," and what "Groups" are used for.
Creating your free Okta account and registering your very first application inside it, step by step, with screenshots-in-words of exactly what to click.
Setting up the actual Next.js project — TypeScript config, Tailwind config, folder structure — that we'll use for the rest of the course.

By the end of Phase 1, you'll understand the theory completely and have both your Okta org and your Next.js project ready, but the app still won't have login yet — that's intentional, so the concepts are solid before the code arrives.

Phase 2 — Basic Authentication

This is where the app gets real login. We implement the full Authorization Code Flow with PKCE by hand using @okta/okta-auth-js, including:

The login route that redirects users to Okta
The callback Route Handler that receives Okta's response and exchanges it for tokens
Where and how to safely store those tokens (and why cookies are the right call in the App Router, not localStorage)
A proper logout flow — both logging out of your app and logging out of the Okta session itself
Protecting pages using Next.js Middleware, so unauthenticated users are automatically redirected
Reading the logged-in user's identity inside Server Components

By the end of Phase 2, your dashboard app has real, working login and logout.

Phase 3 — User Profile & Session Management

What the ID Token, Access Token, and Refresh Token each actually contain and are used for
Silently renewing tokens before they expire, so users aren't randomly logged out mid-session
Fetching and displaying the user's profile data from Okta, and letting them update it
Adding your own custom fields to a user's Okta profile

Phase 4 — Security Features (Okta's Real Power)

This is the phase that makes Okta worth using over a homemade login system. We cover, with hands-on setup in your Okta org each time:

Multi-Factor Authentication — Okta Verify push, SMS, email, and authenticator apps
Adaptive/contextual sign-on policies (e.g., only require MFA from unrecognized devices)
Self-service registration, so users can sign themselves up
Self-service password recovery
Social login — adding Google, Microsoft, or GitHub as sign-in options through Okta
Passwordless authentication using Magic Links and Passkeys (WebAuthn)

Phase 5 — Authorization: Who Can Do What

Authentication tells you who someone is. Authorization decides what they're allowed to do. This phase covers:

Scopes and claims, including adding your own custom claims to tokens
Groups and Role-Based Access Control, and enforcing roles inside your Next.js app
Securing your own Next.js API Route Handlers so they check access tokens properly
An introduction to Okta FGA (Fine-Grained Authorization) for permission models more complex than simple roles

Phase 6 — Backend & Admin Operations

Here we step outside the login flow and use Okta from your server code directly:

Using the Okta Node Management SDK to create, update, and manage users programmatically
Inline Hooks and Event Hooks — running your own custom logic during Okta's authentication process
The basics of SCIM provisioning for enterprise user sync

Phase 7 — Production Readiness

The final phase turns your working demo into something you could actually ship:

Managing environment variables and secrets properly, plus a full security checklist
Testing authentication flows, both unit tests and end-to-end
Using Okta's System Log for monitoring and debugging real login issues
Deploying the finished app to Vercel with production-grade Okta settings
A final capstone lecture tying every phase together

How Each Post From Here Will Work

Going forward, each post will cover as much of a phase as fits properly in one sitting — sometimes a full phase, sometimes two or three lectures' worth of closely related topics — but never rushed. If a topic needs more room to be explained properly, it gets its own post rather than being compressed. You'll always get full explanations and working code, never a shortened summary.

Module 7.3 — Next.js Frontend for the RAG Chatbot

Building the complete chat UI that connects to your Express RAG API

What We're Building

A clean Next.js chat interface that:

→ Loads a PDF via the Express API
→ Shows streaming responses word by word
→ Displays source citations
→ Maintains conversation history
→ Handles loading and error states

This connects to the Express API you built in Module 7.2.

Project Setup

npx create-next-app@latest rag-frontend

Answer the prompts:

✔ Would you like to use TypeScript? → No
✔ Would you like to use ESLint? → No
✔ Would you like to use Tailwind CSS? → Yes
✔ Would you like to use src/ directory? → No
✔ Would you like to use App Router? → Yes
✔ Would you like to customize the import alias? → No

cd rag-frontend

Create .env.local:

NEXT_PUBLIC_API_URL=http://localhost:3001

Project Structure

rag-frontend/
├── .env.local
├── app/
│   ├── layout.js        ← root layout
│   ├── page.js          ← main chat page
│   ├── globals.css      ← global styles
│   └── api/             ← (not needed — we call Express directly)
├── components/
│   ├── ChatWindow.js    ← message list display
│   ├── MessageBubble.js ← single message component
│   ├── InputBar.js      ← question input and send button
│   ├── PDFLoader.js     ← PDF upload section
│   └── StatusBar.js     ← shows what's happening
└── hooks/
    ├── useChat.js       ← handles all chat logic + streaming
    └── usePDFLoader.js  ← handles PDF loading

Step 1 — Hooks (Business Logic)

`hooks/usePDFLoader.js`


    "use client";
    // "use client" = this runs in the browser, not on the server
    // Required for hooks that use useState, useEffect, fetch

    import { useState } from "react";

    export function usePDFLoader() {
        // This hook handles everything about loading a PDF

        const [isLoading, setIsLoading] = useState(false);
        // isLoading = true while PDF is being indexed
        // used to show spinner and disable the load button

        const [isLoaded, setIsLoaded] = useState(false);
        // isLoaded = true after PDF indexed successfully
        // enables the chat input

        const [error, setError] = useState(null);
        // error = string message if something went wrong
        // null = no error

        const [pdfStats, setPdfStats] = useState(null);
        // pdfStats = info about loaded PDF
        // example: { pages: 8, chunks: 32, fileName: "sample.pdf" }

        async function loadPDF(pdfPath) {
            // pdfPath = path to PDF on the Express server
            // example: "./sample.pdf"

            if (!pdfPath.trim()) {
                setError("Please enter a PDF file path");
                return;
            }

            setIsLoading(true);
            setError(null);
            // reset error before new attempt

            try {
                const response = await fetch(
                    `${process.env.NEXT_PUBLIC_API_URL}/api/load-pdf`,
                    // NEXT_PUBLIC_API_URL = http://localhost:3001
                    // read from .env.local file
                    {
                        method: "POST",
                        headers: { "Content-Type": "application/json" },
                        body: JSON.stringify({ pdfPath }),
                        // send pdfPath to Express API
                    }
                );

                const data = await response.json();
                // data = { success: true, stats: {...} }
                // or    { success: false, error: "..." }

                if (!response.ok || !data.success) {
                    throw new Error(data.error || "Failed to load PDF");
                }

                setPdfStats(data.stats);
                // example: { pages: 8, chunks: 32, fileName: "sample.pdf" }

                setIsLoaded(true);
                // enables the chat input

            } catch (err) {
                setError(err.message);
                setIsLoaded(false);
            } finally {
                setIsLoading(false);
                // always stop loading spinner — success or fail
            }
        }

        function reset() {
            // called when user wants to load a different PDF
            setIsLoaded(false);
            setPdfStats(null);
            setError(null);
        }

        return {
            isLoading,
            isLoaded,
            error,
            pdfStats,
            loadPDF,
            reset,
        };
    }

`hooks/useChat.js`


    "use client";

    import { useState, useCallback, useRef } from "react";

    const API_URL = process.env.NEXT_PUBLIC_API_URL;
    // http://localhost:3001

    export function useChat() {
        // This hook manages all chat state and streaming logic

        const [messages, setMessages] = useState([]);
        // messages = array of message objects
        // example:
        // [
        //   { id: "1", role: "user",      content: "What is this about?", isStreaming: false },
        //   { id: "2", role: "assistant", content: "This document...",    isStreaming: false },
        // ]

        const [isStreaming, setIsStreaming] = useState(false);
        // true while AI is generating a response
        // used to disable input and show typing indicator

        const [error, setError] = useState(null);
        // error message to show if something goes wrong

        const sessionId = useRef(`session_${Date.now()}`);
        // useRef = persists across renders without causing re-renders
        // sessionId is created once and stays the same
        // example: "session_1752672000000"
        // sent with every message so Express API tracks conversation

        const abortControllerRef = useRef(null);
        // abortControllerRef = lets us cancel a streaming request
        // if user clicks Stop or navigates away
        // .current = the actual AbortController object

        const addMessage = useCallback((role, content, id) => {
            // adds a new message to the messages array
            // role = "user" or "assistant"
            // content = message text
            // id = unique identifier

            setMessages(prev => [
                ...prev,
                // keep all existing messages

                {
                    id: id || `msg_${Date.now()}_${Math.random()}`,
                    // unique ID — used as React key
                    // Date.now() + random = guaranteed unique

                    role,
                    // "user" or "assistant"

                    content,
                    // the actual text

                    isStreaming: false,
                    // false = complete message
                    // true = currently being streamed (partial)

                    timestamp: new Date().toISOString(),
                    // when this message was created
                }
            ]);
        }, []);
        // useCallback = memoizes the function
        // only recreated if dependencies change (empty array = never)

        const updateLastMessage = useCallback((updater) => {
            // updates the last message in the array
            // updater = function that returns new message object
            // used during streaming to append new tokens

            setMessages(prev => {
                const messages = [...prev];
                // copy array

                const lastIndex = messages.length - 1;
                // index of last message

                messages[lastIndex] = updater(messages[lastIndex]);
                // call updater with current last message
                // updater returns updated message object

                return messages;
            });
        }, []);

        async function sendMessage(question) {
            if (!question.trim() || isStreaming) return;
            setError(null);

            // Add user message
            addMessage("user", question);

            // Add empty assistant message with loading state
            setMessages(prev => [...prev, {
                id: `msg_${Date.now()}`,
                role: "assistant",
                content: "",
                isStreaming: true,
                timestamp: new Date().toISOString(),
            }]);

            setIsStreaming(true);

            try {
                // Use non-streaming endpoint — cleaner for RAG
                const response = await fetch(
                    `${API_URL}/api/chat`,
                    {
                        method: "POST",
                        headers: { "Content-Type": "application/json" },
                        body: JSON.stringify({
                            question,
                            sessionId: sessionId.current,
                        }),
                    }
                );

                const data = await response.json();

                if (!response.ok || !data.success) {
                    throw new Error(data.error || "Request failed");
                }

                // Update the assistant message with the final answer
                updateLastMessage(msg => ({
                    ...msg,
                    content: data.answer,
                    // data.answer = clean LLM response, no raw chunks
                    isStreaming: false,
                }));

            } catch (err) {
                if (err.name !== "AbortError") {
                    setError(err.message);
                    updateLastMessage(msg => ({
                        ...msg,
                        content: "Sorry, something went wrong. Please try again.",
                        isStreaming: false,
                    }));
                }
            } finally {
                setIsStreaming(false);
            }
        }

        //  This is the original streaming version of sendMessage, which is commented out. It uses Server-Sent Events (SSE) to stream tokens from the server as they arrive. The current implementation uses a non-streaming endpoint for simplicity and better RAG handling.

        // async function sendMessage(question) {
        //     // question = what the user typed

        //     if (!question.trim() || isStreaming) return;
        //     // ignore empty input or if already streaming

        //     setError(null);

        //     // Add user message immediately
        //     addMessage("user", question);
        //     // user sees their message right away
        //     // doesn't wait for API response

        //     // Add empty assistant message — will be filled by streaming
        //     const assistantMsgId = `msg_${Date.now()}`;
        //     setMessages(prev => [
        //         ...prev,
        //         {
        //             id: assistantMsgId,
        //             role: "assistant",
        //             content: "",
        //             // starts empty — tokens appended as they stream in
        //             isStreaming: true,
        //             // true = show typing indicator
        //             timestamp: new Date().toISOString(),
        //         }
        //     ]);

        //     setIsStreaming(true);

        //     // Create AbortController to allow cancellation
        //     abortControllerRef.current = new AbortController();
        //     // AbortController = browser API to cancel fetch requests
        //     // .signal = passed to fetch() so it can be cancelled

        //     try {
        //         const response = await fetch(
        //             `${API_URL}/api/chat/stream`,
        //             {
        //                 method: "POST",
        //                 headers: { "Content-Type": "application/json" },
        //                 body: JSON.stringify({
        //                     question,
        //                     sessionId: sessionId.current,
        //                     // send sessionId so Express tracks conversation history
        //                 }),
        //                 signal: abortControllerRef.current.signal,
        //                 // signal = allows this request to be aborted
        //             }
        //         );

        //         if (!response.ok) {
        //             const errorData = await response.json();
        //             throw new Error(errorData.error || "Request failed");
        //         }

        //         // Read the streaming response
        //         const reader = response.body.getReader();
        //         // response.body = ReadableStream of SSE data
        //         // getReader() = get a reader to consume the stream

        //         const decoder = new TextDecoder();
        //         // TextDecoder = converts raw bytes to string
        //         // SSE data comes as bytes (Uint8Array)

        //         let buffer = "";
        //         // buffer = accumulates incomplete SSE lines
        //         // SSE data may arrive in chunks that split across lines


        //         while (true) {
        //             const { done, value } = await reader.read();
        //             if (done) break;

        //             buffer += decoder.decode(value, { stream: true });

        //             const lines = buffer.split("\n");
        //             buffer = lines.pop() || "";

        //             let currentEvent = "";
        //             // tracks the current event type
        //             // SSE format:
        //             // event: token          ← event type line
        //             // data: {"content":"x"} ← data line
        //             //                        ← blank line = end of event

        //             for (const line of lines) {
        //                 if (line.startsWith("event: ")) {
        //                     currentEvent = line.slice(7).trim();
        //                     // "event: token" → "token"
        //                     // "event: done"  → "done"
        //                     // "event: error" → "error"
        //                     // "event: start" → "start"
        //                 }

        //                 if (line.startsWith("data: ")) {
        //                     const jsonStr = line.slice(6).trim();
        //                     // remove "data: " prefix

        //                     if (!jsonStr || jsonStr === "") continue;
        //                     // skip empty data lines

        //                     try {
        //                         const data = JSON.parse(jsonStr);

        //                         if (currentEvent === "token" && data.content) {
        //                             // append token to last message
        //                             updateLastMessage(msg => ({
        //                                 ...msg,
        //                                 content: msg.content + data.content,
        //                             }));
        //                         }

        //                         if (currentEvent === "done") {
        //                             // streaming complete
        //                             updateLastMessage(msg => ({
        //                                 ...msg,
        //                                 isStreaming: false,
        //                             }));
        //                         }

        //                         if (currentEvent === "error" && data.success === false) {
        //                             throw new Error(data.error || "Streaming error from server");
        //                         }

        //                     } catch (parseError) {
        //                         if (parseError.message.includes("Streaming error")) {
        //                             throw parseError;
        //                             // re-throw actual errors
        //                         }
        //                         // ignore JSON parse errors from empty/malformed lines
        //                     }

        //                     currentEvent = "";
        //                     // reset event type after processing data
        //                 }
        //             }
        //         }

        //     } catch (err) {
        //         if (err.name === "AbortError") {
        //             // user cancelled — not an error
        //             updateLastMessage(msg => ({
        //                 ...msg,
        //                 content: msg.content + " [stopped]",
        //                 isStreaming: false,
        //             }));
        //         } else {
        //             setError(err.message);
        //             updateLastMessage(msg => ({
        //                 ...msg,
        //                 content: "Sorry, something went wrong. Please try again.",
        //                 isStreaming: false,
        //             }));
        //         }
        //     } finally {
        //         setIsStreaming(false);
        //         abortControllerRef.current = null;
        //         // cleanup
        //     }
        // }

        function stopStreaming() {
            // called when user clicks Stop button
            if (abortControllerRef.current) {
                abortControllerRef.current.abort();
                // cancels the fetch request
                // triggers AbortError in the catch block above
            }
        }

        function clearMessages() {
            setMessages([]);
            setError(null);
            // start fresh conversation
            // Note: this only clears UI — server still has history
        }

        return {
            messages,
            isStreaming,
            error,
            sendMessage,
            stopStreaming,
            clearMessages,
        };
    }

Step 2 — Components

`components/PDFLoader.js`


    "use client";

    import { useState } from "react";

    export default function PDFLoader({ onLoaded, isLoading, isLoaded, error, pdfStats }) {
        // Props:
        // onLoaded  = callback when user clicks Load button
        // isLoading = true while indexing
        // isLoaded  = true after success
        // error     = error message string
        // pdfStats  = { pages, chunks, fileName }

        const [pdfPath, setPdfPath] = useState("./sample.pdf");
        // pdfPath = what user types in the input box
        // default to sample.pdf so they can test immediately

        function handleSubmit(e) {
            e.preventDefault();
            // prevent page reload on form submit
            onLoaded(pdfPath);
            // call parent's load function with the path
        }

        if (isLoaded) {
            // Show success state after PDF is loaded
            return (
                <div className="bg-green-50 border border-green-200 rounded-lg p-4 mb-4">
                    <div className="flex items-center gap-2 text-green-700">
                        <span className="text-xl">✅</span>
                        <div>
                            <p className="font-medium">Document loaded: {pdfStats?.fileName}</p>
                            <p className="text-sm text-green-600">
                                {pdfStats?.pages} pages • {pdfStats?.chunks} chunks indexed
                            </p>
                        </div>
                    </div>
                </div>
            );
        }

        return (
            <div className="bg-white border border-gray-200 rounded-lg p-4 mb-4">
                <h2 className="text-lg font-semibold text-gray-800 mb-3">
                    📄 Load a PDF Document
                </h2>

                <form onSubmit={handleSubmit} className="flex gap-2">
                    <input
                        type="text"
                        value={pdfPath}
                        onChange={(e) => setPdfPath(e.target.value)}
                        // controlled input — updates pdfPath state on every keystroke
                        placeholder="Enter PDF path (e.g. ./sample.pdf)"
                        className="flex-1 border border-gray-300 rounded-lg px-3 py-2 text-sm
                        focus:outline-none focus:ring-2 focus:ring-blue-500"
                        disabled={isLoading}
                    // disable while loading so user can't change path mid-index
                    />

                    <button
                        type="submit"
                        disabled={isLoading || !pdfPath.trim()}
                        // disabled if loading OR if input is empty
                        className="bg-blue-600 text-white px-4 py-2 rounded-lg text-sm font-medium
                        hover:bg-blue-700 disabled:bg-gray-300 disabled:cursor-not-allowed
                        transition-colors"
                    >
                        {isLoading ? (
                            <span className="flex items-center gap-2">
                                <span className="animate-spin">⟳</span>
                                Indexing...
                            </span>
                        ) : (
                            "Load PDF"
                        )}
                    </button>
                </form>

                {error && (
                    <p className="text-red-600 text-sm mt-2">❌ {error}</p>
                    // show error message below the form
                )}

                {isLoading && (
                    <p className="text-gray-500 text-sm mt-2">
                        Loading PDF and creating embeddings — this takes a moment...
                    </p>
                )}
            </div>
        );
    }

`components/MessageBubble.js`


    "use client";

    export default function MessageBubble({ message }) {
        // message = { id, role, content, isStreaming, timestamp }

        const isUser = message.role === "user";
        // isUser = true for user messages, false for assistant

        return (
            <div className={`flex ${isUser ? "justify-end" : "justify-start"} mb-4`}>
                {/* Align user messages right, assistant messages left */}

                <div className={`max-w-[80%] ${isUser ? "order-2" : "order-1"}`}>

                    {/* Avatar */}
                    <div className={`flex items-end gap-2 ${isUser ? "flex-row-reverse" : "flex-row"}`}>
                        <div className={`w-8 h-8 rounded-full flex items-center justify-center text-sm
                ${isUser ? "bg-blue-600 text-white" : "bg-gray-200 text-gray-600"}`}>
                            {isUser ? "U" : "🤖"}
                        </div>

                        {/* Message bubble */}
                        <div className={`rounded-2xl px-4 py-3 
                ${isUser
                                ? "bg-blue-600 text-white rounded-br-sm"
                                : "bg-white border border-gray-200 text-gray-800 rounded-bl-sm shadow-sm"
                            }`}>

                            {/* Message content */}
                            <p className="text-sm leading-relaxed whitespace-pre-wrap">
                                {message.content}
                                {/* whitespace-pre-wrap = preserve line breaks in AI responses */}

                                {message.isStreaming && (
                                    <span className="inline-block w-2 h-4 bg-current ml-1 animate-pulse" />
                                    // blinking cursor while streaming
                                    // animate-pulse = Tailwind's pulse animation
                                )}
                            </p>

                            {/* Timestamp */}
                            <p className={`text-xs mt-1 ${isUser ? "text-blue-200" : "text-gray-400"}`}>
                                {new Date(message.timestamp).toLocaleTimeString("en-IN", {
                                    hour: "2-digit",
                                    minute: "2-digit",
                                })}
                            </p>
                        </div>
                    </div>
                </div>
            </div>
        );
    }

`components/ChatWindow.js`


    "use client";

    import { useEffect, useRef } from "react";
    import MessageBubble from "./MessageBubble";

    export default function ChatWindow({ messages, isStreaming }) {
        // messages   = array of message objects
        // isStreaming = true while AI is responding

        const bottomRef = useRef(null);
        // bottomRef = ref to an invisible div at the bottom of the chat
        // used to scroll to the bottom when new messages arrive

        useEffect(() => {
            bottomRef.current?.scrollIntoView({ behavior: "smooth" });
            // scroll to bottom whenever messages change
            // happens when: user sends message, AI responds, token arrives
            // behavior: "smooth" = animated scrolling
        }, [messages]);
        // dependency array = only run when messages changes

        if (messages.length === 0) {
            // Empty state — shown before any messages
            return (
                <div className="flex-1 flex items-center justify-center text-gray-400">
                    <div className="text-center">
                        <p className="text-4xl mb-3">💬</p>
                        <p className="text-lg font-medium">Ask a question about your document</p>
                        <p className="text-sm">Load a PDF above to get started</p>
                    </div>
                </div>
            );
        }

        return (
            <div className="flex-1 overflow-y-auto p-4">
                {/* overflow-y-auto = scroll when content is taller than container */}

                {messages.map(message => (
                    <MessageBubble key={message.id} message={message} />
                    // key = message.id (unique — required by React for lists)
                ))}

                {/* Typing indicator — shown while streaming before first token */}
                {isStreaming && messages[messages.length - 1]?.content === "" && (
                    <div className="flex justify-start mb-4">
                        <div className="bg-white border border-gray-200 rounded-2xl px-4 py-3 shadow-sm">
                            <div className="flex gap-1">
                                <span className="w-2 h-2 bg-gray-400 rounded-full animate-bounce" style={{ animationDelay: "0ms" }} />
                                <span className="w-2 h-2 bg-gray-400 rounded-full animate-bounce" style={{ animationDelay: "150ms" }} />
                                <span className="w-2 h-2 bg-gray-400 rounded-full animate-bounce" style={{ animationDelay: "300ms" }} />
                            </div>
                            {/* Three bouncing dots — classic typing indicator */}
                        </div>
                    </div>
                )}

                <div ref={bottomRef} />
                {/* Invisible div at the bottom — scrollIntoView targets this */}
            </div>
        );
    }

`components/InputBar.js`


    "use client";

    import { useState, useRef, useEffect } from "react";

    export default function InputBar({ onSend, onStop, isStreaming, isDisabled }) {
        // onSend     = function to call when user submits a question
        // onStop     = function to call when user clicks Stop
        // isStreaming = true while AI is responding
        // isDisabled  = true if no PDF is loaded yet

        const [input, setInput] = useState("");
        // input = current text in the textarea

        const textareaRef = useRef(null);
        // ref to the textarea element
        // used to auto-resize it as user types

        useEffect(() => {
            if (textareaRef.current) {
                textareaRef.current.style.height = "auto";
                // reset height first

                textareaRef.current.style.height =
                    Math.min(textareaRef.current.scrollHeight, 120) + "px";
                // set height to content height (up to 120px max)
                // creates auto-growing textarea
            }
        }, [input]);
        // run whenever input changes

        function handleSubmit(e) {
            e?.preventDefault();
            // e?.preventDefault() = prevent form submit if called from form
            // ? = optional chaining (works even if e is undefined)

            if (!input.trim() || isStreaming || isDisabled) return;
            // don't submit if empty, already streaming, or no PDF loaded

            onSend(input.trim());
            // pass trimmed question to parent

            setInput("");
            // clear input after sending
        }

        function handleKeyDown(e) {
            if (e.key === "Enter" && !e.shiftKey) {
                // Enter = submit
                // Shift+Enter = new line (don't submit)
                e.preventDefault();
                handleSubmit();
            }
        }

        return (
            <div className="border-t border-gray-200 bg-white p-4">
                <form onSubmit={handleSubmit} className="flex gap-2 items-end">

                    <textarea
                        ref={textareaRef}
                        value={input}
                        onChange={(e) => setInput(e.target.value)}
                        onKeyDown={handleKeyDown}
                        placeholder={
                            isDisabled
                                ? "Load a PDF first to start chatting..."
                                : "Ask a question about your document... (Enter to send)"
                        }
                        disabled={isDisabled || isStreaming}
                        rows={1}
                        className="flex-1 border border-gray-300 rounded-xl px-4 py-3 text-sm
                        resize-none overflow-hidden focus:outline-none focus:ring-2
                        focus:ring-blue-500 disabled:bg-gray-50 disabled:text-gray-400"
                    // resize-none = disable manual resize (we auto-resize)
                    // overflow-hidden = hide scrollbar (we expand instead)
                    />

                    {isStreaming ? (
                        // Show Stop button while streaming
                        <button
                            type="button"
                            onClick={onStop}
                            className="bg-red-500 text-white px-4 py-3 rounded-xl font-medium text-sm
                        hover:bg-red-600 transition-colors"
                        >
                            ⏹ Stop
                        </button>
                    ) : (
                        // Show Send button normally
                        <button
                            type="submit"
                            disabled={!input.trim() || isDisabled}
                            className="bg-blue-600 text-white px-4 py-3 rounded-xl font-medium text-sm
                        hover:bg-blue-700 disabled:bg-gray-300 disabled:cursor-not-allowed
                        transition-colors"
                        >
                            Send ↑
                        </button>
                    )}
                </form>

                <p className="text-xs text-gray-400 mt-2 text-center">
                    Press Enter to send • Shift+Enter for new line
                </p>
            </div>
        );
    }

Step 3 — Main Page

`app/page.js`


  "use client";

  import { useChat } from "@/hooks/useChat";
  import { usePDFLoader } from "@/hooks/usePDFLoader";
  import PDFLoader from "@/components/PDFLoader";
  import ChatWindow from "@/components/ChatWindow";
  import InputBar from "@/components/InputBar";

  export default function Home() {
    // Home = the main page component
    // everything comes together here

    const pdfLoader = usePDFLoader();
    // pdfLoader.isLoading, pdfLoader.isLoaded, pdfLoader.loadPDF, etc.

    const chat = useChat();
    // chat.messages, chat.isStreaming, chat.sendMessage, etc.

    return (
      <div className="flex flex-col h-screen bg-gray-50">
        {/* h-screen = full viewport height */}
        {/* flex flex-col = stack children vertically */}

        {/* Header */}
        <header className="bg-white border-b border-gray-200 px-6 py-4 flex items-center justify-between shadow-sm">
          <div>
            <h1 className="text-xl font-bold text-gray-900">📚 PDF Chatbot</h1>
            <p className="text-sm text-gray-500">Ask questions about your documents</p>
          </div>

          {chat.messages.length > 0 && (
            // Only show Clear button if there are messages
            <button
              onClick={chat.clearMessages}
              className="text-sm text-gray-500 hover:text-gray-700 border border-gray-300 rounded-lg px-3 py-1.5 hover:bg-gray-50 transition-colors"
            >
              Clear chat
            </button>
          )}
        </header>

        {/* Main content area */}
        <main className="flex-1 flex flex-col overflow-hidden max-w-3xl w-full mx-auto px-4 pt-4">
          {/* max-w-3xl = max width for readability */}
          {/* mx-auto = center horizontally */}

          {/* PDF Loader — always shown at top */}
          <PDFLoader
            onLoaded={pdfLoader.loadPDF}
            isLoading={pdfLoader.isLoading}
            isLoaded={pdfLoader.isLoaded}
            error={pdfLoader.error}
            pdfStats={pdfLoader.pdfStats}
          />

          {/* Error banner for chat errors */}
          {chat.error && (
            <div className="bg-red-50 border border-red-200 rounded-lg p-3 mb-4 text-red-700 text-sm">
              ❌ {chat.error}
            </div>
          )}

          {/* Chat messages */}
          <ChatWindow
            messages={chat.messages}
            isStreaming={chat.isStreaming}
          />
        </main>

        {/* Input bar — fixed at bottom */}
        <div className="max-w-3xl w-full mx-auto px-4">
          <InputBar
            onSend={chat.sendMessage}
            onStop={chat.stopStreaming}
            isStreaming={chat.isStreaming}
            isDisabled={!pdfLoader.isLoaded}
          // disable input until PDF is loaded
          />
        </div>
      </div>
    );
  }

`app/layout.js`


  import "./globals.css";

  export const metadata = {
    title: "PDF Chatbot",
    description: "Ask questions about your PDF documents",
  };

  export default function RootLayout({ children }) {
    return (
      <html lang="en">
        <body className="antialiased">
          {children}
        </body>
      </html>
    );
  }

Step 4 — Run Everything

Terminal 1 — Start Express API:

cd langchain-production-rag
node src/api.js

Terminal 2 — Start Next.js:

cd rag-frontend
npm run dev

Open http://localhost:3000

What You'll See

┌─────────────────────────────────────────┐
│  📚 PDF Chatbot         [Clear chat]    │
├─────────────────────────────────────────┤
│                                         │
│  ┌─────────────────────────────────┐    │
│  │ 📄 Load a PDF Document          │   │
│  │ [./sample.pdf          ] [Load] │    │
│  └─────────────────────────────────┘    │
│                                         │
│           💬                           │
│   Ask a question about your document    │
│   Load a PDF above to get started       │
│                                         │
├─────────────────────────────────────────┤
│  [Ask a question...            ] [Send] │
│      Press Enter to send                │
└─────────────────────────────────────────┘

After loading PDF:
┌─────────────────────────────────────────┐
│  ✅ Document loaded: sample.pdf         │
│     8 pages • 32 chunks indexed         │
├─────────────────────────────────────────┤
│                              [U]        │
│            What is this about?     ✓    │
│                                         │
│  [🤖]                                  │
│   Based on Source 1 (Page 1), this      │
│   document covers AI engineering...▌    │
│   ← words appear as they stream         │
└─────────────────────────────────────────┘

3-Line Summary

The useChat hook handles all streaming logic — it reads the SSE response using response.body.getReader(), decodes each chunk, parses SSE data: lines as JSON, and appends each token to the last message in state to create the word-by-word effect.
The usePDFLoader hook calls the Express /api/load-pdf endpoint and tracks loading state — once isLoaded is true the InputBar is enabled and the PDF stats are shown at the top.
The component hierarchy is simple — page.js connects the two hooks and passes props down to PDFLoader, ChatWindow, and InputBar — each component has one clear job and none of them fetch data directly.

Module 7.3 — Complete ✅

Coming up — Module 7.4 — Pinecone Cloud Vector Database

We replace the in-memory vector store with Pinecone — a production cloud vector database. Data persists across restarts, scales to millions of documents, and comes with a full web dashboard to visualize your stored vectors.