What Makes a Chatbot Actually Useful
The anatomy of a good chatbot — and the design decisions that separate useful from annoying
Most chatbots fail not because of the AI — but because of poor design decisions made before writing a line of code. This lesson covers the decisions that determine whether your chatbot helps users or frustrates them.
The three types of chatbot
- FAQ / retrieval bot — Answers questions from a knowledge base. Deterministic, easy to test, low hallucination risk. Right for: docs, support, onboarding.
- Conversational assistant — Maintains context across a conversation and can reason about it. Right for: coaching, tutoring, complex product decisions.
- Action-taking agent — Can search the web, query a database, send messages, or trigger workflows. Right for: automation, research, task execution.
Chatbots are tools, not personalities
A hammer is not a personality — it is a tool shaped for a specific job. The same is true of chatbots. A FAQ bot, a conversational AI assistant, and an agent that takes actions are three different tools. Building the wrong one for your use case is the most common failure mode.
Design decisions that determine quality
- Scope — what it refuses to answer is as important as what it answers — An in-scope answer is helpful. An out-of-scope answer that confidently halluccinates destroys trust.
- Failure mode — what happens when it does not know the answer — The right answer is honest: "I don't have information on that. Here's where you can find it."
- Memory model — does each message stand alone or does context accumulate — Stateless (no memory) is simpler to build. Stateful (remember the conversation) is more useful. Persistent (remember across sessions) is the hardest but most powerful.
- Response length — match the question — Short factual questions deserve short answers. A question like "Explain how X works" deserves a structured response.
The stack for this course
- Next.js 15 (App Router) — API routes handle the server-side AI calls. Server Actions for form submissions.
- Anthropic Claude API — claude-sonnet-4-6 for the chat model. Streaming for real-time responses.
- Supabase — Vector storage for the knowledge base (pgvector), conversation history storage.
- Voyage AI — Embeddings for semantic search over your knowledge base.
What you will build by the end of this course
A chatbot that: • Answers questions from a custom knowledge base (your docs, products, FAQs) • Streams responses word-by-word for instant feedback • Remembers the conversation across multiple turns • Can take actions (search, look up data) via tool use • Gracefully handles questions it cannot answer • Runs in production on Vercel
Try this
Before writing any code, write a one-paragraph spec for your chatbot: Who is the user? What are the top 5 questions they will ask? What should the bot refuse to answer? What happens when it does not know? This spec will shape every design decision in the coming lessons.