The demo worked.
Production didn't.

Practical engineering notes on building production-grade AI systems. Written for engineers, not for the hype cycle.

Written by Gursharan Singh, senior software engineer.

Three series, one discipline: patterns over products.
Where to start

Not sure where to begin?

New here New to building AI systems? Start with MCP or RAG. They cover the foundations everything else builds on.
Architecture Interested in how agents are put together? Start with AI Agents from Part 1.
Production After production lessons specifically? Jump to The Boundaries That Keep Agents Safe, the Agents production capstone.
Have a decision Already mid-problem and need the call, not the lesson? See Patterns in Practice: agent vs workflow, RAG vs fine-tuning, validate before commit, and more.
Hands-on Prefer learning by doing? Try the RAG Debugging Lab: several policy questions stopped finding the right document. Trace the evidence, find where it breaks, and prove your fix. Runs locally with Python.

What this is: practical engineering notes on MCP, RAG, and AI agents: the system design and production patterns that matter in real engineering work.

What this isn't: a survey of every framework, a model-picking guide, or a fine-tuning tutorial. The focus is architectural patterns that hold up in production.

Field notes

Retrieval Failure Analysis · Part 2

One Retrieval Method Is Not a Diagnosis

We re-tested the question every chunking strategy failed, this time with BM25 and rank fusion, and with every prediction committed before the run. One of our own diagnoses did not survive.

Retrieval Failure Analysis · Part 1

Why Fixed-Size Chunking Breaks Retrieval

A five-question retrieval experiment shows where fixed-size chunking breaks, why overlap doesn’t always help, and when the real problem is retrieval rather than chunking.
The series

AI Agents in Practice

8 parts

Building the loop that survives production: observe → decide → act → check → repeat, the patterns that extend it, and the boundaries that keep an agent inside its lane.

Show all 8 articles Hide articles
  1. The Demo Worked. Production Didn't.
  2. What Makes Something an Agent
  3. How the Control Loop Actually Works
  4. Five Agent Patterns and the Control Surfaces That Make Them Safe
  5. Workflow, Agent, or Single LLM Call — How to Decide
  6. Building the Production Agent Loop
  7. When the Loop Goes Wrong: Reading Agent Failures from the Trace
  8. The Boundaries That Keep Agents Safe

RAG in Practice

8 parts

Retrieval past the toy example: chunking that holds up, retrieval that returns the right thing, and the failure modes that only show up once real documents are involved.

Show all 8 articles Hide articles
  1. Why AI Gets Things Wrong
  2. What RAG Is and Why It Works
  3. How RAG Works — The Complete Pipeline
  4. Chunking, Retrieval, and the Decisions That Break RAG
  5. Build a RAG System in Practice
  6. RAG, Fine-Tuning, or Long Context?
  7. Your RAG System Is Wrong. Here's How to Find Out Why.
  8. RAG in Production — What Breaks After Launch

MCP in Practice

9 parts

How tools actually connect to models: where the seams are, how authority and scope are decided, and what it takes to wire it up without surprises.

Show all 9 articles Hide articles
  1. Why Connecting AI to Real Systems Is Still Hard
  2. What MCP Is and How AI Agents Connect
  3. How MCP Works — The Complete Request Flow
  4. MCP vs Everything Else
  5. Build Your First MCP Server (and Client)
  6. Your MCP Server Worked Locally. What Changes in Production?
  7. MCP Transport and Auth in Practice
  8. Your MCP Server Is Authenticated. It Is Not Safe Yet.
  9. From Concepts to a Hands-On Example
Across the series, TechNova is used as a fictional company so the examples stay concrete and the trade-offs stay honest.