Reliability, Guardrails & Observability

⏱ ~10 min read · 🎧 10 min listen · ✎ 3 quizzes · 🧪 ~90 min lab

In Lessons 06 and 07 you went 0→demo and wired it into messy enterprise systems. Now comes the transition that kills most pilots: the demo has to survive contact with real users, real adversaries, and real production traffic. Traditional software fails loudly — an exception, a 500, a stack trace. LLM systems fail quietly and persuasively: the model returns a fluent, confident answer that happens to be wrong, and nothing in your logs looks abnormal.³ Production hardening for AI systems is therefore a different discipline, built on three layers: guardrails that constrain behaviour, tracing that makes behaviour visible, and an incident mindset that assumes failures will get through.

How LLM systems fail differently

Start by naming the failure modes, because they drive every design decision that follows. An LLM component is nondeterministic — same input, different outputs across runs. It is confidently wrong — hallucinations arrive with the same fluent tone as correct answers, so there is no error signal to catch.³ And it is instructable by its own input — any text the model reads, including tool results and retrieved documents, can steer its behaviour. That last property has no equivalent in traditional software, and it is the root of the field's signature security failure.

Prompt injection: the signature failure

Prompt injection is what happens when untrusted content the model processes contains instructions — and the model follows them, because it cannot reliably distinguish trusted instructions from data. Simon Willison, who coined the term, names the dangerous combination the lethal trifecta: access to private data, exposure to untrusted content, and the ability to communicate externally. Any agent with all three can be turned into a data-exfiltration channel.¹ The uncomfortable truth you must be able to say to a customer: this is unsolved in the general case. There is no filter that reliably blocks it, because the "exploit" is just persuasive text.

This is not theoretical, and it is current. In July 2026 a researcher showed that Claude's own web-fetch tool could be exploited through links embedded in fetched pages — a honeypot site told the visiting model it was "unauthenticated," and social-engineered it into leaking the user's name, city, and employer before Anthropic closed the hole.² Note who the victim was: not the model vendor, the user of an agent that read hostile content. As an FDE, your systems in Lessons 03–07 read tickets, documents, and API responses — all untrusted. The operating rule: treat every piece of content the model ingests as potentially hostile, and design so that even a fully-hijacked model has limited blast radius — least-privilege tools, no secrets in context, human confirmation on consequential actions.¹⁴

Guardrails: layers, not a wall

Because no single defence is reliable, production systems stack layers, each catching what the previous one missed. A guardrail is any check that runs outside the model and constrains what goes in or comes out. The layers are cheap individually — a regex, a classifier call to a small model, a citation check — and their power is in combination.³

The placement question — input, output, or human gate — is a judgment call you will make constantly. A useful heuristic: filter on input when you can name the bad pattern in advance; validate on output when the failure only manifests in what the model says; gate on a human when the cost of a miss is unacceptable. The MCP specification bakes this into its security principles for tool-running systems: explicit user consent for data access and tool execution, and treating every tool description as untrusted unless it comes from a trusted server.⁴

Observability: you can't fix what you can't see

When an LLM system misbehaves, "check the logs" means nothing unless you logged the right things. LLM observability means capturing, per request: the full prompt (system + messages), every tool call and its result, token counts, latency per step, model and parameter versions, and — the one teams forget — user feedback signals. A trace strings these into one navigable record of a request's life. Langfuse, the open-source standard for this, structures traces as nested observations and adds scoring so your evals from Lesson 05 can run against live production traffic.⁶ This is what makes the difference between "a user says it was wrong once" and "here are the 14 traces where it failed, and they all share a retrieval miss."

Incident thinking completes the picture. Anthropic's team building their multi-agent research system found that agents are stateful, so errors compound — and you can't just restart on a failure that happens hours in. Their production answers: full tracing to diagnose why agents fail, resume-from-checkpoint rather than restart, and rainbow deployments — gradually shifting traffic to the new version while both run, because updating a system under a running stateful agent breaks it.⁵ You won't need all of that for a triage bot, but the mindset transfers: assume failure, make it visible, make recovery cheap.

🧪 Practical steps: harden the triage agent (~90 min)

You built a triage agent in Lesson 03 and wired it to messy systems in Lesson 07. Now you attack it, and armour it. The artifact is a hardening report plus the hardened agent — the red-team story ("here's how I broke my own system, and what I did about it") is one of the strongest interview artifacts in this course. Full instructions live in the lab file; here's the arc:

1. Copy your triage agent into D:\Projects\FDE-Portfolio\l08-hardening\.
2. Add input guardrails: an out-of-scope check and injection heuristics that flag suspicious patterns in user input and in tool results.
3. Add output guardrails: a must-cite-protocol check (every triage recommendation cites a protocol document) and a no-diagnosis rule for the triage context.
4. Add tracing with Langfuse (free tier) — or structured JSON logs if you skip the account. Trace prompts, tool calls, tokens, latencies, guardrail verdicts.
5. Write 10 adversarial prompts across three categories: injection via tool data, scope escape, and confident-wrongness bait.
6. Run the red team. Record per prompt: held / broke, and which layer caught it (or didn't).
7. Fix one failure, re-run, and document what held, what broke, and what you'd still not trust in hardening-report.md.

Feedback loop: bring your hardening report and adversarial prompts back to me in chat and I'll review them like a security-minded tech lead — are the attacks realistic, are the guardrails placed at the right layer, and are you honest about what still breaks? This artifact feeds the Lesson 11 capstone engagement and is tracked in the course's artifact tracker.

You are a security-minded staff engineer reviewing my work: I hardened an LLM triage agent by adding input guardrails (out-of-scope + injection heuristics), output guardrails (must-cite-protocol check, no-diagnosis rule), and tracing (Langfuse or structured logs), then red-teamed it with 10 adversarial prompts (injection via tool data, scope escape, confident-wrongness bait) and documented results.

Grade each criterion as Strong / Adequate / Missing, with one sentence of evidence:
- Adversarial-prompt quality: the 10 prompts are realistic attacks (including injection routed through TOOL RESULTS, not just user input), not toy variations of one idea.
- Guardrail placement rationale: each guardrail sits at the right layer (input vs output vs human gate) and the report says WHY it's there, not just what it does.
- Honesty about what broke: the report names at least one failure that got through, what layer missed it, and what residual risk remains — no "all tests passed" gloss.

Be adversarial yourself — propose 2 attacks my prompt set missed and predict whether my guardrails would catch them. Finish with the single highest-leverage hardening step I should take next.

My hardening report follows below.

Check yourself — think like an FDE

Scenarios, not recall. Diagnose from the mental model — don't scroll up. Wrong picks stay live.