Integrating with Messy Enterprise Systems

⏱ ~10 min read · 🎧 10 min listen · ✎ 3 quizzes · 🧪 ~120 min lab

Your Lesson 06 demo ran on data you controlled. The customer's data is nothing like that. It lives in a SOAP API last documented in 2011, a nightly CSV export with three date formats, and a "data warehouse" that is actually someone's Excel macro. This is not the failure case — this is the normal case, and it's a large part of why 95% of enterprise GenAI pilots show no measurable impact.⁶ The FDE day-to-day is on-site integrations, debugging against the customer's real systems, and data migrations¹ — writing production code directly on customer infrastructure.⁸ The engineers who thrive here aren't the ones who complain the data is dirty. They're the ones with a system for it.

The mess is the job

Every enterprise engagement hits the same four walls: legacy interfaces (SOAP, FTP drops, screen-scraped mainframes), exports of doom (CSVs with inconsistent encodings, duplicated rows, and columns that changed meaning in 2019), inconsistent schemas (three systems, three definitions of "patient"), and auth mazes (VPN → jump box → service account → a ticket queue owned by a team in another country). None of this is exotic — it's the texture of the job.¹ The mistake is treating it as a blocker to escalate; the FDE move is to treat it as the terrain to plan around:

Week one: the data-access strategy

Access requests are the longest lead-time item in any engagement, so they go in on day one — before you've decided exactly what you'll build. The week-one ask list: a schema or data dictionary for each source system, a small recent sample export (a hundred rows beats a million), read-only credentials to the least-sensitive system first, the name of each data owner, and the approval path for anything touching sensitive records. Ask for the smallest thing that can possibly be granted quickly — a redacted sample this week is worth more than a full production dump next quarter.

And while you wait, you don't wait: you build against synthetic data shaped like theirs. Take the schema (or even a screenshot of one), have Claude generate realistic fake records — including the defects you expect, because clean synthetic data trains you for a world that doesn't exist. This is a core FDE skill, not a workaround: it decouples your build from their bureaucracy, it's safe to demo to anyone, and it becomes your test fixture forever. You'll use it again as the first step of the Lesson 11 capstone.

One instinct must become reflex before any real data arrives: PII is radioactive until proven otherwise. In Australian health contexts the stakes are explicit — privacy law has strict rules on how health information is collected, used and disclosed⁷ — but the practical instincts are the same everywhere, and they're yours to hold even when the customer is casual about it. Don't accept data you didn't ask for. Prefer redacted or synthetic samples for development. Keep real data on their infrastructure, not your laptop. And design every interface you build to return the minimum fields the task needs — PII minimization is an engineering property, not a compliance afterthought. (This is practical craft, not legal advice — when in doubt, ask their privacy officer. They exist, and they will love you for asking.)

MCP: wrap the mess once

Once you can read the data, resist the urge to pipe it straight into a prompt. The durable move is to put a stable seam between the mess and the model — and in 2026 that seam is the Model Context Protocol. MCP is an open standard for connecting AI applications to external systems: you wrap the ugly source once as an MCP server exposing a few clean, well-described tools, and then every MCP client — Claude, your L03 agent, the customer's future apps — can use it without knowing a SOAP endpoint exists.³ This is why Anthropic's own FDE posting lists "MCP servers" first among the technical artifacts FDEs deliver to customers²: the cleaning logic, the canonical schema, and the PII minimization all live in one auditable place instead of being re-implemented in every prompt.

Two things to know beyond the basics. First, the MCP spec bakes in security principles you should honour in every server you ship: user consent, data privacy, and caution around tool behaviour — tools are arbitrary code paths and must be treated accordingly.⁴ Second, scale: when an agent has access to many MCP tools, loading every tool definition and passing every intermediate result through the model's context gets expensive. Anthropic's answer is code execution with MCP — present servers as code APIs, load tools on demand, and filter large results before they hit the context window, which in their worked example cut a 150,000-token workflow to about 2,000 tokens.⁵ You don't need that on day one; you need to know it exists before your tool count hits fifty.

Data-quality triage: fix, work around, or escalate

Not every defect deserves the same response, and burning a week polishing data nobody queries is how pilots die. Triage every defect with three questions: does it affect the fields my use case actually reads? Is it mechanical (fixable by rule — date formats, casing, dupes) or semantic (two systems genuinely disagree about facts)? And is it mine to fix — or evidence the customer needs to see? Fix mechanical defects in your pipeline and log every decision. Work around defects outside your use case's blast radius — note them, move on. Escalate semantic conflicts and upstream process failures to the data owner with evidence, because silently "fixing" a business-meaning conflict means your system now disagrees with the system of record, and that surfaces in the worst meeting possible. Honest caveat: there's no canonical FDE playbook for this — it's synthesized field practice, and your own logged decisions become your playbook.

🧪 Practical steps: Mess-to-MCP (~120 min)

This lab produces the most reusable portfolio artifact so far: a deliberately messy synthetic dataset, a defect log, a cleaning pipeline with logged decisions, and a working Python MCP server your Lesson 03 triage agent can query end-to-end. Full instructions live in the lab file (labs/0007-mess-to-mcp.md); the shape:

1. Generate the mess — use Claude (model claude-opus-4-8) to generate a synthetic export for the virtual-care provider: patients.csv, appointments.csv, and triage_notes.json, with deliberate defects — at least three date formats, duplicated patients with variant spellings, mixed schemas across files, and PII-ish fields (fake Medicare-style numbers, phones, addresses).
2. Profile it — write profile.py (pandas) and document at least 5 defects in defect-log.md: what, where, how many rows, and your triage call (fix / work around / escalate).
3. Clean it — clean.py normalizes dates to ISO-8601, dedupes patients with a stated matching rule, and maps everything to one canonical schema. Every non-obvious choice goes in decisions.md: what you decided, why, what you'd ask the data owner.
4. Wrap it — build server.py, a small Python MCP server exposing two tools: lookup_patient and get_appointment_history. Responses return only task-minimal fields — no raw identifiers, no addresses.
5. Connect and prove it — point your Lesson 03 triage agent at the server and capture one end-to-end query (question → tool calls → PII-minimized answer) in demo-transcript.md.
6. Ship it — everything goes to D:\Projects\FDE-Portfolio\l07-mcp-integration\ with a README telling the story: mess → profile → decisions → seam → working query.

Feedback loop: bring your defect log, decisions file, and demo transcript back to me in chat and I'll review them like a customer's lead engineer would — are the defects real and quantified, are the cleaning decisions defensible and transparent, and would I let this server's responses near a privacy review? This artifact feeds the Lesson 11 capstone engagement directly and is tracked in the course's artifact tracker.

You are a customer's lead data engineer reviewing a Forward Deployed Engineer's integration work. They took a deliberately messy synthetic healthcare export (2 CSVs + 1 JSON with inconsistent dates, duplicated patients, mixed schemas, PII-ish fields), profiled it, cleaned it, and wrapped it as a Python MCP server exposing patient-lookup and appointment-history tools, then ran one end-to-end agent query.

Grade each criterion as Strong / Adequate / Missing, with one sentence of evidence:
- Defect-log quality: at least 5 defects, each with location, row counts, and an explicit triage call (fix / work around / escalate) — not vague observations like "dates are inconsistent".
- Cleaning-decision transparency: every non-obvious transformation (dedup matching rule, date disambiguation, schema mapping) is logged with a rationale and a question they'd take to the data owner — nothing silently "fixed".
- PII minimization in tool responses: the MCP tools return only task-minimal fields; identifiers, contact details, and addresses never appear in responses, and the demo transcript proves it.

Be skeptical — hunt for a silent semantic fix that should have been escalated, and for any field in a tool response that the task didn't need. Then ask 2–3 follow-up questions a privacy officer would ask. Finish with the single change that would most improve this integration.

My files follow below.

Check yourself — think like an FDE

Scenarios, not recall. Diagnose from the mental model — don't scroll up. Wrong picks stay live.