Production system Case study 5 departments trained Zero production impact

AI Reply-Training Console for a Home-Services Company

An internal console that teaches a company's own approved voice back to its AI text assistant: pick a real customer question, read three drafts in that department's voice, edit the best, and save it, and the system reuses it on the next similar question. No model retraining.

5departments given their own voice Tens of thousandsof voice vectors embedded 7 dayssingle builder, end to end 0production impact during the build

Built by Patrick Joseph Panuncillon, AI Director and Applied AI Systems Architect. Designed, built, shipped, and operated by one builder with AI augmentation.

The problem

One great AI voice for sales, and four departments without one.

  • The company ran five customer-facing departments across many inboxes and markets, but the live AI reply assistant existed for only one of them, anchored on its top closer's voice. Leadership wanted that quality everywhere.
  • Voice drift. Pour every rep's messages into one model and the drafts come out as an averaged, committee voice. Each department needed its own anchored voice.
  • No safe place to train. The live, customer-facing assistant could not be experimented on without risking the drafts that were already working.
  • No memory of good. Approved answers were never captured, so the assistant never learned from the team's own best replies.
  • Mid-build, a new requirement: drafts had to cite the customer's real estimate, systems, prices, warranties, so pricing answers used real numbers, not generic ones.

The solution

A safe place to teach the AI, that learns as you go.

An internal web console, served from the automation host and kept strictly separate from the live assistant so training can never touch production. A trainer opens a department, picks a real customer question, reads three AI drafts written in that department's voice, edits the best one, and saves it.

Every saved answer is embedded and stored per department, then retrieved and reused on the next similar question. The assistant gets better with each session, with no model retraining. The drafting model is Claude; pricing questions pull the customer's real quote so the reply cites actual numbers.

The trust model Each department's voice is isolated so it stays clean, production is never touched during training, and a pricing answer cites the customer's real estimate, never a guess.

How it works

Eight capabilities behind the training loop.

Described at the capability level: what each part does and why it matters.

IngestMulti-inbox ingestion

A pull pipeline collects conversations from every owned inbox and regroups them by inbox to recover the per-department split, even though the messaging API returns a global set and ignores the inbox filter.

IsolatePer-department voice isolation

Each department's history is embedded into its own isolated section of a managed vector database. The live assistant's data stays untouched, so training can never pollute production.

DraftRetrieve and draft three options

For a chosen question, the app finds the most similar past situations in that department's section and drafts three reply options in its voice, with a separate model call summarizing the full conversation for context.

LearnCapture and reuse loop

When the trainer approves and saves an answer, it is stored in that department's approved-answers section and retrieved on the next similar question. The tool gets smarter immediately, no retraining.

EstimateEstimate-aware drafting

For pricing and warranty questions it resolves the customer's phone, looks up their real estimate, systems, prices, warranties, add-ons, and cites the real quote and estimate number, with a safe fallback when none exists.

QueueGuided train-a-topic

A "train all" flow walks the trainer through every customer message on a topic, one at a time, with a progress bar, so a whole topic can be trained to completion in one pass.

VoiceSwappable voice exemplars

Each department's draft voice can be anchored to one chosen rep instead of averaging everyone, configured as a one-command, swappable setting.

GovernDashboard and access gate

Leadership KPIs and per-department charts, a server-enforced access gate, same-origin serving with no cross-origin exposure, and per-department clustering of the top questions.

The phased build

Designed and shipped end to end in one week.

1

Single-department console live Days 1 to 2

The access gate, the pick, edit, save flow, and the learn loop, working for the first department on real questions.

2

Multi-department ingest

Pulled every owned inbox, recovered the per-department split, isolated each department's voice in its own section, and rebuilt the front end as a desktop dashboard with leadership KPIs.

3

Estimate-awareness and security

Scoped the estimate-aware drafting path, ran a full security review with remediation, and added authorship provenance over the build files.

4

Guided queue and verified estimates

Shipped the guided train-a-topic queue and wired estimate-aware drafting, then verified it cited a real customer's quoted systems and prices end to end.

5

Voice exemplars and full documentation

Staged the swappable voice-exemplar router and produced complete build documentation, a replication runbook, and this case study.

Results and impact

Five voices captured, production never touched.

5
Departments given their own anchored voice
Every
Owned inbox ingested, across all markets
Tens of thousands
Voice vectors embedded from the team's own history
900+
Top questions clustered across the five departments
97%
Estimate-lookup coverage across the conversation corpus
7 days
From zero to live, a single builder
0
Production impact, full isolation from the live assistant
100%
Reproducible: every step scripted and documented
Verified end to end

Estimate-aware drafting was tested live: a customer's drafts correctly cited all four quoted systems and their exact prices, and a customer with no estimate fell back cleanly to a normal reply with no leaked data.

Treated as production

A full security review covered authentication, customer-data exposure, prompt injection, and secret handling. The customer-data endpoints were closed behind an access token and input guards were added.

Built to be rebuilt

Every step, from data pull to embedding to workflow deploy to front end, is a named script with a documented runbook and SHA-256 provenance fingerprints. Given the same accounts and keys, the whole system rebuilds from zero.

What it demonstrates

The skills behind the system.

Per-persona voice isolation, no committee-averaged voice Retrieval-augmented generation on a private corpus Learn-as-you-go without model retraining Estimate-aware drafting from a real-quote lookup Multi-source data pipeline and ingestion Prompt engineering and conversation summarization Security review: auth, data exposure, prompt injection, secrets Reproducible build: scripted, fingerprinted, documented Same-origin web app and workflow automation Full-stack solo delivery: data, AI orchestration, front end, security

Outcome and what is next

Live, and feeding every department's voice.

The console is live and in daily use. The per-department voices it captures feed the customer-facing reply assistant across lines of business, and the learn loop means it keeps improving from the team's own best replies, no retraining required. It is one piece of a broader AI program.

Confidentiality and intellectual property

This is an anonymized portfolio write-up. It names no client or employer and discloses no source code, prompts, credentials, endpoints, schemas, customer data, or exact commercial terms. Third-party vendors in the client's operational stack are described by category rather than by name. Figures are rounded or banded.

Intellectual property in every system described here belongs to the organization that commissioned it. Nothing on this page is a claim of ownership, a transfer of rights, or an offer to license, sell, or reproduce any system. This page records authorship and build credit only, and it is published in a personal capacity, not on behalf of any employer or client. See Authorship and verification for the full disclosure standard.

If any part of this page is inaccurate, or a rights holder considers it to exceed that standard, contact consultwithpatrick@gmail.com and it will be corrected or removed promptly.