AI-Driven Development Lifecycle · Implemented, Not Aspirational

AI-DLC In Practice

From Intent to Operations

This deck follows one real change — add-privacy-classifier — through every AI-DLC phase in this monorepo: the ceremonies, the approval gates, the security review, and the evals that block promotion.

Mob Ceremonies

Elaboration → Construction

Status Gates

proposed → verified → archived

Evals Framework

Tests that block promotion

Governance Security Review Evals Observability Real Artifacts

The Storyline Why This Deck Exists

Teach the method by solving a real problem

This is a teaching deck. Instead of explaining AI-DLC in the abstract, we build something real from my own domain — and let the method prove itself along the way.

1

A real problem

privacy engineering, my expertise area

Given any document — a contract PDF, a scanned form, an email export — which PII/NPI does it contain?
And which regulations does that implicate — GDPR, HIPAA, CCPA, DPDP, LGPD, PIPEDA?
Today this is expert judgment and one-off scripts, tied to whichever system asked. It doesn't scale and it doesn't transfer.

2

A method to trust

AI-DLC, not vibe coding

AI can build this agent fast — but fast without gates means unreviewed architecture and untested claims.
AI-DLC adds ceremonies (mob elaboration, mob construction), approval gates, and evidence (security review, evals).
AI does the execution; a human stays accountable for every decision — in writing.

3

A working result

agents/privacy-classifier, in this repo

A real agent with a proposal, design, task log, security findings, and 8 evals — every excerpt in this deck is an actual file you can open.
Including the part that went wrong: Construction ran ahead of the approval gate once, got caught, and the process tightened.
The problem is mine; the method is yours to reuse.

How to teach with this deck: every phase slide follows the same pattern — the ceremony (what we did), the artifact (the file it produced), and why it matters. Swap in a real problem from your own domain and the method transfers unchanged.

The Map How to Read This Deck

Three phases, one repo — every concept is a file

AI-DLC's phases map to concrete artifacts in openspec/changes/<name>/. The rest of the deck walks them left to right.

1 Inception

Slides 4–5 · "What & why"

Mob Elaboration AI drafts scope, team validates → proposal.md

Approval Gate Repo owner signs off before build → .openspec.yaml (status field)

2 Construction

Slides 6–8 · "Build with gates"

Mob Construction AI proposes design + code, human reviews → design.md + tasks.md

Security Baseline Pre-merge review; findings fixed & logged → design.md "Security baseline"

Evals Tests that gate implemented → verified → evals/*.eval.ts

3 Operations

Slides 9–10 · "Run & retune"

Config Without Code Swappable engines, prompt overrides → .env (PII_ENGINE, PII_SYSTEM_PROMPT)

Observability OTel traces + post-hoc cost, dual export → Phoenix + OpenObserve, summary.json

Then two reference slides: single- vs multi-agent architecture patterns (slide 11) and the AI-DLC glossary with source links (slide 12).

Phase 1 · Inception Mob Elaboration

The intent becomes a reviewable proposal

The ceremony: a one-line intent goes in, AI drafts the scope, the team validates it — and nothing gets built until it's approved.

1

The ceremony

what happened, in order

Intent stated: "Classify a document's PII/NPI content and the compliance regimes it implicates — independent of any consumer."
AI drafts proposal.md: why, what changes, what's explicitly out of scope, impact on shared/.
Team validates: which compliance regimes? local vs cloud detection? columnar files in or out? (out — rejected at a gate).
Nothing is coded yet. The output of Inception is a document, not software.

2

The artifact

openspec/changes/add-privacy-classifier/proposal.md

Why: privacy work is one-off and consumer-tied; the repo needs a generic, standalone building block with a swappable detection backend.
What: new Eve agent · PDF/DOCX/TXT/images with OCR · 4 detection engines · deterministic compliance mapping (GDPR, CCPA/CPRA, HIPAA, LGPD, PIPEDA, DPDP, PDPL).
Excluded, on purpose: columnar files, fingerprinting, subagents — roadmap only.
Impact: 3 new reusable shared/ libs; Python toolchain in sandbox; no changes to other agents.

Why this matters: scope disagreements surface here — in a diff-able markdown review — not three weeks into implementation. The proposal is the contract Construction is measured against.

The Gate Inception → Construction

One status field decides what happens next

Every change carries a .openspec.yaml. Its status: field is the single source of truth for how far the change may advance.

proposed Proposal + design drafted, not yet reviewed

▶

approved Repo owner signed off — only now may Construction start

▶

implemented Code done, typechecked, security baseline passed — privacy-classifier is here

▶

verified Live smoke run + eve eval pass — evals gate this hop

▶

archived Merged and closed out

openspec/changes/add-privacy-classifier/.openspec.yaml — actual file

status: implemented
approval:
  reviewed_by: senthilsweb
  reviewed_date: 2026-07-08
  note: >
    Retroactive Inception-gate approval — design was reviewed AFTER
    implementation had completed. See AI-SDLC-TAILORING.md for the
    process-gap writeup. Status advances to `verified` once the live
    smoke run + eval run pass.

The honest part: this change was built before the gate was approved — the exact "vibe coding" failure AI-DLC exists to prevent. It was caught, approved retroactively, and the lifecycle above now exists so status reaches approved before tasks.md fills in.

Phase 2 · Construction Mob Construction

Design decides, tasks execute — decisions stay in the artifact

AI proposes design and code; the human pairs, reviews, and corrects. Every correction is logged where it happened — not in a chat that disappears.

1

design.md decides

the blueprint, approved before code

Model routing: reasoning-class orchestrator (never sees raw document text) + fast non-reasoning PII detector for glue work.
Engine routing: one env var, four swappable backends (detail on slide 9).
Loop policy: single deterministic pipeline — no subagents, no regeneration loops, one generateObject call per chunk.
Deployment constraint stated upfront: local/on-prem only; not Vercel-serverless compatible.

2

tasks.md executes

checklist + corrections + sign-off, in one file

25+ build tasks checked off: sandbox bootstrap, 15+ tools, 3 shared libs, 8 evals, typecheck clean.
Correction logged: engine genai_local_lightweight renamed to openai_privacy_filter — repo owner corrected a wrong assumption (it's not an LLM).
Correction logged: TS paragraph splitter replaced with real semantic chunking (Chonkie); dead code removed from shared/.
Sign-off recorded: who approved, when, and what's still blocked (live runs).

The audit trail is the artifact itself. **Correction:** and **Sign-off:** entries live inside tasks.md and design.md — next to the decision they changed, reviewable in any diff.

Phase 2 · Construction Quality Gate 1: Security Baseline

Pre-merge review found real vulnerabilities — and blocked the status

A change touching untrusted input cannot reach implemented without this pass. Here is what it actually caught in privacy-classifier.

Finding	Root cause	Fix	Status
Shell injection HIGH · 9/10 `detect_privacy_entities` + 2 more tools	`JSON.stringify()` as arg-escaping doesn't stop `$(…)` command substitution — and the `language` arg was free-form, reachable by the LLM.	New `shellQuote` helper (single-quote, fully literal) at all 3 call sites; `language` constrained to ISO 639-1.	✓ Fixed
Arbitrary file read HIGH · 9/10 `load_input`	Raw host path, no confinement — any authenticated caller could "classify" `/etc/passwd` and read it back from the result.	Input confined to sandbox-staged `/workspace/inputs/`; read via sandbox API, never `node:fs`. Evals restaged to match.	✓ Fixed
Quote breakout MEDIUM · hardening `read_run_file`	A `"` in the path could escape the quoted shell argument.	Hardened; downgraded from blocking — the path is orchestrator-chosen and the orchestrator never sees raw document text.	✓ Hardened

Findings are logged in design.md's "Security baseline" section — severity, confidence, root cause, and fix — then re-typechecked. It's a triggered pre-merge pass today, promoted to a CI gate when the repo takes external contributions (documented in AI-SDLC-TAILORING.md).

Phase 2 · Construction Quality Gate 2: Evals ⭐

Evals are the gate between implemented and verified

First adopter of eve's native eval harness in this repo. Eight evals assert on the actual files a run writes — not on the agent's prose reply.

8 evals, two kinds

Integration (5) — drive a real turn, then read run artifacts:
schema_conformance · result.json validates against schema
pii_entity_detection · planted email + SSN are found
engine_selection · findings carry the right engine tags
ocr_fallback · scanned PNG still yields text
columnar_rejection · CSV is refused at the gate
Unit (3) — pure deterministic checks, no model call:
compliance_mapping · HEALTH_RECORD_ID → HIPAA, not CCPA
label_alias_normalization · "ssn" → GOVERNMENT_ID_SSN
doc_type_classification · columnar gate logic

evals/schema_conformance.eval.ts — abridged

export default defineEval({
  async test(t) {
    const turn = await t.send(
      "Classify this document for PII/NPI…");
    turn.expectOk();

    // read the file the pipeline actually wrote
    const runId = extractRunId(turn.toolCalls);
    const result = readRunJson(runId, "result.json");
    t.check(result,
      matches(DocumentClassificationSchema));
  }
});

Production gates, not post-hoc checks. Assertions run against result.json and findings.normalized.json on disk. A schema drift or a missed entity fails the eval — and the change stays at implemented.

Phase 3 · Operations Retune Without a Rebuild

Swappable engines + prompt overrides — ops decisions, not PRs

Detection accuracy/cost/privacy trade-offs are runtime configuration. Same codebase, no rebuild, no code review cycle.

Engine routing — `PII_ENGINE`

Engine	Presidio	GenAI	Use when
`presidio`	✓	—	Fully local, no LLM
`presidio_genai`	✓	✓	Hybrid, merged findings
`genai_only`	—	✓	Frontier-model accuracy
`openai_privacy_filter`	—	—	Reserved — fails fast (TODO)

No default. Unset = startup error — same "no silent defaults" rule as model resolution across this repo.

Detection-prompt precedence — highest wins

1Per-invocation system_prompt argument — one-off experiments
2PII_SYSTEM_PROMPT — inline env override
3PII_SYSTEM_PROMPT_FILE — a maintained prompt file, versionable per deployment
4Bundled default — agent/config/detection_prompt.default.md

Example: "be stricter about credit-card patterns" ships as an env var today and becomes a reviewed prompt file tomorrow — the code never changes.

Phase 3 · Operations Observability

One trace per run, two backends, zero custom instrumentation

LLM calls use the AI SDK's native telemetry; non-LLM steps get a thin withSpan wrapper. Both ride the shared OTel pipeline.

How it's wired

GenAI calls: generateObject({ telemetry: { isEnabled: true } }) — tokens, latency, and cache stats come free on the AI SDK's own span.
Presidio / Docling / Chonkie (no LLM tokens): wrapped in withSpan(...) carrying runId, chunkId, engine — the AI SDK span nests inside via normal OTel context.
Dual export: set both endpoints and every span fans out to local Phoenix (dev UI) and OpenObserve (durable, queryable) independently.
Cost: computed post-hoc from summary.json against the shared rate card — no runtime overhead, same as every agent here.

one run, one trace — per-chunk spans

TRACE · privacy-classifier run
├─ extract_document_text   Docling · 2.3s
├─ chunk_text              Chonkie · 0.9s
├─ detect_pii_genai        chunk_0 · engine=genai_only
│  └─ ai.generateObject    tokens 450→180 · cache stats
├─ detect_pii_genai        chunk_1 … (bounded ×5)
├─ normalize_findings      deterministic · 45ms
└─ assemble_report         → result.json, summary.json

1 chunk = 1 span, by design. The single-agent loop policy (next slide) exists partly to keep this mapping clean.

Reference Architecture Patterns

Single-agent vs multi-agent — two real answers in this repo

Same monorepo, two deliberate choices. Pick the pattern the work demands, not the fancier one.

	privacy-classifier — single agent	api-test-generator — multi-agent
Shape	One orchestrator; PII detection is a direct `generateObject` call per chunk	Sonnet orchestrator + Opus pairwise-designer + Haiku assertion-writer subagents
Why it fits	Stateless, parallelizable extraction — no multi-turn reasoning needed	Factor analysis needs heavy reasoning; assertion writing is bulk template work
Tracing	Clean 1:1 chunk-to-span mapping (slide 10 depends on it)	Spans across 3 agents, correlated per run
Cost model	One cheap fast model for all detection calls	Per-role optimization — Opus only where reasoning pays for itself
The trade	No multi-hop refinement — a chunk gets one shot	Sandbox + conversation overhead per subagent; slower, more moving parts

The design.md says it out loud: "spinning up an isolated-sandbox subagent per chunk would add overhead with no benefit and would blur the 1:1 chunk-to-telemetry-span mapping." Architecture choices are documented with their reasons — that's the AI-DLC habit.

Reference

AI-DLC glossary — the new vocabulary

AI-DLC renames the Agile ceremonies. Same intent, rebuilt for AI-speed execution.

Term	What it means	Example
Intent	High-level business goal in plain language — the single seed.	"Generate API tests for the Patient Management API with pairwise coverage."
Mob elaboration	Inception ritual: AI drafts questions, plans, and requirements; the team validates.	AI asks: auth method? coverage strength (1-3)? which endpoints to include?
Unit of work	Independently buildable slice. Replaces the Agile epic.	Parse spec · Design pairwise · Generate assertions · Validate · Execute.
Mob construction	Construction ritual: AI proposes design, code & tests while humans pair and approve.	AI drafts pairwise factors; engineer reviews constraint definitions.
Bolt	Lightning iteration in hours or days. Replaces the Agile sprint.	Ship API test collection for 5 endpoints in a day, not a two-week sprint.
Deployment unit	Construction's finished output: validated, tested, packaged for CI.	A Postman collection that passes all 8 validation gates.
Production	Operations phase: units run in CI; results feed the next Intent cycle.	Newman runs in GitHub Actions; structured JSONL results for analysis.

AI-DLC doesn't bolt AI onto Agile — it rebuilds the SDLC from first principles: "we need automobiles, not faster horse chariots." AI does the execution; humans stay accountable for every decision.

Source: AWS AI-DLC Method Definition Paper — Raja SP, AWS

References — concepts & inspirations: AI-DLC·specs.md·fabriqa.ai·dltHub AI Workbench·ai-agents repo