Risk Management & Compliance, a US freight-insurance agency, was chasing renewals manually across hundreds of trucking carriers. Carrier data arrives as spreadsheets of varying quality and has to be reconciled against a federal identifier, enriched, summarised, and pushed into the agency's CRM — with zero tolerance for a fabricated identifier or an invented fact reaching an underwriter.
An unverified identifier reaching the model's context is a correctness defect — and it gets tracked as one.
A provider-pluggable LLM pipeline (OpenAI + Gemini selected by configuration) generating structured Letters of Experience from carrier submissions — schema-constrained, with deterministic post-processing before anything downstream consumes the output.
Prompts constrained to declare absence: where a fact is missing from the submission, the model must state that verbatim rather than infer a plausible value.
The import pipeline, CRM integration, VIN verification flow, and per-carrier microsites with engagement analytics. Seven modules carry unit tests; work ships through issue-linked pull requests with a review gate.
A missing field is a good outcome; a confidently invented one is a disaster. The LOE prompt is required to output 'Information not provided with this submission' verbatim when a fact isn't in the source — the difference between a demo and a document an underwriter will sign.
View deep diveA fallback identifier was being treated downstream as authoritative. It was traced through three separate consumers — the rendered microsite, the CRM sync, and the LLM prompt — and each was fixed as its own tracked issue. Output verification is table stakes; keeping unverified data out of the model's context is the other half.
View deep diveOpenAI and Gemini sit behind one interface, selected by configuration. When a provider deprecates a model, the fix is an env change — not an emergency rewrite in a system an agency runs its renewals on.
View deep dive