Case study · Pipelines
AI does the typing. Rules do the trusting.
Two data-engineering internships at NiaDelta (remote, 2023 and 2024) building an AI-assisted pipeline that consolidated scattered, hand-maintained product information into a validated catalogue — then extending it into trend reporting with the monitoring & evaluation team.
Fig. 01 — The shape of the pipeline: LLM extraction feeding a validation gate before anything reaches the catalogue.
02 — The problem
Twenty-five products, seventeen teams, no shared truth
Product information lived wherever each team kept it — different documents, different formats, different levels of freshness. Consolidating it by hand was slow and error-prone, and the errors compounded downstream in reporting. The bet: a language model could do the tedious extraction and normalisation, if — and only if — validation gates kept its mistakes from ever landing in the catalogue.
Fit the company's existing stack — Google Workspace and Sheets, not new infrastructure.
LLM output can't be trusted raw: every field passes validation before it lands.
Fully remote, async collaboration with seventeen teams that own the source data.
03 — Process
Extract, validate, then earn a second stint
STEP 01 · Extract
LLM-assisted normalisation in Apps Script
A pipeline wired through Google Apps Script called the OpenAI API to read each team's source material and normalise it into a standard product schema — replacing the manual retyping that made the catalogue perpetually stale.
STEP 02 · Validate
Rule checks before anything lands
Extraction was the easy half. Field-level validation rules and review loops across the 17 teams and 25 products caught model mistakes and source ambiguities, so the catalogue's accuracy came from the checks — not from hoping the model was right.
STEP 03 · Extend
Back in 2024: from catalogue to trends
NiaDelta brought me back for a second stint to build on the validated data — trend analysis across 18 products with the monitoring & evaluation team, turning the catalogue from a record into a decision input.
Fig. 02 — Pipeline detail figure coming soon (the work is under the company's roof; the numbers are from my internship record).
04 — Outcome
A catalogue the company could argue from
The pipeline held up in daily use — and the clearest signal it worked is that the internship became two: NiaDelta brought me back in 2024 to extend the same foundation into monitoring-and-evaluation trend reporting. It's also where my default stance on AI systems set in: let the model do the typing, and make deterministic checks do the trusting.
Next project