Dataset in, finished pages out
A business publishing content on a fixed schedule did every step by hand: reading the source data, formatting pages, checking they were complete, uploading them. The work was repetitive but not trivial, and the failures were embarrassing: a missing image or a dead link shipped to readers because nobody had time to check.
What shaped the work
- Editors keep editorial control: the pipeline may never decide what gets published.
- A release must be able to fail safely — half-published is worse than not published.
- The output had to be plain files that the existing host could serve, with no new monthly platform.
- Non-technical staff needed to re-run a single item without touching the database.
What it runs on
Python · cron · static output · validation gates
What we actually built
- Source data in, pages out: templates plus per-item validation, run on a schedule.
- Quality gates before release: required fields, image presence, link resolution, duplicate slugs. Anything failing is held back and reported, not published.
- Failures are loud: the run leaves a log tail and an email, and the previously published version stays live.
- Single-item re-run, so an editor who fixes one record does not wait for tomorrow's batch.
- Static output is diffed before release, so a template change cannot alter hundreds of pages at once.
The choices that mattered
Gates block, humans decide
The pipeline never publishes around a failed check.
Static output
Cheap to host, trivial to back up, nothing to keep patched.
Re-run one item
The most-used feature in practice; without it, staff work around the system.
Report, do not retry blindly
A missing field is not a transient error. Retrying it just hides it three more times.
What we would do next
- Editorial preview links so a draft can be reviewed by someone outside the team before release.
- Scheduled release windows per section instead of one global batch.
What is not on this page
No client name, no screenshots of their data and no metrics: those are theirs. What is here is the engineering, which is the part a buyer can evaluate. On a call we will walk through the code.