Sample article — starter content for NeuralSys.
Introduction
The demo took a weekend. Production takes a quarter. The gap isn't model quality — it's everything around it: data flows, failure modes, cost controls, evaluation, and on-call reality.
The Production Checklist
1. Data contracts
Pin versions of everything: embeddings model, chunking config, prompt templates, tool schemas. Log the exact context that produced every consequential answer.
2. Failure modes first
Enumerate how each stage fails, then decide the degraded behavior:
| Failure | Degraded behavior |
|---|---|
| Retrieval empty | Say so + offer alternatives, never hallucinate |
| Model timeout | Retry once, then cached/fallback answer |
| Tool error | Structured error to the model, max 2 retries |
| Budget exceeded | Summarize progress, ask a pointed question |
3. Cost as a feature
Track cost per task from day one. Most AI bills are fixed with caching, smaller models for routing, and shorter contexts — not with a cheaper provider.
// Emit this on every request; alert on drift
const usage = {
tokensIn: 8420,
tokensOut: 610,
retrievalMs: 320,
modelMs: 2100,
estimatedCostUsd: 0.021,
};4. Evals in CI
Golden tasks run on every prompt/config change. If evals aren't in CI, they're folklore.
Reference Architecture
Key Takeaways
- Version prompts, embeddings, and chunking like code.
- Design degraded behavior before launch, not during the incident.
- Cost-per-task and eval pass rate are the two production KPIs.