Most consultants show you slides. Here's my production stack.

The design rule the whole stack obeys: no application ever talks to an AI provider directly. Every call — local or commercial — goes through one router (LiteLLM) that exposes a single OpenAI-compatible API and enforces:

• · Routing policy — which workloads may use which models. Sensitive routes pin to local models on my hardware — those bytes never leave. Routes I allow onto frontier providers send the minimum necessary context, under no-training API terms. The boundary is configuration, not convention — and the logs show exactly what crossed it.
• · Cost visibility — per-call tracking into dashboards. I know what every model, team, and workflow costs, in real time.
• · Budgets — per-key limits so an experiment can't torch a month's budget in an afternoon.
• · Swappability — a provider change is a config diff. When a model is deprecated or a price doubles, applications never notice.

Around the router:

• · Chat workspace: LibreChat — tracking upstream, deliberately NOT forked. My customizations live in build-time patch scripts that locate structural anchors in the code, apply idempotently, and fail loudly if upstream refactors — so I get upstream's fixes and features forever without maintaining a divergent fork. Two years of upstream churn survived; zero forks maintained.
• · Local inference — GPU-backed models on unified-memory hardware, systemd-supervised, started before the containers that need them.
• · Document pipeline — the routed OCR system (page-type routing, honest quality manifests, versioned outputs). Detail below.
• · Monitoring: Prometheus + Grafana — exporters on every datastore, container metrics, cost dashboards, alert rules.
• · Configuration as a single source of truth — one manifest compiles into every service config. A pre-flight check refuses to render when live files have drifted. Every change is a git commit.

• · Tested recovery. Bare-metal rebuild of the full stack in under two hours — rehearsed, documented, timed. Backups every six hours across five datastores, synced off-site, with restore procedures that have actually been run.
• · Staged changes. Every risky change goes through a written pre-flight: image bumps researched against upstream changelogs, volume changes diffed, destructive commands individually flagged, staging before production. Boring by design.
• · Runbooks over memory. Every operating procedure is a script or a document in the repo. The standard I hold: the stack must be operable by someone who isn't me. That's the same standard your handover gets.
• · Real post-mortems. When it breaks, the failure gets a written root-cause and the fix gets automated. Two favorites from the scars: an out-of-memory crash that taught me hard limits on every heavy container, and a health-check misconfiguration that was quietly costing ~€20/month in API calls — found through the cost dashboards, fixed in one line, verified by the same dashboards. Small numbers; the discipline is the point.