Problem
Client AI initiatives were slow to ship: model deployment was manual, onboarding new engineers took weeks, and there was no secure, shared substrate for running multi-model LLM and agent workloads in production. Every team was re-inventing the unsafe parts — auth, cost control, deploys — instead of shipping product.
Architecture
A Kubernetes/AWS platform running multi-model LLM and generative-AI pipelines, fronted by SSO-secured agent infrastructure and an internal developer platform with self-service onboarding. Standardized CI/CD on GitHub Actions, with SLAs, incident-response processes, and full observability (Prometheus, Grafana, OpenTelemetry).
Approach
I treated the platform as a product with internal customers, not a pile of bespoke pipelines. One secured, model-agnostic substrate: a self-service portal for onboarding, an orchestration layer with MCP tooling and Rust agent runtimes, and governance — SSO, per-team cost attribution, and an audit trail — baked in from day one. Everything shipped through standardized CI/CD with SLAs and real observability, so teams could move fast without owning the dangerous parts themselves.
System design
What I built
- Multi-model orchestration pipelines for LLMs and generative AI
- Integrated Model Context Protocol (MCP) infrastructure with per-team cost attribution
- Self-service developer platform with automated onboarding
- Enterprise AI governance (Obot/Bifrost) — audited, access-controlled LLM usage
- Standardized CI/CD and incident-response across a ~50-engineer org