OpsPilot

## Context Most LLM demos break down in production because the platform pieces are missing: tenancy, observability, eval gates, and cost controls. OpsPilot is my production-shaped, open-source project to build those muscles on top of a real incident/runbook domain. ## Constraints - Must be multi-tenant and cost-aware. - Must be measurable (latency + quality baselines), not vibes. - Must be safe by default (audit logs, minimal permissions, no data leakage). ## What I am building - **Layer 0**: deterministic data + baseline RAG with citations, plus an eval harness that I can re-run as a release gate. - **Layer 1**: multi-tenant isolation + a Spring Boot gateway for auth/rate limits/audit logs + end-to-end traces. - **Layer 2**: LLM infra excellence (local inference, caching/batching, routing, budgets) with measured deltas. - **Layer 3**: tool-using agents via MCP, tool governance, threat model, and failure drills. ## Impact - A defensible portfolio artifact for AI platform engineering: quality gates, cost controls, and observability that you can audit in code. - A reusable foundation for incident + FinOps copilots across data/ML and platform systems (without over-claiming “magic AI”). ## Stack (baseline + planned) Python (FastAPI + LangGraph/LangChain), Spring Boot gateway, Postgres + pgvector, llama.cpp (local), optional vLLM, OpenTelemetry traces, Prometheus/Grafana, Kubernetes.