utkarshp845/ai-infra-starter-kit

AI Infra Starter Kit is a learning lab for people who understand normal production systems but are new to AI infrastructure specifically. Instead of starting with GPU schedulers, model servers, and distributed inference tools, it begins with a plain web service and adds AI-related pieces one step at a time. The project has two small FastAPI services connected through Docker Compose. The first, demo-service, behaves like a real production API: it emits logs, metrics, and occasional intentional failures such as latency spikes and error responses. The second, ai-sre-assistant, reads those logs and explains what is happening in plain terms: which endpoints had problems, what the likely cause was, and what safe next steps to try. No API key or GPU is required to get started. If no AI provider is set up, the assistant falls back to a rule-based analyzer that does not need a language model at all. If you do want a language model, it can connect to any OpenAI-compatible provider or to Ollama running locally. The README lays out a four-week learning roadmap: week one covers the local demo service and assistant with Docker Compose, week two adds observability basics like dashboards and structured logging, week three introduces Kubernetes manifests and deployment, and week four covers security, cost, and optional advanced tools such as vLLM, Triton, and KServe. Each stage is meant to work on its own before moving to the next. Everything runs on a normal laptop on day one, with no GPU or Kubernetes cluster needed to try the basics. The project also documents itself as it goes, with a build log meant to record what broke, what became clearer, and what was intentionally left out at each step.