h9-tec/production-ai-stack

Production AI Stack is not a piece of software you install, it is a written reference guide about which tools to actually use when building AI systems that run in real production, not just demos. The author says it comes from years of building large language model systems for enterprises, including retrieval systems over more than one hundred million pages, real time Arabic voice agents, and multi agent backends, plus projects that cut monthly costs from around fifty two thousand dollars down to under five thousand. Each part of the stack is written in the same format: a default choice the author reaches for first, a note on when to switch to something else, and a list of traps that caused real problems in past projects. The stated biases up front are favoring boring proven technology, starting with Postgres before adding specialized databases, preferring tools that can be self hosted but also offer a managed option, and insisting that everything in the system can be replayed later for debugging. The guide is organized into topics covering model serving, embeddings and rerankers, gateways and routing, retrieval and storage, document parsing, chunking, orchestration, agent frameworks, structured output, real time voice, observability, guardrails and security, fine tuning, caching, and notes specific to Arabic and multilingual systems. A summary table lists a default tool, a strong alternative, and a managed cloud option for each layer. For model serving specifically, the default recommendation is vLLM, valued for its wide hardware support and well documented failure modes, with SGLang recommended instead when the workload involves heavy shared prompts or large scale structured JSON output. The author also warns that benchmarks should be run on your own traffic patterns, and that quantized models can quietly lose quality on Arabic text even when they still pass English language tests. The full README is longer than what was shown.