neural-maze/production-ocr-course

This repository is a six-week, hands-on course that teaches you how to build and deploy a document recognition system capable of real understanding, not just text extraction. Instead of simply calling an existing text-scanning service, you build a complete system that can interpret charts, tables, handwriting, and layout the way a human reader would. The system uses a small language model called Qwen 3.5, chosen because it can reason about a document's content without requiring the massive computational resources of larger models. The course targets engineers who already know how to use basic document scanning tools and want to understand what it takes to run one as a real production system. Each week covers a specific piece of the pipeline. You start by setting up cloud infrastructure with GPU-enabled virtual machines, then move through evaluating different document understanding approaches, deploying the inference engine that runs the language model, building a high-performance file intake gateway in Rust, constructing an asynchronous processing architecture, and finally securing everything behind an enterprise API gateway. The repository includes weekly articles, live coding sessions, and step-by-step deployment guides for both Azure and Google Cloud. The technical approach separates the system into distinct stages. A Rust-based gateway receives incoming files and places them into a Redis queue, decoupling file intake from the heavier GPU work. A Python worker then collects documents in short batching windows and passes them to the inference engine through shared memory, which avoids unnecessary data copying. The language model itself runs through a serving framework called vLLM, which optimizes how multiple documents are processed simultaneously. The entire system scales automatically based on demand, spinning up additional GPU nodes when busy and scaling back down to zero when idle. Security is treated as a first-class concern. The pipeline sits behind an internal load balancer with no public internet exposure, and access is controlled through an enterprise API gateway with authentication and rate limiting. The course also covers wrapping the pipeline as an MCP server, which lets AI agents like Claude Code use it directly. The course runs on cloud platforms and requires GPU access, which the README notes is the step where most people get stuck since free trial accounts typically cannot provision GPUs. The full README is longer than what was shown.