fguzman82/gategpt

gateGPT is a working AI language model built entirely from hardware logic on a Xilinx Virtex-5 FPGA chip, rather than running as software on a CPU or GPU. It implements a small character-level text generation model, specifically trained to produce names, and outputs them on the board's built-in LCD screen at roughly 50,000 to 69,000 characters per second. An FPGA is a type of chip whose internal connections can be reconfigured by the programmer. Instead of running code on a processor, gateGPT describes the transformer computations as digital logic circuits written in Verilog, a hardware description language. The result is a custom piece of silicon behavior that runs the AI inference directly in the chip's gates and arithmetic units. The model itself is small by design: one transformer block with 24 dimensions, a vocabulary of 27 characters (the alphabet plus a period), and a context window of 16 characters. All arithmetic uses fixed-point numbers rather than floating point, which is better suited to FPGA resources. The architecture is organized as a microcode sequencer, meaning a small instruction memory controls which hardware module runs at each step, rather than hardwiring a single monolithic circuit. The engineering history documented in the README shows how the design improved its speed 28 times over the first working version. The biggest single gain came from adding a KV cache, which is a way of saving intermediate computation results so the chip does not need to reprocess the entire context from scratch for every new character. Other gains came from running multiply operations in parallel and reworking the memory access patterns. A rotary encoder on the board lets you adjust generation speed and how varied the outputs are. The project includes Python tools for generating the microcode, checking the bit-exact match between the hardware and a software reference, and training the weights. The design uses 23% of the chip's logic slices and 96% of its dedicated multiply units, making the multipliers the main limiting resource.