drhawktopus/flux2-32b-qlora-rocm-windows

This repository provides a working recipe for fine-tuning a very large AI image model (the 32-billion-parameter FLUX.2 dev model) on a single consumer AMD graphics card with 24GB of memory. The setup targets native AMD ROCm on Windows, avoiding workarounds like CUDA shims or ZLUDA. The author notes that the wider community considers fine-tuning models above roughly 13 billion parameters impossible on 24GB cards, so this 32B attempt pushes well past that assumed ceiling. The current status is that the model trains entirely on the GPU at about 20 seconds per step at 512 resolution, with loss dropping as expected. The README is essentially a troubleshooting log. It lists a sequence of technical obstacles, called "walls," that the author hit and solved to make this work. These include crashes when loading the 64GB base model file, out-of-memory errors during quantization (compressing the model to fit in memory), and deadlocks in the AMD driver caused by certain features like layer offloading. The recipe also addresses confusing situations where the training process appeared to be running on the CPU but was actually on the GPU, due to misleading memory reports from separate processes on ROCm for Windows. The hardware setup is modest for this kind of work: a 32GB system RAM machine with a large 100GB pagefile to absorb one-time loading spikes. The GPU is an RX 7900 XTX, which lacks certain low-precision hardware, so the model uses a 4-bit integer format (uint4) via a library called optimum-quanto. The text encoder is a 24-billion-parameter language model (Mistral Small). Evaluation is done separately in ComfyUI using compressed GGUF versions of the models, rather than during training. Configuration details are provided for the training tool, covering settings like batch size, learning rate, gradient checkpointing, and resolution. The author notes that resolution is the main speed control. Training at 512 is fast, while 768 and 1024 spill data over the PCIe bus into system RAM and slow down dramatically. The repo also includes code patches for a custom AMD fork of the training software to fix file loading and caching. The full README is longer than what was shown.