cliffnodes/fedoraitoolkit

FedorAiToolkit is a fork of the open source ai-toolkit project used for training LoRA and LoKr adapters on diffusion image models. The fork adds a second training stage built specifically for the Krea 2 model: after a normal supervised fine-tuning stage, the adapter is further optimized directly against differentiable reward signals computed on images the model generates while training, rather than only against a fixed dataset. A version for the Ideogram 4 model is described as still in progress. Everything from the original ai-toolkit continues to work unchanged, including support for other models, the web UI, and standard training. To use this fork, you set up its own virtual environment separate from any existing ai-toolkit install, install PyTorch and the listed requirements, and either launch the browser based UI or run training from the command line using a provided YAML config file that points at your dataset and trigger word. The added reward stage measures two things during training: a face reward that compares the generated face against reference images using the ArcFace face recognition model, and a body geometry reward that estimates body shape and proportions from generated images using Meta's SAM 3D Body model, then compares that shape against a prototype built from reference images while ignoring pose. The two rewards can be combined and weighted, and if the body model's checkpoint is not installed, training falls back to using the face reward alone. Setting up the body reward takes extra steps because the SAM 3D Body checkpoint is gated on Hugging Face: a user must request access, authenticate with a token, clone a separate repository for the model code, and download the checkpoint before enabling body training. Face-only training does not require any of this. Hardware requirements vary by configuration: an example config exists for GPUs with as little as 8GB of memory using face-only rewards at a smaller resolution, while combined face and body reward training on Krea 2 needs around 32GB of VRAM. The project is a fork, so upstream ai-toolkit's own licensing and documentation still apply beyond what this fork changes.