soumith/ganhacks

★ 11,623Audience · researcherComplexity · 1/5Setup · easy

Mindmap

mindmap
  root((repo))
    What it is
      17 training tips
      GAN stability guide
      NIPS 2016 conference notes
    Topics covered
      Input normalization
      Loss function choice
      Batch normalization
      Label handling
    Who it helps
      ML practitioners
      GAN researchers
      Grad students
    Limitations
      Not maintained post-2020
      Assumes GAN background
      No runnable code

mindmap root((repo)) What it is 17 training tips GAN stability guide NIPS 2016 conference notes Topics covered Input normalization Loss function choice Batch normalization Label handling Who it helps ML practitioners GAN researchers Grad students Limitations Not maintained post-2020 Assumes GAN background No runnable code

Click or tap to explore — scroll the page freely

Things people build with this

USE CASE 1

Debug an unstable GAN training run by checking each of the 17 tips against your current setup.

USE CASE 2

Review batch normalization and loss function choices before starting a new GAN project.

USE CASE 3

Identify early signs of training divergence using the symptom descriptions in the checklist.

Getting it running

Difficulty · easy Time to first run · 5min

No license information was mentioned in the explanation.

In plain English

A GAN (Generative Adversarial Network) is a type of AI model used to generate content, such as synthetic images, by pitting two networks against each other: one tries to produce realistic output, and the other tries to detect fakes. Training GANs is notoriously difficult and unstable. This repository is a collection of practical tips for making GAN training work reliably. It started as notes from a presentation at a major AI research conference (NIPS 2016) and was compiled by several AI researchers including Soumith Chintala, Emily Denton, and Martin Arjovsky. The authors note at the top that the list is no longer maintained and may be less relevant than it was circa 2016 to 2020. The 17 tips cover topics like how to normalize input data, which loss function to use in practice versus what the original GAN paper proposes, how to sample random noise for the generator, how to set up batch normalization, how to avoid certain types of broken gradients, how to handle labels during training, and how to spot early signs that training has gone wrong. This is not a beginner resource. It assumes the reader already knows what a GAN is, understands how neural network training works, and is comfortable with terms like batch normalization, optimizer, and discriminator. It reads more like a checklist for a practitioner debugging a training run than like a tutorial. For someone new to AI or GANs, this would be a difficult starting point without prior background in machine learning. For someone who does have that background and is running into stability problems with a GAN project, this short document is a compact reference to common remedies.

Copy-paste prompts

Prompt 1

My GAN training is producing blurry output and the discriminator loss dropped to near zero. Using the ganhacks tips, diagnose what is likely wrong and suggest PyTorch fixes.

Prompt 2

I'm starting a new GAN project in PyTorch. Based on ganhacks, what loss function, normalization, and noise sampling choices should I begin with?

Prompt 3

Explain the label smoothing and feature matching tips from ganhacks and show me how to implement them in a PyTorch GAN training loop.

Prompt 4

My GAN generator is stuck in mode collapse and only produces one type of image. Which ganhacks tips address this and how do I apply them?

Open on GitHub → Explain another repo

← soumith on gitmyhub — every repo by this author, as a profile.

Verify against the repo before relying on details.