tencent-hunyuan/gear

GEAR is a research project from Tencent Hunyuan and Peking University that proposes a new way to train AI image generation models. It accompanies a published paper and provides the official PyTorch code. Most modern AI image generators that use autoregressive (token-by-token) generation follow a two-step pipeline: first a "tokenizer" compresses images into a sequence of discrete codes (tokens), and then a separate model learns to predict those codes in order to generate new images. These two components are almost always trained independently. GEAR's core contribution is training them together in a single end-to-end pass, so the tokenizer learns to produce tokens that are easier for the generator to predict. The technical challenge is that the tokenization step involves choosing the single best code for each image patch (an argmax operation), which is not differentiable and normally blocks gradient information from flowing back into the tokenizer during generator training. GEAR works around this with a dual-path approach: one path uses the hard discrete codes to train the generator as usual, while a second, mathematically softer version of the same step carries a gradient signal back to update only the tokenizer. The two remain decoupled, so neither interferes with the other's training objective. The practical result, shown on the standard ImageNet benchmark, is roughly 10 times faster convergence to a strong generation quality score (gFID), compared to training the tokenizer and generator separately. On a text-to-image task the improvement is even larger on certain metrics. The repository releases pre-trained tokenizer weights for three different quantizer variants (VQ, LFQ, IBQ) and provides training and evaluation scripts. An NVIDIA GPU with CUDA support is required to run any part of it. This project is intended for AI researchers, not general users.