polyu-vclab/wrc

This repository holds the code for a research paper called Weighted Reverse Convolution, or WRC, from researchers at Nanjing University of Aeronautics and Astronautics, The Hong Kong Polytechnic University, and Nanjing University. The paper looks at a problem in computer vision: large pretrained image models like DINO and CLIP produce useful features for understanding images, but those features are spatially coarse because the models split images into fairly large patches. That coarseness hurts tasks that need precise detail, such as marking exact object boundaries or matching the same point across two images. WRC is a method that takes those coarse features and reconstructs a sharper, higher resolution version of them, without retraining or modifying the original pretrained model. It treats the problem as an inverse math problem: given the low resolution features, it solves for what the high resolution version likely looked like, using a weighted least squares approach with regularization. The authors say this can be solved efficiently using Fourier transforms, which keeps the method fast and fully differentiable so it can be plugged into existing pipelines. The code is organized as a Python package. There is a training script, a folder of Hydra configuration files for datasets, models and optimizers, an evaluation folder with scripts and wrappers for external benchmarks, and a custom C++ and CUDA extension that implements the core computation for speed. Setup involves creating a Python 3.12 environment with conda, then installing PyTorch and the other pinned dependencies with pip and uv. The README shows results on several benchmarks, including semantic segmentation on Cityscapes, video object segmentation on DAVIS, unsupervised object discovery on COCO20K, and keypoint matching on SPair-71k, using a frozen DINOv2 vision model as the backbone. It reports that WRC produces sharper similarity maps and improves accuracy compared to other upsampling techniques while keeping inference fast on a single GPU. This is academic, research grade code rather than a polished consumer tool, aimed at people working in computer vision research who study or build on vision foundation models.