cchen156/learning-to-see-in-the-dark

This repository contains the code for a research project presented at CVPR 2018, a major academic conference on computer vision. The project is called "Learning to See in the Dark" and it addresses a specific photography problem: how to get a bright, clear image from a camera when the available light is extremely low, such as at night or in a dimly lit room. A conventional camera in low-light conditions produces a noisy, dark image because the sensor does not collect enough light. The typical approach is to raise the camera's ISO setting, but that amplifies noise along with the signal. This research takes a different approach: it trains a neural network, a type of machine learning system, on pairs of very short-exposure raw photos and the matching long-exposure photos of the same scene. The network learns to map the grainy, dark short-exposure image onto something that looks like the properly lit long-exposure version, without the photographer having to wait or use a tripod for a long exposure. The project works with raw sensor data rather than processed JPEG or PNG images. Raw data preserves more detail than a processed image and gives the network more to work with. Pretrained models are provided for photos taken with Sony and Fuji cameras, the README notes that these models are tuned to specific camera sensors and will likely not work well on photos from a different camera. Running the provided models requires Python 2.7, TensorFlow, and a library called Rawpy for reading raw image files. The dataset of paired images is available as downloads totaling around 25 GB for the Sony set and 52 GB for the Fuji set. Training the models from scratch requires a GPU and a large amount of RAM: at least 64 GB for the Sony model and 128 GB for the Fuji model. This is a research prototype, not a production tool. The README explicitly states it is intended to prove a concept rather than ship as a product. The code is released under the MIT License.