kepengxu/prism-vl

PRSIMVL is an AI research project that asks whether vision-language models (AI systems that can answer questions about images) perform better when given raw camera sensor data instead of the standard processed photographs we normally see. The research paper behind it is titled "Allegory of the Cave: Measurement-Grounded Vision-Language Learning." The core idea is that the photos on our phones and computers have already been processed by camera software: the sensor's raw readings go through color correction, noise reduction, tone mapping, and compression before becoming the RGB image you see. That processing often removes or distorts information that could help an AI reason about the image, especially in difficult conditions like low light or bright highlights. PRSIMVL instead feeds the model a representation derived directly from the raw sensor readings, called Meas.-XYZ, along with camera settings like ISO, shutter speed, and aperture. The results show this helps the model give better answers in cases where standard photos mislead it. In one example, a model reading a standard photo of an illuminated shop sign gets the name wrong, the same model given the raw-derived input gets it right. The research team built and released several components alongside the code. MeasL-Bench-V1 is a benchmark dataset of 2,183 test examples for evaluating how well models handle measurement-sensitive tasks. MeasL-150K-V1 is a training dataset of about 152,000 instruction-following examples used to teach the model. The models themselves are based on Qwen3-VL, an existing open vision-language model, fine-tuned using LoRA (a technique that adapts a model without retraining all its weights). Adapters for 2-billion, 4-billion, and 8-billion parameter model sizes are available on Hugging Face. The headline result is that the 8-billion parameter PRSIMVL model outperforms the same base model trained on standard photos by about 4.5 points on an AI-judged accuracy test and by significant margins on text-matching metrics, particularly on tasks involving low-light scenes, high-contrast scenes, and reading text in images. The repository includes scripts for running inference on a single image, running the full benchmark evaluation, and inspecting the training and evaluation datasets. A service mode uses a deploy command to start a local API that you can query with images. The full README is longer than what was shown.