im-sanjay-sai/cua_mr_brain

This is a Python desktop application that helps users visually locate medical terms from a written report on a set of medical images. You load a folder of images, paste in a report or diagnosis text, and the app uses AI to find which images show the regions mentioned in that text, then draws boxes or markers directly on the matching images. The process runs in two stages. First, an AI model reads the pasted report and pulls out key terms, the words that describe regions to look for. You can choose between two providers for this step: OpenAI's model or the Lightcone/Tzafon service. Second, the Lightcone/Tzafon localization model scans the images for each extracted term and returns coordinates indicating where that region appears. The app converts those coordinates from a fixed 0-to-999 grid into actual pixel positions, then draws the annotation on the image. Each image gets a stable study ID, and the app tracks which terms were found and which were not. If a term is not visible in an image, that image is left unmarked and the result is logged as "No region." The app ranks the best matching image for each term using a score that weighs model confidence heavily, with box size as a supporting signal. The README notes this is a visual localization prototype and is not a diagnostic medical device. It should not be used for clinical diagnosis or treatment decisions. The tech stack is Python, with the OpenAI API and the Lightcone/Tzafon API providing the two AI stages.