aiscientists-dev/caliper

Caliper is an AI research assistant designed to run scientific data analysis pipelines from end to end. What sets it apart from other AI analysis tools is that it reports a calibrated confidence level for every result and automatically escalates to a human reviewer when it is not confident enough to answer on its own. The project is built around three layers. The first is a Domain Pack, a small registry of vetted tools specific to a scientific field. It ships with a working genomics pack and a skeleton astronomy pack to show the design is not field-specific. The second layer is an Agent Core that plans a sequence of analysis steps, executes them, and keeps a reproducible record of what ran. The third is a Trust and Feedback layer that assigns a confidence estimate to each result, enforces a user-defined error ceiling (for example, no more than 1 in 10 unchecked answers should be wrong), and re-calibrates itself each time a human corrects it. The deployment model keeps research data on the user's own infrastructure. A small control server hosts the web interface and the agent, while the actual computation runs on a private server where the raw data already lives. Only small results and logs travel back to the control host, raw data does not leave the private server. Users can set a plain-English reliability rule, and Caliper learns from a set of expert-reviewed examples how confident it needs to be before answering without supervision. Anything below that confidence threshold gets escalated. The guarantee holds even when the confidence scorer itself is imperfect. Caliper is currently a research preview. The genomics domain pack is working and tested. Planned next steps include a stricter calibration method, a reproduced published genomics study, and a complete astronomy pack. The project is released under the Apache 2.0 license and requires Python 3.9 or newer.