github/scientist

★ 7,710RubyAudience · developerComplexity · 2/5LicenseSetup · easy

Mindmap

mindmap
  root((scientist))
    What it does
      Run old and new code together
      Compare results silently
      Record mismatches
    How it works
      Control block returns result
      Candidate runs in background
      Timing and exceptions captured
    Configuration
      Traffic percentage control
      Custom comparison logic
      Ignore known mismatches
    Use cases
      Safe code rewrites
      Production traffic testing
      Monitoring integrations
    Audience
      Ruby developers
      Backend engineers

mindmap root((scientist)) What it does Run old and new code together Compare results silently Record mismatches How it works Control block returns result Candidate runs in background Timing and exceptions captured Configuration Traffic percentage control Custom comparison logic Ignore known mismatches Use cases Safe code rewrites Production traffic testing Monitoring integrations Audience Ruby developers Backend engineers

Click or tap to explore — scroll the page freely

Things people build with this

USE CASE 1

Replace a slow database query with a new one and verify both return the same data under real traffic before switching over.

USE CASE 2

Rewrite an authentication method and confirm the new version matches the old one on a percentage of live requests.

USE CASE 3

Instrument two versions of a pricing calculation to catch any edge-case differences before fully deploying the new logic.

USE CASE 4

Gradually roll out a refactored API endpoint by running the new code on 10% of traffic and monitoring for mismatches.

Tech stack

Ruby

Getting it running

Difficulty · easy Time to first run · 30min

Requires adding the 'scientist' gem to your Gemfile and implementing a custom Publisher class to record results.

Open source under a permissive license, use freely in any project including commercial ones.

In plain English

Scientist is a Ruby library that helps developers safely replace or rewrite parts of their application without risking breakage for real users. The core idea is that you wrap your existing code in a "control" block and your new replacement code in a "candidate" block. When a user triggers that code path, the library runs the old code as normal and returns its result to the user, but it also quietly runs the new code in the background, compares the two results, and records whether they matched. This approach lets you test new code under real production conditions without affecting anyone's experience. If the new code produces different results, you find out through your monitoring and logging tools rather than through user complaints. The library records timing data for both paths, captures any exceptions that occur in the candidate block (without letting them reach users), and lets you publish the comparison results wherever you want, such as a metrics service or a database. You can control how often the candidate code actually runs, which is useful when starting out and wanting to test only a small percentage of traffic. You can also define rules for ignoring certain known mismatches, specify custom comparison logic when the default equality check is not appropriate, and clean up the recorded values to strip out details you do not need to store. The library is designed specifically for read-only code paths, meaning places in your application that look up and return data rather than writing or changing data. The README includes guidance on how to handle situations involving data writes safely. Scientist was created by GitHub and is available as a Ruby gem. Ports of the same concept exist in many other programming languages including Python, Java, Go, JavaScript, and others, each listed in the README.

Copy-paste prompts

Prompt 1

I have a Ruby method that queries a database. Show me how to use GitHub Scientist to run a new version of that method alongside the old one and log any differences.

Prompt 2

Using the Scientist gem, write a Ruby experiment that compares two implementations of a search function and publishes mismatches to a metrics service.

Prompt 3

How do I configure a Scientist experiment to only run the candidate code on 5% of traffic and ignore known differences in timestamp fields?

Prompt 4

Set up a Scientist experiment in Rails that wraps an existing service method, captures exceptions in the candidate block, and writes results to the Rails logger.

Prompt 5

Explain how to use Scientist's custom comparator to handle cases where two result objects are semantically equal but not identical in Ruby.

Open on GitHub → Explain another repo

← github on gitmyhub — every repo by this author, as a profile.

Verify against the repo before relying on details.