rafaelpta/dupehound

Dupehound is a command-line tool that finds duplicate functions in a codebase. It was built specifically for projects where AI coding assistants write most of the code, since those tools tend to re-implement things that already exist rather than reuse them. The tool compares functions by their structure rather than their text, so it catches copies even when variable names, string values, or numbers have been changed. The tool has three commands. "scan" goes through a directory, groups duplicate functions into clusters, and produces a "slop score": the percentage of code that could be deleted if each cluster were consolidated into one copy. "history" reads past git commits and draws a chart showing how duplication changed over time, without needing to check out each version. "check" is designed for automated build pipelines: it compares newly added or changed functions against the rest of the codebase and stops the build if a duplicate is found, naming the original function the developer should reuse instead. Under the hood, function bodies are parsed with a tool called tree-sitter, which understands the grammar of each supported language. After parsing, identifiers, strings, and numbers are replaced with placeholder tokens so that only the structural skeleton remains. That skeleton is broken into overlapping chunks and fingerprinted. Two functions match when enough fingerprints overlap. The approach is deterministic: given the same code, it always produces the same result, which matters when the output is a CI gate. Everything runs locally. There is no network connection, no API key, and no machine learning model involved. Prebuilt binaries are available for macOS, Linux, and Windows, and the tool can also be installed through Rust's package manager or Homebrew. Supported languages include TypeScript, JavaScript, Python, Rust, Go, Java, Ruby, and Swift. On a very large codebase like VS Code, with roughly 3 million lines and 53,000 functions, a full scan takes about 3.6 seconds on a laptop. Test files are excluded from duplicate checks by default, since table-driven tests are intentionally repetitive. Generated, minified, and vendored files are also skipped. Any match can be inspected with the "--explain" flag, which prints the actual code side by side as confirmation.