smicallef/spiderfoot

★ 17,744PythonAudience · ops devopsComplexity · 3/5LicenseSetup · moderate

Mindmap

mindmap
  root((repo))
    What it does
      OSINT automation
      200 plus data sources
      Chain scanning
    Inputs
      IP address
      Domain name
      Email address
    Outputs
      Subdomains found
      Breach hits
      Social accounts
    Tech
      Python
      SQLite storage
      Web UI and CLI

mindmap root((repo)) What it does OSINT automation 200 plus data sources Chain scanning Inputs IP address Domain name Email address Outputs Subdomains found Breach hits Social accounts Tech Python SQLite storage Web UI and CLI

Click or tap to explore — scroll the page freely

Things people build with this

USE CASE 1

Map your organization's internet footprint to find exposed subdomains, email addresses, and cloud storage buckets before attackers do.

USE CASE 2

Run authorized penetration test reconnaissance to gather intelligence on a target domain, IP address, or email.

USE CASE 3

Detect if your company's credentials appear in data breach databases by starting a scan from a known email address.

USE CASE 4

Build custom correlation rules in YAML to automatically flag suspicious patterns across collected intelligence data.

Tech stack

PythonSQLiteYAML

Getting it running

Difficulty · moderate Time to first run · 30min

Some modules require API keys for third-party data sources, core scanning works without them.

Use freely for any purpose, including commercial use, as long as you keep the copyright notice.

In plain English

SpiderFoot is an open-source OSINT (open-source intelligence) automation tool, OSINT means gathering information from publicly available sources. It is designed to help security professionals and organizations understand what information about them is exposed on the internet, or to gather intelligence on a target during a penetration test or security assessment. The way it works is by taking a starting point, which can be an IP address, domain name, email address, phone number, username, person's name, Bitcoin address, or network range, and then running it through over 200 modules that each query different data sources. These modules feed their results back into each other in a chain: discovering a domain might reveal email addresses, which then get checked in data breach databases, which might reveal associated usernames, which then get searched on social media platforms. The tool extracts hosts, sub-domains, email addresses, social media accounts, files exposed in cloud storage buckets, threat intelligence blacklist hits, and more. It provides both a web-based interface (launched by running a local web server) and a command-line mode. Results can be exported in CSV, JSON, or GEXF formats and are stored in a local SQLite database for custom querying. A YAML-based correlation engine lets you define rules to automatically detect patterns of interest across the collected data. It is written in Python 3 and licensed under MIT. You would use it when you need to map out your organization's internet footprint, investigate potential threats, or gather reconnaissance data during authorized security testing. The full README is longer than what was provided.

Copy-paste prompts

Prompt 1

Using SpiderFoot, how do I scan my company domain to find all exposed subdomains and email addresses? Give me the CLI commands and what modules to enable.

Prompt 2

Write a SpiderFoot YAML correlation rule that flags when an IP address appears on a threat intelligence blacklist and is also associated with my target domain.

Prompt 3

I ran a SpiderFoot scan and exported results to SQLite. Write a Python script to query the database and list all discovered email addresses with their associated data breach hits.

Prompt 4

How do I run SpiderFoot in command-line mode instead of the web UI and export results as JSON for further analysis?

Open on GitHub → Explain another repo

← smicallef on gitmyhub — every repo by this author, as a profile.

Verify against the repo before relying on details.