ashuigordon/spider-x

Spider-X is a tool that helps you collect posts and comments from X.com (formerly Twitter) based on keywords you care about. Instead of manually scrolling through the platform and copy-pasting tweets, you type in a search term and the tool automatically gathers matching posts along with their replies, saving everything into a structured file you can read or analyze later. Under the hood, it works by automating a web browser. You give it access to your X.com account through a cookie string, essentially a snapshot of your logged-in browser session that tells the site "this is me." The tool then opens Chrome behind the scenes, searches for your keywords, collects the URLs of matching tweets, visits each one, and extracts the post text and comments. Everything gets saved as JSON files in an output folder, which is a simple format that both humans and other programs can easily read. This would be useful for marketers tracking brand mentions, researchers studying conversation trends around a topic, or founders monitoring discussions about their product or industry. If you wanted to see every tweet mentioning "AI startups" this week along with what people replied, you could run this tool and get a clean dataset instead of scrolling for hours. Setting it up requires some technical steps, you need Python installed, a Chrome browser, and you have to manually extract your cookie string from your browser's developer tools. The README doesn't go into detail on rate limiting or how to handle large searches, so you may need to adjust timing settings in the code to avoid hitting X.com's limits. The tool also notes that you should follow X.com's terms of service and keep your cookie string private since it essentially contains your login credentials.