el10savio/hntrends

hnTrends is a tool that looks through a big batch of old Hacker News posts and figures out what topics people were talking about, without anyone telling it what those topics are ahead of time. Instead of sorting posts into categories someone already defined, it reads the text of each post, turns it into a set of numbers that capture its meaning, and groups posts with similar meaning together. Those groups become the topics. Once it has done that work, it shows the results on a dashboard you can open in a browser. The dashboard has a chart of how big each topic group is over time, a ranked list of the biggest topic groups, and a way to click into any group to see the actual Hacker News posts inside it. You can also switch between different time windows, from the last day up to the last five years, to see how topics have shifted. Under the hood it stores everything in DuckDB, a lightweight database file, and can either keep that file on your own computer or use a cloud service called MotherDuck if you want the heavy processing to happen remotely instead. Running it locally needs Docker installed. One part of the setup loads and processes the data, and once that finishes, a second part starts up the dashboard so you can explore it. If you choose the MotherDuck option instead, you need to sign up for a MotherDuck account, get an access token, and put it in a configuration file, then flip a setting to tell the tool to use the cloud version rather than the local one. The dashboard you see afterward looks the same either way. The project describes itself as a way to explore and report on trends, rather than a finely tuned system for deep topic analysis. It documents which techniques it borrows from, such as sentence embeddings and a grouping method called K-Means, and points to further reading for each one. The full README is longer than what was shown.