Hacker News AI Detector

Gallery

About

For the past few months, I've been sending every story that reaches the HN front page through Pangram's AI detector and noting the scores. The reason I made it is that I kept noticing posts that sounded like AI-generated to me. Sometimes the comments pointed it out, sometimes a post was near the top all day and no one said a word. I wondered if the share was really increasing or if I was just primed to see slop everywhere.How it operates: The site identifies the top 30, gets each article, and evaluates the text using Pangram. Besides the live front page view, there are daily and monthly trends, with leaderboards for domains and authors that keep scoring high. I personally find it quite helpful as a sanity check on my own intuition before declaring something AI in the comments.6,500+ stories in, it turns out the front page is still very human after all, which is the part that shocked me the most. A fraction of 13% of stories were AI flagged over the last 30 days. That's slowly ramping up (roughly a point higher than the previous month) but still not close to the feeling of 'everything is slop now' I had. At the moment it's 1 flagged story out of 30.The flagged ones still manage to get by quite well. One of them is at #6 today with 545 points and a 99% score, and the discussion section is divided on whether it's AI or not.Standard disclaimers: a detector score is not a piece of evidence, false positives do occur, and a high score can also be the result of a human coming up with the ideas and then using a model for the prose. To me, it is an invitation to examine the matter more closely rather than a judgment.Feel free to ask me anything about the setup. And yes, I ran this post through the detector before submitting it.