What this is about. I added a counter to my own analytics to find out how many robots visit my sites, and whether any of them would say who they were. Nearly every number it gave me was measuring something other than what I thought.

Why. Because a number you have not grouped by anything is not yet information. There is a difference between belief and knowing.

5 reads, 62 bots

The counter went live, I posted a new piece on LinkedIn and Facebook, and came back to "5 reads, 62 bots" a few minutes later. Sixty-two robots on one post felt like arrival.

It was not. The single biggest source was Facebook, fetching the page over and over from different addresses to build one preview card. LinkedIn did it twice more. Sharing a link looks almost exactly like being crawled, and nothing on my dashboard said so.

The actual AI crawlers had sent one request each. GPTBot, ClaudeBot, PerplexityBot and Common Crawl had each knocked once and left. That is discovery, not reading.

So the headline split in two: AI crawlers in one box, link previews in another. By the end of the day Facebook had fetched that single page from 130 different addresses, every one of them provably Facebook. I had assumed a preview card was one request.

21,791 visits, and 20,610 of them were me

The next morning the counter said 21,791.

I was briefly delighted. Then I grouped it, and found 20,709 visits from a client calling itself curl, hitting the homepage, from three addresses, none of which had an address at all.

Every one of my app containers has a health check. It runs curl against its own homepage every fifteen seconds to decide whether the container is still alive. Seven containers, four probes a minute each, is about forty thousand requests a day. My bot filter is deliberately broad, it matches the word curl, and so every single heartbeat was filed as an automated visitor.

Ninety-five percent of my bot traffic was my own server taking its own pulse.

The fix is one line. Working out which signal to trust took the rest of the morning. Not the user-agent, because anything can claim to be anything. Not the path, because real crawlers ask for the homepage too. The health check is distinguishable because it talks to the app directly and never passes through the web server in front, so it arrives with no client address at all. No address means it came from inside the house.

This can break, and I would rather it broke this way round. If the proxy ever stopped handing me the real client address, I would quietly undercount real robots instead of confidently counting imaginary ones.

Deleting them mattered more than I expected

Fixing the code only stops the bleeding, so I deleted the 20,787 heartbeats already recorded.

That sounds like tidying. It was not. The counts are also rolled up into a small permanent table, so that this October stays comparable with next October: the raw records are deliberately short-lived, the daily totals are not. Cleaning only the raw table would have left today's summary looking healthy while the permanent history went on insisting I had twenty thousand robot visitors on the day I shipped the counter. Wrong numbers that expire are a nuisance. Wrong numbers kept forever are a liability.

I nearly made it worse. My first instinct was to delete everything with no client address, which is the obvious signature. Checking the query before running it showed that would also have deleted perfectly good records for Googlebot, Bingbot and ClaudeBot, recorded before I started keeping origins at all. The predicate had to be narrower.

Pruned on both sides, the real figure for that first day is 914 visits from 26 bot families: far less impressive, and the first one I believe.

I was wrong about asking them who they are

The thing I actually wanted was an email address. If a robot is reading everything I write, I would quite like to know who to talk to.

Note: I do NOT track human users like this. Please read on!

I assumed this was hopeless. There is a header in the HTTP spec, From:, meant to carry the email address of whoever is responsible for an automated client, and my firm expectation was that nobody had used it since about 1996. I said so out loud, more than once, and logged it mostly as a joke.

Once the heartbeats were out of the way, 66 visits had filled it in.

googlebot(at)googlebot.com bingbot(at)microsoft.com gptbot(at)openai.com

Google, Microsoft and OpenAI all announce themselves, unprompted, in a header I had written off. Two smaller crawlers I did not recognise went further and gave real abuse-contact addresses. The obfuscated "(at)" is their own, presumably so the addresses survive being scraped out of server logs by somebody else's robot, which is a pleasing piece of symmetry.

So the answer to "can I ask a robot who runs it" turns out to be: mostly you do not have to ask. The well-behaved ones have been telling me all along and I was not reading it.

What happened, and what else I could have done

Three numbers were wrong in three different ways. The first counted link previews as readers. The second counted my own health checks as visitors. The third, the one I never even displayed, was sitting there full of answers because I had assumed it would be empty.

The fix for the big one was a single condition, and the alternatives were all worse:

I went hunting for robots and found a mirror. Before you trust a number, make it sit still and tell you what it is made of. Mine was 95 percent heartbeat.

Now comes the hard question: are you sure your (third party) tracker is made up of the same suspicios minds? Are you sure they filter out the noise or do they sell you snake oil? Give us a shout and we can check for you!

Next in this series: the most requested page on my server, which is a page that does not exist, and what I did to the robots asking for it.