AI Visibility Rankings Are Mostly Statistical Noise — New Research Shows Why

If someone shows you a dashboard with your AI citation “rank” and tells you the number is reliable, that dashboard is overselling itself.

New research from IQRush, shared with Search Engine Journal last week, shows that AI citation numbers bounce around from one round of queries to the next. The researchers ran the same questions over and over across SearchGPT, Gemini, and Perplexity. They found that rankings often don’t settle down until you’ve collected 33 to 94 usable answers per topic. Three topics out of thirty never settled at all — all on SearchGPT.

That matters because most AI visibility tools on the market today report a single number as if it were a fact. A three-point difference between you and a competitor might be real. Or it might just be the normal variation between two measurements of a system that is designed to give different answers every time you ask.

How Much Data Is Enough for a Reliable Answer?

The paper’s central finding: there is no single number. It depends on the platform and the topic you are measuring.

Gemini tends to mention the same handful of sites many times within a single answer, so you need more query rounds to collect enough independent data. SearchGPT gives fewer citations per answer but spreads them across more sources — each round carries more independent information — yet some SearchGPT topics never sorted themselves out even after 125 queries.

The practical takeaway: a measurement plan that works on one platform may not work on another. Any visibility check should be built around the platform you care about most.

What This Means for Your Business

Most small businesses are not buying AI visibility dashboards yet. But if you are — or if someone offers you one — here is what to ask:

• Do you run the same query multiple times, or just once? A single measurement is not a measurement.
• Do you report a range or a margin of error alongside the number? A clean number is a warning sign, not a good sign.
• Do you have a way of knowing when you have collected enough data? Guessing is not a method.

For everyone else, the insight is simpler: do not chase small changes. If someone tells you your AI visibility “went up by 4 points” after you updated your website, that gain could easily be normal fluctuation. Run the check again before you declare victory.

What’s Actually Worth Watching

The parts of a ranking worth trusting are the top spots. With enough data, the leaders pull away from the middle and the tail. Past the front of the list, the margin of error widens fast — neighboring positions become a coin flip.

This fits a pattern that shows up across a lot of what gets written about AI search. As we covered in a recent post, AI search rewards the same things that have always worked: a real reputation, real reviews, and real word of mouth. Those signals hold up across 125 query rounds. A content tweak that moves you from position 7 to position 5 on a single batch run is noise until you can prove otherwise.

The same thing applies to where your citations actually come from. Most AI citations point to third-party pages, not your own website. A bump in your citation share that comes from a single measurement could just be random variation on someone else’s blog — not a sign that your own content strategy is working.

The research paper is a preprint, not a finished peer-reviewed study. It is based on 30 platform-topic tests over a single week. The exact numbers will not transfer cleanly to your business or your city. But the shape of the problem is real: AI visibility is a moving target, and treating a single snapshot as a fact will lead you in the wrong direction.