In September, a marketer posted in r/DigitalMarketing that every client call now ends with the same question: why did our "AI ranking" drop? They'd gone digging for an answer and come back unsure whether the numbers meant anything at all.
Is anyone else's "AI visibility" dashboard just... making numbers up?
Been doing SEO for a few years now and lately every client call ends with someone asking why their "AI ranking" dropped from a tool we're paying for. So I went down a rabbit hole trying to figure out if these numbers mean anything, and hone...
We ran into the same doubt from the other side. On September 26 we asked OpenAI's API five questions a help desk buyer might ask, three times each, and counted which of 13 help desk brands each answer named. HubSpot was named in four of the five answers in the first run, one in the second and four again in the third. Anyone who checked once, on the wrong day, would have reported a collapse that never happened.
In those same three runs, Help Scout was named in four of five answers every time.
So is AI visibility worth tracking yet? We think it is, with a narrow definition of tracking. Some of the numbers hold still when you rerun the questions, and those are worth watching. The rest is noise, and a lot of what gets reported as AI visibility sits in that second group. This piece covers both, along with who should wait. Sources were checked on September 27, 2026.
The case against tracking is a fair one
The skeptics have good points, and we'd rather deal with them than wave them away.
Start with traffic, because it's small. A benchmark covering 3.3 billion sessions across 10 industries, reported by Search Engine Land in November 2025, found AI referrals made up just over 1% of website visits:
![]()
ChatGPT sent 87.4% of those AI referrals, and organic search still brought in far more visits in every industry. If you judge a channel by the clicks it sends, AI search is still a tiny channel for most sites.
The answers also move. SparkToro's January study found the same prompt returned the same list of brands less than once in 100 tries, and the same order less than once in 1,000. A dashboard that says you're "ranked #3 in ChatGPT" is reporting one draw from a very wide spread of possible answers.
Then there's volume. Google will tell you roughly how many people search a keyword each month. No AI company publishes how often people ask a question, so every tracked prompt is an educated guess about what buyers type.
The argument is loud enough that Lily Ray described her feed in July as split down the middle:
My feed is half people talking about tracking more prompts & all the right prompts and half people saying prompt tracking is fundamentally useless
Lily Ray 😏@lilyraynycJul 28, 2026We don't think either half has it right. Adding more prompts won't rescue a metric that measures noise, and calling all tracking useless ignores the numbers that turn out to be stable.
What our reruns showed
Here's the per-run count for six of the 13 brands we tracked, out of five answers per run:
| Brand | Run 1 | Run 2 | Run 3 |
|---|---|---|---|
| Freshdesk | 4 | 5 | 5 |
| Help Scout | 4 | 4 | 4 |
| Zendesk | 4 | 4 | 5 |
| Zoho Desk | 3 | 4 | 4 |
| HubSpot | 4 | 1 | 4 |
| Crisp | 3 | 1 | 2 |
Zendesk's count includes the question asking for Zendesk alternatives, which names it by definition.
Across all 65 brand-question pairs, 48 came out the same in all three runs. In 25 of them the brand appeared every time, and in 23 it never appeared. That's 74% agreement. The other 17 pairs flipped at least once, and 14 of those flips belonged to the nine smaller brands. The four most-named brands flipped only three times between them.
The question mattered as much as the brand. The broadest one, "What's the most affordable help desk software with live chat?", produced six of the 17 flips on its own, and its first recommendation changed between runs. For three of the other four questions, the first product recommended was identical in all three runs.
Order was the least stable thing we measured.
The top four brands never came back in the same order across all three runs, for any of the five questions.
Engines disagreed more than runs did. For the best help desk for a small team, OpenAI's API named Freshdesk first in all three runs, while Perplexity's API named Help Scout first in both of its runs. For the best shared inbox, OpenAI led with Help Scout every time, and Perplexity led with Front both times.
We call the numbers that barely move the stable signal. For an established brand, it's how often you're named across the whole question set and which competitors keep appearing next to you. In our runs, the differences between engines were just as consistent. Your position inside a single answer isn't part of it, and neither is any one question on any one day.
Five questions on one topic is a small test, so treat our percentages as an illustration. The pattern still matches the larger studies: presence across many runs holds up, and ranking position doesn't.
15 hours a week manually. Or 15 minutes with RankControl.
Track citations, monitor competitors, and fix content gaps across every AI search engine. Automatically.

Why we still think it's worth tracking
The 1% figure measures clicks, and most AI influence never produces one. In June, when Similarweb and Rand Fishkin studied what happened after ChatGPT recommended a brand, most of the visits that followed arrived through a search engine. A referral report misses every one of those visits.
Buyer choice follows visibility, too. In July, Kevin Indig shared a study of how people pick products after reading ChatGPT's answers. The brands participants chose had a 24% share of voice in those answers, against 11% for the brands they passed over. That's the closest thing we've seen to evidence that being named changes what people pick.
The trend points the same way. In the benchmark above, AI referrals were growing about 1% month over month. IT companies already got 2.8% of their traffic from AI, more than twice the average across industries. If you sell software, that IT figure is the closer benchmark.
Then there's the baseline problem, which is the argument we find most convincing.
You can't go back and measure last quarter. If AI answers start recommending a new competitor in your category next spring, a team with six months of weekly data will see when it started and which sources changed. A team that starts tracking the week it notices has nothing to compare against.
The cost is modest. A small question set checked monthly takes a few hours, which is about what one solid blog post takes to write and edit.
What's worth tracking and what isn't
Here's how we'd split it, based on what held still in our runs and what didn't:
| Worth tracking | Skip or treat with caution |
|---|---|
| Mention rate across a fixed question set | Your position inside a single answer |
| Citation rate, and which of your pages get cited | Screenshots from a single run |
| Competitors that appear alongside you | Daily checks |
| Sources the engines cite for your category | Question sets padded with your brand name |
| Whether answers describe your product correctly | A composite score with no breakdown |
| AI referral visits and branded search trends | Estimated prompt volumes |
Branded questions deserve a closer look. A question that includes your brand name will mention you almost every time. A set weighted toward those questions makes visibility look great and tells you nothing about buyers who haven't heard of you yet. Keep branded questions to one or two per set, and use them to check accuracy.
Our comparison of mention tracking and citation tracking explains why those two numbers move separately. Our AI visibility dashboard guide shows how to lay them out in a monthly report.
The practitioners in that r/DigitalMarketing thread landed in a similar place. One commenter said the approach that held up for them was tracking the share of mentions and citations across a fixed prompt set, then reading the trend over time instead of positions.
AI search traffic grew 835% this year. Is your content ready?
RankControl generates 26 content formats optimized for ChatGPT, Claude, and Perplexity. Published on your domain, matched to your brand.

Who should wait
Tracking pays off only when you can act on what it shows. Some teams can't yet, and for them we'd hold off.
If you have no capacity to publish, start there. The first thing a question set tells many brands is that they're missing from questions they should own, and the fix is publishing pages that answer those questions directly. Without anyone to write those pages, a monthly gap report just becomes a monthly reminder.
If your buyers don't research purchases with AI assistants, the channel matters less for now. Many local service businesses get most of their leads from maps and word of mouth, and a quick check of a few buying questions twice a year is enough there.
And if the only reason to track is to hand someone a number, be careful.
A single score will swing from week to week, and each swing means more time spent explaining noise. That's roughly where the r/DigitalMarketing poster found themselves.
Everyone else should start now with a small set, especially B2B software companies in a defined category where buyers compare named competitors.
How to start small
Pick 20 to 30 questions buyers ask before they know your name. Mix category questions with comparisons that name competitors. Add at least one price question, then a single branded question to check accuracy. Keep the list fixed, so month-over-month changes come from the answers.
Run each question at least twice on at least two engines. For every answer, log whether you're named, whether you're cited, which competitors appear and which sources are linked. Our guide on how to check your AI visibility walks through a full check with a worked example.
Then add the free first-party data. Search Console now has a generative AI report. It doesn't show prompts or clicks, but Aleyda Solis made a good case in July for using it as a baseline anyway:
I finally got access to the new Search Console Generative AI performance report… and: it's easy to feel underwhelmed, especially when compared to the one of Bing or Clarity AI Search dashboards 😅 It doesn't feature prompts, topics or clicks, just impressions and URLs with AI https://t.co/xVFJgUgROS
Aleyda Solis 🕊️@aleydaJul 17, 2026Bing Webmaster Tools reports how often your pages are cited in Copilot and Bing's AI answers, and since June it has been adding citation share for each query. Add AI referral segments in Google Analytics 4, which our GA4 guide covers, plus a free-text "How did you hear about us?" field on demo forms. A commenter in the r/DigitalMarketing thread recommended that last one, and it catches buyers who asked an assistant and then searched your name.
Review everything monthly. Compare the totals across the whole set with last month's, and only dig into a single question when the change shows up in every run.
When a total does move in two checks in a row, look at the cited sources before you touch any pages. Compare them with last month's list and look for what's new, such as a fresh comparison page or a competitor's rebuilt review profile. In our experience that's a faster route to the cause than rewriting your own pages on a hunch.
If someone asks you to justify the hours, our piece on justifying time spent on AI search visibility covers that conversation.
Our answer
Is AI visibility worth tracking yet? Yes, if you track the stable signal and ignore the rest. It's still a small traffic channel, and it may stay one, because much of its effect shows up as branded searches and shortlists that referral reports never see.
Done by hand, checking 50 questions on seven engines takes about six hours a week, and publishing eight solid articles a month to close the gaps takes 16 to 24 hours more. Those are rough estimates.
RankControl handles both jobs. AI visibility tracking runs your 50 questions every week on seven engines, marks each one as mentioned, cited or both, and records which competitors and sources each engine used. The content engine writes each article against the pages already ranking for its keyword and publishes it on your own domain, so the gaps the tracking finds get pages.
You can start with a spreadsheet and 20 questions this week. Or RankControl's agents can run the checks every week while your team decides what to publish.

Built by the team that got cited in 48 hours.
Content generation, backlink building, AI visibility tracking, and Google rankings. One platform, zero guesswork.








