Automated traffic passed human traffic for the first time in 2024, reaching 51% of all web interactions according to the Imperva Bad Bot Report. Not all of that is agentic browsing, but the trajectory is clear enough: the non-human audience for your SaaS is already larger than the human one, and most SaaS products are completely invisible to it.
Your marketing site might rank on Google, and your docs might get cited by Perplexity now and then. But when ChatGPT Atlas browses the web to compare project management tools, or a Perplexity Comet agent researches CRM options across tabs, can it read your product page well enough to find the pricing and understand what you do? For most SaaS products the answer is no, and this guide walks through the technical changes that fix that.
How AI agents actually perceive your website
Before optimizing anything, you need to know what agents actually see when they visit, and it's probably not what you'd guess. There are three approaches, and they differ more than you'd expect.
The first is pure vision. Anthropic's Computer Use takes screenshots of the browser and decides what to click from pixel-level analysis, and Google's Project Mariner does something similar with its "observe-plan-act" loop. It works, but it's computationally expensive, and a layout change can throw the whole thing off.
OpenAI went a completely different direction with ChatGPT Atlas, which parses the accessibility tree. Their developer FAQ spells it out: Atlas reads ARIA tags, the same labels and roles that support screen readers, to interpret page structure. Microsoft's Playwright MCP takes the same approach and provides accessibility snapshots instead of screenshots. The accessibility tree strips away visual noise and keeps what actually matters, which for an agent means buttons, links and form fields, plus the headings and landmarks that give a page its shape.
Increasingly, the best agents combine the two. OpenAI's Computer-Using Agent layers screenshot analysis with DOM processing and accessibility tree parsing, and Perplexity Comet's research infrastructure uses "hybrid context management combining accessibility tree snapshots with selective vision."
| Platform | Primary Approach | What It Reads |
|---|---|---|
| ChatGPT Atlas | Accessibility tree | ARIA tags, roles, labels |
| Perplexity Comet | Hybrid | Accessibility tree + selective vision |
| Claude Computer Use | Vision | Screenshots, pixel analysis |
| Playwright MCP | Accessibility tree | Accessibility snapshots only |
| Google Mariner | Hybrid | Visual + code structure |
Even the platforms that started with vision are moving toward accessibility data, and the ones built for reliability lead with the accessibility tree. So I'd stop treating your website's accessibility tree as a compliance checkbox, because it's the primary interface AI agents use to discover and understand your product.
The llms.txt file: your product's AI-readable summary
The fastest win for AI agent discoverability is a file you can create in 10 minutes. llms.txt sits at your domain root and tells AI models which pages contain your most important content, a curated table of contents written specifically for machines. Where robots.txt is a gate (allow/deny), llms.txt is a map (here's what matters and why).
We've already covered the technical details of llms.txt configuration in our AI crawlers checklist. What matters here is the strategic angle for SaaS, and that mostly comes down to what goes in the file:
| Include | Why |
|---|---|
| Product overview | One paragraph explaining what your SaaS does, who it is for, and what problem it solves |
| Pricing page | AI agents comparing tools need pricing to make recommendations |
| Documentation root | API docs, getting started guides, integration pages |
| Changelog | Signals active development and the current feature set |
| Comparison pages | If you have "us vs competitor" pages, include them |
Does it actually work? Vercel reports that roughly 10% of their documentation traffic comes from AI platforms, and their llms.txt is one of the most complete implementations in the SaaS space: a single file that tells AI exactly what the product does, how it works and when to recommend it.
Where most people go wrong is the description itself. llms.txt works best when it's honest and specific, because an AI agent that recommends your product from an accurate description generates qualified leads, and an inflated description generates churn instead.
We'll show you exactly where your brand stands in AI search.
No commitment. $0 due today, cancel anytime. See how ChatGPT, Perplexity, Claude, Gemini, Grok, and Google AI Mode talk about your brand today.
Semantic HTML: the foundation agents read
None of this is new advice. Web standards advocates have been pushing semantic HTML for two decades, but the audience has expanded: it used to mean screen readers and a relatively small percentage of users, and now it means every AI agent that visits your site.
UC Berkeley and University of Michigan researchers tested Claude Sonnet on 60 real-world web tasks under different accessibility conditions, and the results were sharp:
| Condition | Task success rate |
|---|---|
| Standard conditions | 78% |
| Keyboard-only navigation | 42% (half the performance) |
| Magnified viewport | 28% (one-third the performance) |
That drop from 78% to 42% is the difference between an agent that can evaluate your product and one that gives up and recommends a competitor. The fix is straightforward, and it starts with native elements. A <button> element automatically appears in the accessibility tree with the role "button" and its text as the accessible name, while a <div onclick="doSomething()"> doesn't, so the agent has no idea it's clickable.
Forms need labels. Every input should have an associated <label>, because agents read labels to figure out what data a field expects, and autocomplete attributes with standard values tell them whether a field wants an email, a phone number or a street address.
Your heading hierarchy has to make sense too, with h1 through h6 in logical order, since agents use headings to understand page structure and locate specific sections. A pricing section buried under a generic <div class="pricing-wrapper"> is invisible to them. Put it under an <h2>Pricing</h2> and it's immediately findable.
The last piece is landmark regions. HTML5 landmarks (<nav>, <main>, <aside>, <footer>) tell agents where they are on the page, and a <nav> element is unambiguously navigation where a <div class="nav-wrapper"> requires guesswork.
One detail shows where the whole industry is heading: Microsoft's Playwright test agents generate test code that uses accessible selectors by default. When the AI writes a test, it writes page.getByRole('button', { name: 'Search' }), relying on accessible roles and names instead of CSS selectors or XPath.
Server-side rendering: the visibility requirement
This one catches a lot of SaaS founders off guard. Browser-based agents like ChatGPT Atlas and Perplexity Comet run on Chromium, so they execute JavaScript and can render your single-page application. AI crawlers don't.
GPTBot, ClaudeBot, PerplexityBot and GoogleOther index your content for retrieval and citation, and most of these crawlers don't execute client-side JavaScript. If your product's landing page is a blank <div id="root"></div> until React hydrates, they see nothing, and your entire marketing site might as well not exist. Content that isn't in the initial HTML doesn't get cited, and without citations, AI agents have no data to recommend your product from.
Frameworks like Next.js, Nuxt and Astro make SSR simple. If you're building content pages for AI visibility, treat server-side rendering as the baseline.
Watch what you hide behind interactions, too. Prices, feature comparisons and integration details tucked into accordions or "click to reveal" patterns create failure points for every agent that visits your page.
Know exactly what AI says about your competitors.
RankControl's Recon Agent monitors competitor citations across ChatGPT, Perplexity, Claude, Gemini, Grok, and Google AI Mode. See where they show up and you don't.
Testing which agents can actually see you
You wouldn't ship a website without testing it in a browser, and checking how agents perceive your site should become the same kind of habit.
The most direct tool is a Playwright MCP accessibility snapshot. Microsoft's Playwright MCP generates a structured view of any page showing the roles, names and states that agents actually work with, and if your critical interactive elements don't show up in it, or show up without useful names, agents will struggle. A snapshot of your pricing page should show something like this:
[heading level=1] Pricing
[heading level=2] Pro Plan
[text] $49/month
[button] Start free trial
If yours shows unlabeled divs and anonymous click handlers, you have work to do.
Screen reader testing is the best manual proxy for agent compatibility. If VoiceOver (macOS), NVDA (Windows) or TalkBack (Android) can navigate your product page, find the buttons, read the form labels and follow the content structure, agents can likely do the same, since both audiences read the same accessibility tree.
Your server logs are the third check. Look for GPTBot, ClaudeBot and PerplexityBot user-agent strings, and if pages that should be discoverable show zero AI bot traffic, something is blocking them. It's usually robots.txt, a WAF rule, or a Cloudflare setting you forgot about.
For a quick sanity check, right-click your page and choose "View Page Source" instead of opening DevTools. An empty <div id="root"> with nothing else in it is exactly what AI crawlers see too.
The real problem is knowing when it breaks
You could audit your site today: fix the accessibility tree, add llms.txt, make sure SSR is working and verify AI crawler access. That alone puts you ahead of most SaaS products. Agent discoverability doesn't stay fixed, though.
AI crawlers update their behavior. Your development team ships a new feature that breaks SSR on the pricing page, or a Cloudflare rule change blocks PerplexityBot, or a competitor publishes a better-structured comparison page and takes over the recommendation slot you held. Doing the audit once is the easy part, and the real problem is knowing when something stops working.
The initial audit runs 8-12 hours. It covers semantic HTML cleanup and SSR verification, llms.txt creation and a robots.txt review, and then testing. After that, maintenance takes 3-5 hours a month, spent monitoring crawler access, checking for regressions and updating llms.txt as features change.
You can do all of that manually. Or RankControl's agents can run the crawler access check daily and the citation checks weekly, and show you when an AI crawler stops reaching your site.
26 content formats. Published on your domain. Matched to your brand.
Guides, comparisons, listicles, case studies, and more. RankControl generates content that gets cited by ChatGPT, Perplexity, Claude, Gemini, Grok, and Google AI Mode.
