SwarmSystem Field Guide No. 1 · Technical SEO

What the bots actually see

Your SEO tool reports the title tag is fixed. Your browser agrees. Then you look at the raw HTML the server sent and it says something different. Both are real. Here is which machines read which version, and which one decides whether a homeowner ever finds you.

Published September 10, 2026. Contains an original measurement taken the same day.

The short version
If you read nothing else

Google runs your JavaScript. It loads your page the way a browser does, so it usually ends up seeing the version a person sees.

The AI assistants do not. ChatGPT, Claude, and Perplexity read the raw HTML your server sends and stop there. Whatever a script was going to add never gets added.

So a page can rank fine on Google and be invisible to every AI assistant at the same time. That is not a theory. We measured it on a live site this morning and the numbers are further down this page.

The best case has not changed in twenty years. Put the words in the HTML the server sends. Everything else is a fallback.

The rest of this guide is for whoever touches the code. The four paragraphs above are the whole answer for everyone else.
01 · The mechanism

How a page actually gets read

A search engine does not read your page in one motion. Google splits it into three jobs and they happen at different times.

First it crawls. It asks your server for the URL and gets back a block of HTML. That is the only thing that exists at this point. If your content is not in that block, it does not exist yet.

Then it queues the page for rendering. Google's own documentation is blunt about this: "Googlebot queues all pages with a 200 HTTP status code for rendering." The page sits in line until Google has spare capacity.

Then it renders. A headless Chromium browser opens the page, runs the JavaScript, and produces a finished DOM. Google indexes that finished version, not the raw one.

0sHTML fetched. This is all a non rendering bot ever gets.
10sMedian wait. 25% of pages clear in under 4 seconds.
~3hNinetieth percentile. The long tail is real but rare.
100%Of pages studied got a full render, JS interactions included.

Timings from the Vercel and MERJ study of more than 100,000 Googlebot fetches, with 37,000 rendered page and beacon pairs matched on nextjs.org across April 2024. Source 2 below.

That study settled an old argument. For years the working model was "two waves of indexing," where Google indexed your HTML first and came back for the JavaScript days later, if ever. The data says the wait is usually seconds, not days, and that content loaded asynchronously through API calls got indexed fine.

If Google were the only machine that mattered, client side rendering would be a solved problem. Google is no longer the only machine that matters.

02 · The two models

Server side and client side, side by side

Both models end up showing a person the same page. They differ in who does the assembly and when.

Server side

The server builds the finished HTML and sends it. The browser gets a complete page in the first response. Includes plain static HTML, static site generators, and frameworks running in SSR mode.

  • Every machine sees the content, including ones that cannot run code
  • Content appears on the first paint, so the page feels fast on a bad phone
  • Schema, title, meta, and headings are all in the source where any parser finds them
  • Costs more server work per request, and caching gets more complicated
  • Highly interactive apps need extra plumbing to stay interactive
Default choice for anything you want found
Client side

The server sends a near empty shell plus a bundle of JavaScript. The browser downloads the code, runs it, fetches the data, and builds the page in place.

  • Cheap to host, since the server hands out the same static shell every time
  • Navigation between views feels instant once the app has loaded
  • Suits dashboards, editors, and anything behind a login where search does not apply
  • Any machine that does not run JavaScript sees the empty shell
  • Slow phones and flaky connections pay for the bundle before they see words
Invisible to everything that will not run code

There is a middle ground and it is where most good sites live. Static generation at build time, prerendering, and hydration all put real HTML in the first response and then attach JavaScript on top for interactivity. Ahrefs puts it plainly: "Any kind of SSR, static rendering, and prerendering setup is going to be fine for search engines."

03 · The split

Who runs JavaScript and who does not

This is the table that matters, and it is the part most people still have wrong.

MachineRuns JavaScriptWhat that means for you
GooglebotYesRenders in headless Chromium. Sees the JavaScript version, after a wait.
Google GeminiYesRuns on Googlebot's rendering infrastructure, so it inherits the same ability.
AppleBotYesRenders. Feeds Siri and Spotlight suggestions.
BingbotPartialRenders some JavaScript, not all frameworks, and not reliably at scale. Microsoft recommends serving pre-rendered HTML.
GPTBot (OpenAI)NoRequested JavaScript files in 11.50% of its fetches and executed none of them.
ClaudeBot (Anthropic)NoRequested JavaScript files in 23.84% of its fetches and executed none of them.
PerplexityBotNoReads the raw HTML response and stops.
OAI-SearchBot, ChatGPT-UserNoThese are the live fetchers, the ones that pull your page while a person is waiting on an answer. They do not render either.
Meta-ExternalAgent, BytespiderNoRaw HTML only.
Facebook, LinkedIn, Slack unfurlersNoThey read meta tags out of the source. A JavaScript written og:image never makes the preview card.
The results consistently show that none of the major AI crawlers currently render JavaScript. Vercel and MERJ, The rise of the AI crawler, December 2024

That study was published in December 2024 and it is still the most rigorous public measurement of the question. Write ups since then report the same result for GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot, Meta-ExternalAgent, and Bytespider, though we have not seen a second study run at that scale, so treat the recent end of this as well supported rather than proven. Gemini remains the one real exception, and only because it borrows Google's renderer.

The uncertainty does not change what you should do. If these crawlers start rendering JavaScript next year, a site that already ships its content in the HTML loses nothing. The recommendation is the same either way, which is a good sign that it is the right one.

Worth noticing in that table: the live fetchers do not render either. People assume the bot that grabs your page in the middle of a conversation must be doing something smarter than the training crawler. It is not. It requests the URL, reads what comes back, and answers from that.

04 · The measurement

We measured a live site this morning

Plenty of guides will tell you client side injection is risky. We wanted a number, so we took one.

The subject is a working home service company in Texas. Its site runs a mainstream SEO automation tool, the kind that installs one script tag and then improves your on page SEO from a dashboard. That category is popular for a good reason. It is genuinely easier than editing templates. We are not naming the vendor, because the mechanism is the point and most of the tools in that category work the same way.

We requested one service page twice. Once with curl and a browser user agent, which gives you the raw response and runs nothing. Once with headless Chromium, six second settle, which gives you what a browser ends up with.

What we looked atRaw server HTMLAfter JavaScript ran
Structured data blocks
The JSON that tells a machine what this business does
1 block
LocalBusiness only
4 blocks
Service, two ImageObject sets, LocalBusiness
Meta description "Protect your yard with expert mosquito control in San Antonio. Safe, effective treatments to reduce mosquitoes and enjoy your outdoor space again." "Keep mosquitoes under control with professional mosquito control in South Texas for your yard and outdoor spaces. Schedule service today."
Title tag Identical on this page. On four other pages of the same site, checked a week earlier, the rendered title was a different city than the raw one. Identical here.
The optimizer script Present, as a script tag. A tag is not its output. Executed. Everything above happened here.

Three of the four structured data blocks exist only after JavaScript runs. One of the missing three is the Service schema. That is the block that says what this company sells and where it sells it, which is the exact question an AI assistant is trying to answer when a homeowner asks it for a mosquito guy in San Antonio.

The rewritten meta description has the same problem. Someone paid for that copy and it is doing real work in Google. Every AI crawler reads the old one.

Nothing here is broken. The tool did exactly what it promises and Google sees all of it. The gap is between what Google sees and what everything else sees, and that gap did not matter much in 2019.

Why this is not a small gap anymore

Cloudflare measured crawl to referral ratios across its network in August 2025. For every visitor Anthropic's crawler sent back to a site, it had requested close to 50,000 pages. OpenAI's ratio was 887 to 1. Perplexity's was 118 to 1. In news and publishing the ratios were much lower, 2,500 to 1 and 152 to 1, because those sites actually get cited.

You can read that as a bad trade, and plenty of publishers do. Read it the other way and it says these machines are working through an enormous amount of the web and forming an opinion about who does what. The same study found about 80% of that crawling was for training rather than answering someone's live question, which matters here: training is how a model comes to know your company exists at all, and it is the pass that reads raw HTML.

Source 7 below. The ratios move over time and by industry. The order of magnitude does not.
05 · The dead end

Do not reach for dynamic rendering

There is an obvious looking fix here. Detect the bots, serve them a pre-rendered copy, serve everyone else the JavaScript app. It has a name, dynamic rendering, and for a few years Google suggested it.

Google changed that documentation in August 2022. The current wording is that dynamic rendering is a workaround and not a long term solution, and the recommendation is server side rendering, static rendering, or hydration instead.

The practical objection is better than the official one. You now maintain two versions of every page and only ever look at one of them. When they drift apart, and they do, the version you never look at is the one the search engines have. Ahrefs makes the same point: it "makes setups more complex and harder for SEOs to troubleshoot."

You also have to keep a list of which user agents get the good copy. That list was wrong the moment GPTBot existed, and it will be wrong again the next time somebody launches a crawler.

06 · The practice

Best to worst, ranked

One rule sits above all of these. Everything a machine needs to understand the page belongs in the first HTTP response. That means the title, the meta description, the H1, the body copy, the internal links, the canonical, and the schema. If a script adds it later, treat it as a bonus, not as the thing.

  1. Static HTML, built ahead of time

    The server hands over a finished file. Fastest for people, readable by every machine ever written, and nothing can fail at request time. This is what a service page or a location page should be.

    All 10 machines above
  2. Server side rendering

    The server builds the page per request. Right answer when content changes often or depends on who is asking. Next.js, Nuxt, SvelteKit, Rails, Laravel, and WordPress all do this natively.

    All 10 machines above
  3. Static generation with hydration

    Real HTML ships first, then JavaScript attaches to make it interactive. You get the crawl safety of static and the behavior of an app. This covers most modern framework sites that are set up correctly.

    All 10 machines above
  4. Client side rendering with the important parts server rendered

    The app is client side, but the copy, headings, links, and schema are in the source. Acceptable. Verify it rather than assuming it, because framework defaults change between versions.

    All 10, but only if you check
  5. Pure client side rendering

    Correct for a dashboard behind a login. Wrong for anything you want a stranger or a machine to find. Google will probably cope. Nothing else will.

    3 of 10. Google, Gemini, Apple

The bar on each rung is how many of the ten machines in the table in section 03 can read your content under that setup. Bingbot is counted as a miss because Microsoft recommends against relying on its rendering.

Server-side or pre-rendering is still a great idea because it makes your website faster for users and crawlers, and not all bots can run JavaScript. Google Search Central, JavaScript SEO Basics

Google wrote that sentence about its own crawler being good at JavaScript, and still recommended against relying on it. Two years later that recommendation reads less like caution and more like a forecast.

07 · What it costs

What this is worth in booked jobs

Here is the part that is easy to miss when the conversation is about rendering pipelines.

A homeowner's air conditioner dies on a Saturday in July. Ten years ago they typed "AC repair near me" into Google. Today a growing share of them ask an assistant instead, and the assistant hands back three companies and a sentence about each. Those three names came from whatever the crawler could read.

If the schema that says you do emergency AC repair in Round Rock only exists after JavaScript runs, you were never a candidate. You did not rank low. You were not in the pool. Nobody sends you a report about the call that did not happen.

For a shop doing between $750,000 and $5 million a year, a handful of missed jobs a month is real money, and it is the quietest kind of loss there is. You cannot see it in a dashboard. You feel it in a slow week you cannot explain, which is the week you end up answering the phone yourself at nine at night because you would rather work than sit with the number.

The fix is not exotic. It is making sure the words that describe your business are in the file your server sends. More calls is what you buy. Your time back is what you get.

08 · Check yours

Two minutes to check your own site

You do not need a tool for this. You need to look at your page the way a machine that cannot run code looks at it.

The one command version. Ask your server for the page and count the structured data blocks in what comes back.

# what a non rendering crawler receives
curl -s https://yoursite.com/services/ac-repair/ | grep -c 'application/ld+json'

# the title and description in the raw source
curl -s https://yoursite.com/services/ac-repair/ | grep -Ei '<title>|name="description"'

Now compare that against the rendered page. Open the same URL in Chrome, right click, Inspect, and look at the Elements panel. That is the rendered DOM, not the source. If the two disagree, the rendered version is what Google has and the raw version is what everything else has.

The three checks that catch almost everything. Turn JavaScript off in your browser and reload a service page. Can you read the copy? Does the H1 show up? In Search Console, run URL Inspection, then Test Live URL, then View Tested Page, and read the HTML tab. And paste the URL into Slack or LinkedIn to see if the preview card fills in, since those unfurlers read meta tags out of the source and will fail loudly if the tags are written by a script.

If you find a gap

Do not rip out the tool that is doing the injecting. It is probably earning its keep on Google. The move is to promote the important parts, the title, the description, the H1, and the schema, into the template so they ship in the source, and let the tool keep handling the rest. Belt and suspenders. The raw HTML is the belt.

09 · Sources

Where this came from

Every claim above traces to one of these. Read them yourself. We have linked the primary research rather than the summaries of it.

  1. Google Search Central, "Understand JavaScript SEO basics." Google's own documentation on the crawl, render, index sequence and the render queue. Source of the pre-rendering recommendation quoted above. developers.google.com
  2. Vercel and MERJ, "How Google handles JavaScript throughout the indexing process." July 31, 2024. Giacomo Zecchini, Alice Alexandra Moore, Ryan Siddle, Malte Ubl. Over 100,000 Googlebot fetches, 37,000 matched render pairs on nextjs.org across April 2024. Source of the render timings and the 100% full render finding. vercel.com
  3. Vercel and MERJ, "The rise of the AI crawler." December 17, 2024. The measurement that none of the major AI crawlers execute JavaScript, with the GPTBot and ClaudeBot fetch percentages. vercel.com
  4. Ahrefs, "JavaScript SEO: What You Need to Know." Patrick Stox, updated May 9, 2025. The rendering options ranking and the case against dynamic rendering. ahrefs.com
  5. Semrush, "JavaScript SEO: How to Optimize JS for Search Engines." Carlos Silva. Render queue economics and the Search Console testing method. semrush.com
  6. Google Search Central, dynamic rendering documentation. Updated August 2022 to state that dynamic rendering is a workaround and not a long term solution. Covered at the time by Search Engine Land.
  7. Cloudflare, "A deeper look at AI crawlers: breaking down traffic by purpose and industry." August 28, 2025. Crawl to referral ratios by operator and by industry, and the split between training and live user action. blog.cloudflare.com
  8. Bing Webmaster Guidelines and Screaming Frog's independent testing of Bingbot rendering. Basis for the partial rating in the table.
  9. SwarmSystem, first party measurement. September 10, 2026. One service page on a live Texas home service site running a client side SEO injector, compared as raw curl response against headless Chromium with a six second settle. Method and raw output are reproducible with the commands in section 08.

Search Atlas is deliberately absent from this list. We use it. That makes us an interested party on this specific question, and a source you cannot check independently is not a source.

10 · Questions

The questions people actually ask

These four have been in this page's schema since it was published. Now they are on the page as well, which is the point section 11 makes.

Does Google see content that is added by JavaScript?

Yes, in most cases. Google crawls the page, queues it, then renders it in a headless Chromium browser and indexes what the rendered page contains. A 2024 study by Vercel and MERJ across more than 100,000 Googlebot fetches found that 100% of HTML pages resulted in full page renders, with a median delay of about 10 seconds between crawl and render.

Do AI crawlers like GPTBot and ClaudeBot run JavaScript?

No. Vercel and MERJ measured the major AI crawlers and found that none of them render JavaScript. GPTBot requested JavaScript files in 11.50% of its requests and ClaudeBot in 23.84%, but neither executed them. Google Gemini is the exception, because it runs on Googlebot's rendering infrastructure.

Is inline HTML better than client side JavaScript for SEO?

Yes. Everything a machine needs to understand the page should be in the first HTTP response the server sends. Google states directly that server side or pre-rendering is still a great idea because it makes your website faster for users and crawlers, and not all bots can run JavaScript.

Should I use dynamic rendering to serve HTML to bots?

No. Google changed its documentation in August 2022 to say that dynamic rendering is a workaround and not a long term solution. Google recommends server side rendering, static rendering, or hydration instead.

11 · This page

This guide proves its own point

It would be strange to publish this on a page you needed JavaScript to read. So we did not.

Every word above is in the HTML the server sends. So is the article schema and the FAQ schema. The JavaScript on this page moves a scroll indicator and runs the check below. If you disabled it right now you would lose an animation and nothing else.

The receipt underneath is not decoration. When this page loads over HTTP it fetches its own raw source, strips the scripts and styles out of it, counts the words and the schema blocks, and compares that against what is on screen after everything has run. On a page built this way the two numbers match. On a client side rendered page they would not be close.

SELF CHECK · WORDS IN RAW HTML ... OF ... RENDERED · SCHEMA BLOCKS ... RAW / ... RENDERED · EM DASHES 0 · UNVERIFIED

Opened from a file rather than a server, the fetch is blocked by the browser and the check will say so. That is the check working.

Book a call