Lab component 03

What the crawler gets versus what you see

Drag the handle. The right side is an iframe with scripts genuinely disabled, which is the document as delivered. The left side is the same bytes with JavaScript allowed to run. The counters underneath are a real diff, not an illustration.

Genuinely sandboxed Scripts actually disabled on the left

Crawler view vs browser view

Rendered DOM Raw response
0Nodes added
0Text chars added
0Links added
0Headings added
0%Content JS dependent

Loading both documents.

Show the exact bytes being served to the left pane

Both panes receive byte identical HTML. The right pane is sandboxed without allow-scripts, so its JavaScript genuinely never executes. The left pane is the same string with scripts permitted. The counters are a real DOM diff between the two documents, measured after both settle. This is the closest a browser can get to showing you what a non rendering crawler receives.

Rendering is a queue, not a guarantee

Google will render JavaScript. The question is never whether, it is when, how consistently, and whether every other system in the retrieval chain will bother. Most do not.

01

Two wave indexing

The initial crawl indexes the raw response. Rendering happens later from a separate queue, which means a CSR page can sit indexed but empty for a while.

02

LLM crawlers mostly do not render

Several major AI training and retrieval crawlers fetch raw HTML only. If your content needs JS to exist, you are invisible to them by construction.

03

Link discovery dies first

Nodes added is a nuisance. Links added is the real problem, because JS injected links may never enter the crawl frontier at all.