How Silktide crawls

Silktide does not crawl like a traditional HTML-only bot. It loads each page in a real browser - headless Chrome - so it sees roughly what a person sees after scripts run, layouts reflow, and resources load.

That is why Silktide can test modern, JavaScript-heavy sites that source-only crawlers miss, and why results can include layout, contrast, screenshots, and the final HTML after the page has settled.

What a crawl does

A typical test run:

Starts from the website's home page (and any forced pages you configured).
Loads each in-scope page in the browser on desktop and, where enabled, mobile.
Discovers more URLs from links, and often from sitemaps.
Keeps going until everything in scope is covered, or the website's is reached.

You control which URLs belong in that scope with What we test. Advanced crawler settings change how aggressively Silktide discovers and loads those URLs - see Loading a page.

Pages vs files

Not everything Silktide finds is loaded the same way:

HTML pages are opened in the browser. JavaScript runs, the page can be scrolled, and Silktide can click a configured control (for example to dismiss a cookie banner) before capturing results.
PDFs and many other documents are discovered while crawling - usually as links from pages - and downloaded as files rather than rendered as full browser pages. They still count toward document capacity, and they still have to be reachable.

If a PDF URL does not look like a PDF (for example it has no .pdf extension), Silktide may classify it as a page and consume page allowance instead. See Why can page limits prevent PDFs from testing?.

Why this matters

Because Silktide uses a real browser:

It experiences client-rendered content, not only the raw HTML response
It can interact with pages - dismiss banners, run user flows, or complete configured authentication
It can capture screenshots and measure visual qualities such as contrast and layout
Requests still come from Silktide's infrastructure (or a proxy), so firewalls and bot defenses can still interfere

For a shorter answer to “does Silktide see what I see?”, see How does Silktide see websites?.