Discovery

Discovery is how Silktide builds the list of URLs to fetch. A URL that is never discovered cannot be tested, even if it exists on the website.

Where URLs come from

Silktide typically finds URLs from:

1. The homepage - every crawl starts here.
2. Links - as each page loads, Silktide follows in-scope links to find more pages and documents.
3. Sitemaps - when Silktide can find them automatically (for example \/sitemap.xml), unless you turn that off in Advanced settings.
4. Forced pages - specific URLs you add under What we test so important pages are included even when nothing links to them.

Forced pages still have to be reachable when the test runs. Forcing a URL does not bypass authentication, firewalls, or a broken link.

What stops discovery

Discovery stops or skips URLs when:

• The URL is outside scope (denied, or not matched by an allowed rule when allowed rules are set)
• Link discovery is disabled, so Silktide only tests explicitly provided URLs
• Automatic sitemaps are ignored and nothing else links to the content
• The website's page allowance is reached - shorter, earlier URLs tend to be chosen first; see How does Silktide choose which pages to test?
• The page is unreachable (redirects, errors, bot blocks) - see When crawling fails

Orphan content - pages that nothing reachable links to, or that are not forced or listed in a sitemap that Silktide reads - will not appear in Inventory.

After the crawl

Use Inventory coverage to understand gaps compared with a CMS list. Where Analytics is connected, Untested pages highlights visitor URLs that never entered the tested inventory.