How scoring changed in Silktide 5

Silktide 5 scores your website on the same idea as Silktide 4: every score starts at 100 and loses points for each problem we find. What changed is how those points are counted. Silktide 5 uses one simple rule everywhere, gives each part of a score a fixed share, and shows you the exact cost of every check.

Most websites score a little lower in Silktide 5 than in Silktide 4. That is deliberate. Silktide 5 tests for more problems, including the newest accessibility guidelines, and we would rather your score be slightly stricter than flattering. This page explains why scores changed and how each one is calculated, so you can trace any score back to the checks behind it.

Why your score changed

• The points are counted differently. Silktide 4 spread points across a tree of nested categories and mixed them according to how many pages each category covered. Silktide 5 pools checks into a few parts, each with a fixed share of the score. See What changed from Silktide 4.
• Silktide 5 runs more checks. It tests against WCAG 2.2, the latest version of the accessibility guidelines, and adds ARIA and accessibility best practice checks. Each new check can only take points off, never add them.
• Some checks moved. Marketing and Privacy no longer exist as separate modules, so their checks now count towards User Experience, Content, or Compliance.
• Some things are measured differently. For example, page speed is now measured on your key pages, and on mobile over a faster connection than before. The smaller changes are listed at the end of this page.

To see exactly which checks were added, moved, or retired for one of your websites, open What changed in your score from any module. It puts your Silktide 4 and Silktide 5 scores side by side from the same scan. See What changed in your score.

What stayed the same

• The scale. Scores run from 0 (worst) to 100 (perfect), for your website as a whole and for each module.
• The labels. The same ratings apply in both versions:

• How a check measures your site. Each check scores your whole site from 0 to 100, not page by page. That score blends how many of your pages are affected with how many issues each affected page has. A typo on one page out of ten thousand barely registers; the same typo on half your pages is a serious problem.
• How scores respond to your work. Fixing issues and retesting improves your scores, and decisions such as ignoring or approving an issue update your scores without a retest.
• New checks never inflate scores. A check can only take points away. When Silktide adds a check that your site passes, your score stays where it was.

How a score is calculated

Every score starts at 100

Each check that applies to your site can take points off, up to a maximum called its impact. How much it actually takes depends on how badly your site fails it:

points taken off = impact x (100 - check score) / 100

A check with an impact of 10 that scores 70 on your site takes off 10 x 30 / 100 = 3 points. If it scores 100, it takes off nothing. If it scores 0, it takes off its full 10.

A score is 100 minus the points every check takes off, and it never drops below 0. A check with nothing to test on your site, such as a video check on a site without videos, takes nothing off.

Where a check's impact comes from

The most important checks keep the impact they had in Silktide 4, so the problems that mattered most before still matter most now. For example, a missing main heading can take up to 18.2 points off web accessibility, and broken links can take up to 20 points off Content.

Every other check's impact comes from two things Silktide assigns it: its priority and its confidence.

The weight is the priority weight times the confidence multiplier, so a P1 automated check weighs 16 x 9 = 144 and a P3 assisted check weighs 4 x 3 = 12. Each part of a score then turns weight into points at its own rate, so that a score lands where it should for that part. This is what a check can take off its part at most:

Web accessibility pools many checks, so each one is worth less; each User Experience criterion has only a handful, so each is worth more. The priorities are the ones Silktide assigns, which keeps scores comparable between organisations. Changing a check's priority in your settings changes how it is reported, not its impact on your score.

Review and informational checks never take points off, whatever their priority. They point out things no machine can judge, for your team to look at.

Checks are pooled into parts

Each module is made of one or more parts. Each part pools the checks of its criteria, and its score is 100 minus the points those checks take off. A check that belongs to two criteria in the same part only takes points off once.

The module score combines its parts by their shares:

module score = sum(share x part score) / sum(share)

A part with nothing to test on your site is left out, and the other parts fill its share. A website without PDFs is scored on web accessibility alone, so it is never penalised, or rewarded, for having no PDFs.

A check can count in more than one module. Missing alt text matters to both Content and Accessibility, and sideways scrolling on small screens matters to both Accessibility and Mobile. Each module charges it separately, at its own impact.

On a module's How it's scored screen, each check's impact is shown on the module's 100-point scale. A web accessibility check therefore shows 90% of its impact within its part, because web accessibility is 90% of the Accessibility score.

WCAG levels

The Accessibility module shows a score for each WCAG conformance level. Each level includes every level below it, exactly as conformance does: AA covers the level A and AA checks, and AAA covers all three. Checks cost the same at every level, so your AA score can never be higher than your A score, and your AAA score can never be higher than your AA score.

A worked example

Suppose a website's Accessibility checks score like this:

Web accessibility scores 100 - 1.36 - 0.91 - 1.50 = 96.2. PDF accessibility scores 100 - 10 - 2.5 = 87.5. The Accessibility score is:

(90 x 96.2 + 10 x 87.5) / 100 = 95.3

The website scores 95, an Excellent rating. Every number in that chain is visible in the product: open any module's How it's scored screen to see its parts, the checks in each criterion, their impacts, and the points each one takes off.

Compliance

Compliance is new in Silktide 5, so it has no Silktide 4 score to compare with, but it follows the same rule: every regulation and policy starts at 100 and loses points for each problem. The Compliance score is the average of the ones chosen for the website.

• Accessibility regulations are scored exactly like the Accessibility module at the WCAG version and level they require, with the same impacts: web pages count for 90% and PDFs for 10%. A regulation built on WCAG 2.1 AA therefore scores the same as your WCAG 2.1 AA conformance, with your PDFs added.
• GDPR, cookie policy, and security keep the impacts their checks had in Silktide 4's Privacy module. A missing SSL certificate, for example, can take up to 30 points off Security.

What changed from Silktide 4

Why we changed it:

• Adding checks had to stay safe. Silktide 5 adds many checks, including WCAG 2.2. Pricing most of them by priority and confidence lets us keep adding checks without retuning everything else, while the checks that matter most keep the impacts they have always had.
• Scores should not move unless your site does. In Silktide 4, adding pages to a test changed how much your PDFs counted. In Silktide 5, each part's share is fixed.
• Every point should be explainable. "This check cost you 3 points" is only true if points add up simply. In Silktide 5 they do: a part's score is 100 minus the points its checks took off.

Neither version is wrong; they apply different rules to the same results. When you track progress over time, compare scores within one version rather than across the two.

Related

• What changed in your score - compare your Silktide 4 and Silktide 5 scores from the same scan.
• Criteria - the groups of checks each module is built from.
• Checks - review individual checks.