Automated scanning vs manual testing: what each one actually catches
We sell an automated scanner, so it is worth being direct about its limits. An automated accessibility test is very good at a specific class of problem and structurally incapable of deciding another class. Knowing which is which is the difference between a site that passes a scan and a site a blind customer can actually buy from.
This is the honest split, and a manual pass you can run yourself in about twenty minutes.
What a scanner decides with certainty
Automated testing works where a criterion can be reduced to a rule with an unambiguous answer. There is no judgment in whether an image element has an alt attribute, or whether two colours meet a 4.5:1 ratio.
- Missing alt attributes, missing form labels, missing accessible names on links and buttons.
- Colour contrast ratios for text and for non-text UI components.
- Missing or invalid lang attributes.
- Heading levels skipped, multiple H1s, empty headings.
- ARIA used incorrectly — invalid roles, required attributes absent, references pointing at IDs that do not exist.
- Duplicate IDs, nested interactive elements, controls that cannot receive focus.
- Tables without header associations, lists built from divs.
- Autoplaying media, missing captions tracks, viewport meta tags that block zoom.
Why that matters more than it sounds
That list is not a consolation prize. The six error types WebAIM found across the top million home pages in 2026 — low contrast on 83.9% of pages, missing alt text on 53.1%, missing form labels on 51%, empty links on 46.3%, empty buttons on 30.6%, missing document language on 13.5% — are all in the machine-decidable set.
So the automated half clears the failures that are most widespread, cheapest to fix, and most likely to be cited in a complaint, and it does it across a whole catalogue in one run rather than page by page. That is exactly the work you should not be paying a human to do.
What a scanner cannot decide
The other class of criteria depends on meaning, and meaning is not machine-checkable. A tool can confirm that alt text exists; it cannot confirm that the alt text describes the image.
Treat any single coverage percentage with suspicion, because there are two different denominators and they answer two different questions. Counted by volume of individual findings, automation does most of the work: Deque, which makes the axe engine, published a study of nearly 300,000 findings across 13,000 pages putting automated coverage at about 57% of issues by volume — a vendor measuring its own category, so weigh it accordingly. Counted by success criteria, the picture inverts: only roughly a fifth to a third of WCAG criteria can be machine-tested at all, because the rest turn on judgment.
Both figures are true and neither settles the question alone. Which one matters depends on what you are doing. If you are clearing a backlog, volume is the useful denominator. If you are deciding whether a scan is sufficient evidence of conformance, criteria is the only denominator that answers the question — and on that axis most of the standard is still waiting for a person. WebAIM puts the same point without a number: the absence of detected errors does not indicate that a page is accessible or conformant.
- Whether alt text is accurate and useful, or filler that passes the rule and tells the user nothing.
- Whether link text makes sense out of context, which is how screen-reader users encounter it.
- Whether the heading structure reflects the actual organisation of the page.
- Whether focus order follows the visual and logical order, especially after a modal opens.
- Whether an error message tells the user how to fix the problem, or just that there is one.
- Whether captions are correct and synchronised, and whether a video needs audio description.
- Whether a custom component behaves the way its ARIA role promises — a div labelled as a tab list that does not respond to arrow keys passes markup checks and fails users.
- Whether the site is usable end to end. No rule fires when a checkout is technically conformant and practically impossible.
The 20-minute keyboard pass
The highest-value manual test is also the cheapest. Put the mouse out of reach and try to complete your site's primary task — usually buying something — using only Tab, Shift+Tab, Enter, Space, the arrow keys, and Escape. Work through it in this order.
- Load the home page and press Tab once. A skip link should appear and, when activated, move focus to the main content — not just scroll the page.
- Keep tabbing. At every stop you must be able to see where you are. If the focus indicator disappears anywhere, note the component.
- Watch for focus landing behind a sticky header, cookie banner, or chat bubble. That is a WCAG 2.2 AA failure in its own right.
- Open the main navigation with the keyboard. If it only opens on hover, it is unreachable. Close it with Escape.
- Open a modal — search, cart drawer, newsletter popup. Focus must move into it, stay inside it while open, close on Escape, and return to the control that opened it.
- Reach a product and change a variant. Size and colour pickers built from styled divs are a frequent dead end.
- Add to cart and go to checkout. Complete every field by keyboard, then deliberately submit an invalid one and check whether the error is announced and associated with the field, not just coloured red.
- Zoom the browser to 200%, then narrow the window to 320px wide. Nothing should require horizontal scrolling and no content should be cut off.
- Turn images off, or run the page in a text browser. Anything that becomes unusable was carrying information in a picture.
- Finally, run the same purchase with a screen reader — VoiceOver on macOS and iOS, NVDA on Windows, both free. The first ten minutes will tell you more than the previous nine steps combined.
How to combine them
The sequence that wastes the least time is: scan first, fix everything mechanical, then test manually. Running a manual audit against a site that still has four hundred missing alt attributes means paying a specialist to write down what a scanner would have listed for free, and the real problems get buried in the noise.
After that, the cadences differ. Automated scanning should run continuously — weekly or daily — because it is cheap and because regressions arrive through theme updates and new products rather than through deliberate changes. Manual testing is an event: after a redesign, before a launch, when a new template ships, and on a fixed schedule for the purchase path.
What this means for the tools you buy
Be wary of any product that claims the automated half is the whole job. No scanner can judge whether your alt text is accurate, whether your focus order makes sense, or whether a screen reader user can actually finish checkout. Those need a person. A tool that implies otherwise is selling a feeling of safety rather than a record of work, and the feeling is what evaporates when someone complains.
An honest tool tells you what it found, names the criterion, tells you what it could not assess, and does not issue a compliance badge. That is how AdaChecked is built. The report states on its first page that the score is a severity-weighted health signal rather than a conformance claim, and it separates what was machine-tested from what was not — so the manual walkthrough above has a defined scope instead of being guesswork.