Your usability tests can’t see the users who already left

The finding nobody was looking for

I was lead product designer on Amazon’s Employee Document Management platform for more than two years. EDM manages 4.8 billion documents a year across 67 countries, for everyone from candidates signing offer letters to alumni requesting records. When I joined, the team already had a healthy research foundation: heuristic evaluations, usability tests, SUS scores, and journey maps from pre-hire through alumni.

None of it had surfaced what I heard in interviews: most employees with disabilities weren’t using the document portal at all. They were downloading PDFs straight to their computers so they could open them with their own assistive tools. The default viewer wasn’t screen-readable, and people had adapted by memorizing menu positions and workflows just to get through basic tasks.

That isn’t a minor inconvenience. It’s a complete breakdown of the intended experience for a meaningful part of the workforce, and it had gone undetected for a long time.

Why good research missed it

Nothing was wrong with the existing research. It was doing exactly what it was designed to do: measure how well the product worked for the people using it. That’s the blind spot. Most research methods start from the product and look outward, so they are structurally unable to see the people who have given up on it.

Worse, a workaround can look like success in your data. An employee who downloads a document and reads it in their own screen reader still generates a “document accessed” event. The breakdown shows up in the metrics as completion.

Standard testing doesn’t catch people who abandon the product entirely. You have to go looking for them.

Absence is a signal

The shift I’d ask any team to make is to treat workarounds as first-class findings. When someone exports, downloads, prints, or asks a coworker to complete a task, they’re telling you where the product failed them. Those behaviors are easy to dismiss as preference. Often they’re adaptation.

In practice, that means asking different questions in interviews. Not just “how do you do this in the tool?” but “what do you do instead?” and “when do you avoid the tool entirely?”

The fix was smaller than the problem

Once we understood the problem, the solution wasn’t exotic. I worked with engineering to evaluate and implement the Mozilla PDF reader, which gave us full screen-reader compatibility with minimal operational impact. Documents that required signatures needed a separate approach, which meant bringing in legal and financial teams to align on one.

After launch, I went back to the same participants. The response was immediate. People were visibly moved that they’d been considered at all. We also saw something we hadn’t predicted: application abandonment dropped after the viewer redesign, and the platform met accessibility compliance for the first time.

The counterargument: “we can’t research every population”

Research budgets are real, and it’s fair to ask whether dedicated accessibility recruitment is worth it on every project. My answer is that you don’t need a separate study. You need inclusive recruitment built into the studies you’re already running. That costs far less than discovering the problem years later.

The work also compounds. When we later added an AI layer to EDM’s employee portal, with plain-language summaries of legal agreements and a mobile briefing card for new hires, the accessibility foundation was what made it possible. Designing for the people on the margins made the product better for everyone else too.

What to do Monday

• Add participants with disabilities to your next study’s recruitment criteria, not to a follow-up study.
• Ask “what do you do instead?” in every interview, and log workarounds as findings.
• Look at your data for workaround behavior: downloads, exports, and repeated drop-off at the same step.
• When you fix something, go back to the same participants. It builds trust, and it tells you whether the fix worked.