Methodology

How we tested Our testing procedure

Our research is inspired by the WebAIM Million project ^[1]. To get similar insights into Europe's accessibility problem, we needed to test tens of thousands of websites. This cannot be done manually, so we built a custom scraper as an automated solution. The scraper takes a list of domains in CSV format, visits all of them, runs automated tests, and uploads the results as JSON files to a virtual folder.

The scraper can use ChatGPT to assign one of fifteen possible categories to each page it visits: Retail, Healthcare, Manufacturing, Real estate, Education and Science, Public Sector, Non-profit, Media and Entertainment, Mobility & Logistics, Technology, Financial Services, Business Services, Energy and utilities, Tourism, or Uncategorized. The categories are based on the IAB Content Taxonomy. This data allows for careful comparisons across sectors and countries.

We chose to use Deque's Axe API ^[2] for the automated accessibility tests because it's open source and reliable. Axe also has a CLI tool and browser extension available. This allows us to manually check the results we get from the scraper with the exact same rule set; especially useful when we first started testing and needed to make sure our automated test results were correct.

The scraper’s output is downloaded by our researchers, and filtered by a custom Python script to remove unusable and unhelpful results. Only 60% of domains in the raw original dataset are accepted and used in the final results.

Data source Tranco list

Our list of domains comes from the Tranco list ^[3], a research-oriented ranking for top sites. This year’s results use their full list from 20 March 2025, which can be downloaded from the Tranco website.

CSVs are extracted from that list for each European country we want to test. We do this by filtering on all official top level domains extensions (TLDs) for any given country.

Why are some European countries not in the list? Because we had fewer than 4,000 websites from these countries in our sample to test (after filtering). This would result in significant differences compared to other countries, making it difficult to compare results across Europe.

The following table lists the eighteen countries we tested in the first column, and shows the TLDs we’ve included for them in the second column. The third and final column shows the number of domains we’ve accepted and included in the results: 266,020 in total.

Results How we use Axe’s test results

The European Accessibility Act ^[4] enforces accessibility requirements for anyone offering essential products and services to users inside the EU. Companies must conform with the European standard for ICT accessibility: EN 301 549 ^[5]. This standard draws heavily from the Web Content Accessibility Guidelines (WCAG) version 2.1, level A and AA. It will soon be updated to reference version 2.2.

Axe’s testing rules ^[6] are mapped on WCAG. Each rule tests for a specific issue, and is linked to specific criteria. Our research uses only rules that test for WCAG version 2.1 level A and AA issues, 61 in total.

For example: the rule “color-contrast” checks whether text elements on the page have sufficient contrast against their background. It uses the minimum contrast ratio defined by WCAG for calculating this. A page can pass this test (all text has sufficient contrast), fail (not all text has sufficient contrast), or the test can be inconclusive (manual checks are needed to judge text contrast). Inconclusive results are ignored in our results.

We say 93% of European websites fail accessibility requirements, because 93% of the homepages we checked failed at least one such test. A single failed test can include multiple similar problems for the same sample. For example: a page failing the “color-contrast” test might have many instances of hard-to-read text throughout the design. These are recorded as separate violations, but together they still count as one failed test.

Axe results are processed with custom Python scripts to obtain specific data insights: how many pages included a violation, how do different categories compare, which issues are most common, etc. Each script answers a specific research question. We can compare these results to last year’s research to look for noticeable trends, or the lack thereof.