When AI Becomes the Front Door: Measuring Digital Accessibility Through Real Tasks

A new intervention by Disability Belongs asks organisations to judge AI-assisted access through people's experience. The practical test is whether users can complete an activity independently, understand the information and recover when something goes wrong

Imagine a person trying to apply for a service online, using a keyboard and a screen reader. The page offers an AI assistant that can explain the process in everyday language. It may give a clear account of the documents required and help the person prepare an answer. The application can still fail at a later step if an essential control cannot be operated from the keyboard or an error message is never announced. This illustrative experience captures a distinction that matters as organisations add AI to digital services. An improvement in one interaction can coexist with a barrier elsewhere in the same task.

In a statement published on 10 November 2026, Disability Belongs argues that AI can support digital accessibility while people with disabilities remain essential to assessing whether a service works in context. The organisation points to advances such as captions and image descriptions, alongside continuing gaps in accuracy and usability. This is an advocacy intervention, rather than a new controlled evaluation of a product. Its significance is the question it directs towards service owners: whose experience will determine whether the added technology has made access more dependable?

The answer affects more than convenience. A difficult interface can make someone rely on a relative, colleague or paid assistant to do something they would otherwise do privately. A well-designed tool could give that person more control over when and how to act. An unreliable tool could make the same task appear easier while leaving the user to discover a consequential error. Evaluating AI for accessibility therefore requires attention to the complete activity, the support the person needs and the options available when the technology fails.

The promise of interpretation

Generative systems can make information available in forms that are easier for some people to use. An illustrative assistant could draft a description of an image, turn a long passage into a shorter explanation or help locate an instruction in a collection of documents. These possibilities are especially attractive where the original material is difficult to navigate. They can also help authors identify work that needs attention. The proposed benefit is more usable information at a manageable cost, subject to checking whether the transformation preserves what the person needs to know.

The context can change what counts as a good description. A photograph used to establish who attended an event may call for different information from the same photograph used to demonstrate a particular piece of equipment. A diagram explaining a route may need relationships that a general account of its appearance omits. A model can produce confident language without knowing the author’s purpose. Reviewing the output against that purpose is therefore part of creating the accessible material. The relevant question is whether the description supplies the information needed for the activity in which it appears.

A simplified explanation presents a similar trade-off. In a hypothetical benefits application, removing a qualification could make a sentence easier to read while changing the eligibility condition. Retaining every qualification might make the explanation hard to follow. A service could provide a clear summary linked to the precise rule and a route for resolving uncertainty. The user should be able to recognise which version they are reading and which decision it supports. A transformation is most dependable when its role is explicit and its accuracy can be checked against an identified source.

A scan cannot stand in for a journey

The distinction between a useful automated check and a complete evaluation is established in accessibility practice. W3C’s guidance on selecting evaluation tools explains the role of tools in finding issues while recognising the need for human judgement. A report can help a team inspect particular properties of a page. Assessing whether the service is accessible requires a broader evaluation. An AI-based checker extends the set of things software may be able to flag, but the scope of the resulting claim still has to match what was actually examined.

Consider an illustrative booking process whose individual pages pass many checks. A person might still lose their selected options when the service returns from a payment screen, or encounter a time limit too short for the method they use to enter information. The problem lies in the sequence of steps as experienced by the user. A team could test a small set of complete tasks, including recovering from an error and obtaining confirmation. That would reveal a different class of issue from reviewing a screenshot or inspecting a page outside the context in which it is used.

A recent preprint on AI assessment of accessibility in heritage tourism gives a related example of the limits of available evidence. In a five-park field study in Spain and Italy, the authors report that an AI desk audit could score only 66% of the dimension–site combinations and could not observe some embodied and organisational aspects. This is a small study in tourism, not a test of digital form completion. It illustrates a general measurement problem: an assessment built from online descriptions has limited access to conditions that only a person visiting or using a service can discover.

Whose experience enters the evaluation?

People with disabilities bring different experiences, tools and strategies to the same interface. A person who reads visually at high magnification may encounter different barriers from someone who uses speech output or a switch device. Experience with a particular screen reader can also affect how easily a task is completed. A test with one participant can reveal an important failure without representing every user. The range of tasks and participants needs to be related to the claim the service owner hopes to make, with limitations stated clearly enough for another team to understand what remains uncertain.

W3C’s guidance on involving users in evaluation treats participation by people with disabilities as a way to understand accessibility issues and usability in practice, alongside other evaluation methods. That participation can begin while choices are still reversible. In an illustrative pilot, participants might compare a conventional route with one that offers AI assistance, explain where they lost confidence and show how they recovered. The team could then examine whether the assistant reduced effort, introduced new uncertainty or merely shifted a difficult step to a different part of the process.

The terms of participation matter. Asking people to spend time discovering failures creates work that should be recognised and resourced. A participant may also need accessible materials, an appropriate testing environment or a support arrangement of their own choosing. A team that recruits only the people easiest to reach can miss those whose circumstances place greater demands on the service. In practice, involving users means allowing their findings to change the design, including decisions about whether an AI feature should be optional, restricted to a particular purpose or accompanied by another route.

The cost of checking an answer

An AI assistant can shift work from producing information to verifying it. For a user who finds the original material difficult to access, the second task may be especially demanding. A fluent answer about a deadline is hard to verify if the underlying document is an inaccessible scan. A generated image description can be helpful while leaving a blind reader with little independent basis for detecting an omission. This asymmetry makes the service owner’s responsibility consequential. The organisation supplying the information is better placed to check recurring content than each person who needs to use it.

There are practical ways to make that responsibility observable. A hypothetical service could approve explanations of stable rules, show the source and date, and limit generated responses when a question requires an individual decision. It could record corrections and ensure that a known mistake is addressed in the material other users receive. These are proposed design choices. Their effectiveness would need to be tested in the intended setting, including whether users can tell when the assistant has reached its limit and can obtain the next form of support without starting the whole process again.

A human contact route also has to be usable. A telephone number offers little help to someone for whom voice communication is difficult, and an email address may be inadequate for a matter with a short deadline. A service could give people a choice of ways to continue and carry forward the relevant context with their agreement. That would make recovery part of the service itself. If the only way to resolve an assistant’s mistake is to find another person willing to navigate the same barrier, the apparent efficiency has been purchased with someone else’s time and independence.

Access includes control over personal information

Assistance can involve sensitive context. A person may explain a disability, a difficulty reading or a need for a different communication method because that information helps them complete the current task. They may have a reasonable expectation that the detail stays within that interaction. In a hypothetical integrated system, the same information could be retained in conversation history or supplied to another component. Making those flows understandable is part of giving users control. Consent is more meaningful when people can obtain an accessible service without having to disclose more personal information than the task requires.

Personalisation creates a related tension. Remembering a preferred format could spare someone from repeating an instruction every time. Remembering a broad inference about their abilities could become inappropriate in a later situation. A sound evaluation would ask whether users can inspect and change relevant preferences and whether an assistant carries assumptions between contexts. It would also examine how those controls can be used with the assistive technology the person already relies on. An inaccessible privacy setting is a concrete barrier to exercising the control that the product claims to provide.

These questions connect AI accessibility to organisational decisions about procurement and maintenance. A service owner can ask what information is retained, how updates are evaluated and who is responsible for correcting harmful outputs. It can also require evidence from complete tasks performed with relevant assistive technologies. A supplier’s demonstration can identify a promising capability. The organisation adopting it needs a continuing process for learning what happens after deployment, particularly when software changes alter navigation, the wording of explanations or the behaviour of the assistant.

What progress would look like

The most persuasive evidence would show that people can complete important activities with greater independence, acceptable effort and a reliable way to resolve errors. Completion rates would be one part of that picture. The time spent checking an answer, the need for outside help and the ability to recover from a failure would add information about the quality of access. Results should distinguish among tasks and users where the evidence permits, rather than folding every experience into one optimistic average. A useful pilot could discover a narrow application that works well and a broader one that needs further work.

The November intervention by Disability Belongs gives service owners a reason to ask those questions as AI features become easier to add. Better captions, descriptions and explanations can make a real difference when they are fitted to a person’s task and checked in use. The social gain lies in what the person can do afterwards, including applying privately, learning independently or making a choice with information they can understand. Designing and evaluating around those outcomes gives the people affected a continuing role in determining what counts as improvement.

Image credit: Hand on a high-contrast accessible keyboard by Francisclarke, CC BY-SA 4.0. Illustrative photograph. The source file is unedited. WordPress may resize or crop display versions.

See other related publications