Beyond Checks: GitHub's Alt Text AI Aims for Real Accessibility

Automated checks can confirm that alt text exists on images, but they often fail to determine if that text is actually useful. The GitHub Blog recently detailed how the company developed a plugin for its Accessibility Scanner to address this gap, aiming for genuine web accessibility.

Web accessibility is a growing concern, with recent reports indicating that a significant percentage of images on popular websites either lack alt text or provide descriptions that are vague or redundant. While machines can easily flag missing alt text, judging the quality of provided text is far more complex.

The Limits of Simple Checks

Many current automated tools verify the presence of an accessible name for an image but don't evaluate its descriptive quality. This is a deliberate design choice: overly strict rules can lead to false positives, prompting developers to disable the tools altogether. For instance, an alt attribute like "IMG_2847.png" or "image" passes automated checks but offers no real information to a visually impaired user.

GitHub's approach distinguishes between what can be objectively proven and what requires interpretation. They established five deterministic rules that don't require AI. These rules flag absent or whitespace-only alt text, generic filenames, placeholder text like "TODO", single generic words such as "image" or "logo", and repeated alt text across adjacent images.

Layout Matters for Repetition

Identifying repeated alt text presented a unique challenge. Initially, the system flagged identical alt text on icons even if they were physically separated on the page. The solution involved analyzing page layout, specifically the proximity of images, rather than just their order in the markup. This ensures that only truly redundant descriptions within a visual grouping are flagged.

The system uses Playwright for image identification, excluding elements with empty alt attributes (alt=""), as these are intentionally used for decorative images. The quality checker prioritizes avoiding false positives, opting for conservative rules that developers are more likely to keep enabled.

AI for Contextual Understanding

Deterministic rules are limited because they cannot understand the image's context. Whether "a smiling person" is an adequate description depends heavily on surrounding content. To address this, GitHub's plugin includes an opt-in AI-powered quality check.

This feature extracts contextual information, including the nearest heading, page title, captions, and nearby prose. It also identifies if an image is part of a link, which significantly changes the purpose of the alt text, it should then describe the link destination, not just the image.

The alt text, image content, and page context are then fed to a vision model via GitHub Models. This allows for a more nuanced evaluation, moving beyond simple string checks to assess descriptive adequacy. StartupHub.ai data shows that developer tools focused on accessibility have a low StartupHub score of 2/100, indicating a significant market opportunity for tools that genuinely improve usability.

Why This Matters

This development highlights a critical evolution in AI's role in software development. Beyond code generation, AI is now being applied to enhance crucial but often overlooked aspects like accessibility. By moving beyond basic compliance checks, GitHub is pushing the industry towards a more inclusive web.

For developers, this means tools are becoming smarter, offering deeper insights than ever before. For businesses, it translates to more accessible products and a better user experience for a wider audience. The challenge remains in balancing AI's interpretive power with the need for deterministic, reliable checks that developers will trust and use.