GitHub pilots AI for accessibility

GitHub is actively exploring the capabilities of AI agents, piloting an experimental tool designed to enhance web accessibility. This initiative aims to provide developers with real-time answers to accessibility questions and automatically remediate simple issues before code reaches production.

The general-purpose accessibility agent is currently evaluating front-end code changes, having reviewed 3,535 pull requests with a 68% resolution rate. Key issues addressed include clarifying structure and relationships for assistive technologies, ensuring clear names for interactive controls, providing alerts, offering text alternatives for non-text content, and maintaining logical keyboard focus.

These automated fixes directly reduce barriers for users relying on assistive technologies, improving the overall usability of GitHub's platform. The company plans to share its findings and lessons learned to aid other teams in their accessibility endeavors.

The Agent's Mandate

The accessibility agent has two primary objectives: delivering instant, reliable accessibility guidance via GitHub Copilot CLI and VS Code integrations, and proactively catching and correcting straightforward accessibility flaws. This proactive approach is designed to catch issues early in the development cycle.

Lessons from the Trenches

GitHub emphasizes that accessibility is a complex, holistic concern, not a problem to be solved in isolation. The agent acts as an augmentation tool, supporting developers in removing barriers inherent in UI design.

Crucially, the agent's effectiveness hinges on a robust, structured dataset of past accessibility issues. GitHub's mature system for logging, verifying, and centralizing these problems provided an ideal training corpus. The non-deterministic nature of LLMs proved beneficial in extrapolating code and language patterns from this historical data.

Organizations must invest in manual cataloging and remediation of accessibility issues to provide the necessary context for AI tools. This manually curated data, rich with organizational conventions and contextual examples, significantly enhances the agent's performance.

Optimizing AI Performance

Given that accessibility is a contextual and cross-disciplinary concern, a general-purpose agent can consume a large number of tokens, leading to increased costs, slower responses, and potentially unreliable output. To mitigate this, GitHub evolved its agent from a monolithic structure to a sub-agent architecture.