The race towards Artificial General Intelligence (AGI) lacks clear benchmarks. Google DeepMind aims to address this with a new cognitive framework. Their paper, "Measuring Progress Toward AGI: A Cognitive Taxonomy," lays out a scientific approach to evaluating AI's general intelligence.
The framework draws from psychology and neuroscience, identifying 10 core cognitive abilities crucial for AGI. These include perception, generation, attention, learning, memory, reasoning, metacognition, executive functions, problem-solving, and social cognition.
Deconstructing Intelligence
Each ability is defined based on extensive cognitive science research. The goal is to create empirical tools to track AI development beyond task-specific performance.
The proposed evaluation protocol involves benchmarking AI systems against human capabilities across these cognitive tasks. This method aims to prevent data contamination and provide a relative measure of intelligence.
To move from theory to practice, DeepMind is partnering with Kaggle for a hackathon. The event focuses on building evaluations for five critical abilities: learning, metacognition, attention, executive functions, and social cognition. This initiative is a key part of advancing AGI evaluation framework development.