The Black Box of Agency: Why Computational Sophistication Defines Intelligence

Dr. Jeff Beck, a computational neuroscientist focused on the mathematical and physical underpinnings of intelligence, recently challenged the conventional philosophical boundaries defining agency in artificial systems. Beck, speaking with the interviewer, dissected the foundational concepts of agency, intelligence, and the architectural shifts driving modern AI research, particularly focusing on predictive models, energy-based systems, and the risks of human cognitive decline in an automated future.

Beck argues that the structural distinction often drawn between an agent and a mere object is mathematically weak. When viewed through the lens of policy execution—where a system maps inputs to outputs—a rock responding to gravity is fundamentally no different from a complex algorithm. The true difference lies solely in complexity. Beck notes that from a purely mathematical standpoint, “there’s no structural difference between an agent and an object. It’s really just a question of degrees.” An agent is simply an object capable of extremely sophisticated computation, allowing for things like planning and counterfactual reasoning over vast timescales.

This leads directly to the “black box problem” of agency: how do we know if a system is truly thinking or merely executing a pre-computed lookup table? If we can only observe the input-output policy, we cannot definitively conclude that an agent is engaging in deliberation, counterfactual reasoning, or goal-oriented behavior. The only way to infer true agency is to examine the internal states and computations—the means, not just the ends. This challenge pushes researchers toward modeling approaches that make internal states tractable and interpretable. Science, Beck suggests, is fundamentally about data compression; we adopt the simplest computational model that adequately describes the observed behavior. If the simplest model requires internal states representing planning, then we pragmatically conclude the system possesses agency, even if the absolute truth remains unknowable.

The search for robust, generalizable AI architectures is deeply informed by this need to model complex internal dynamics. Beck highlights the foundational relevance of Energy-Based Models (EBMs), pioneered by figures like Yann LeCun. Unlike traditional neural networks, which only optimize weights to minimize a cost function on outputs, EBMs incorporate the cost function over all states—including the latent variables. This architectural choice forces the model to treat its internal representations as trainable latent variables, allowing for optimization at inference time. This is the essence of test-time training, a critical technique gaining traction across deep learning, and one that aligns neatly with Bayesian inference, where energy functions are analogous to log probabilities. The ability to optimize internal states during inference gives EBMs a powerful flexibility that standard feed-forward networks lack, effectively building a flexible inductive prior into the system.

A powerful manifestation of this shift is Yann LeCun's Joint Embedding Predictive Architecture (JEPA). JEPA moves beyond predicting every pixel in an image—a task that often forces models to waste capacity learning high-frequency, pixel-level artifacts—to predicting compressed, abstract representations in the latent space. This non-contrastive approach seeks to maintain semantic richness and fidelity while avoiding the "mode collapse" problem, where embeddings become trivially identical. By predicting in this compressed latent space, JEPA encourages the network to learn robust, high-level features essential for understanding the world. This focus on abstract representation learning, rather than brute-force pixel prediction, is seen as a necessary step toward building models capable of true generalization.

Beck views true artificial general intelligence not as a monolithic, centralized entity, but as a system capable of constructing novel, specialized models autonomously. He draws a compelling parallel to biological evolution, suggesting that intelligence progresses through the combination of specialized, modular components. He posits that the highly complex, non-smooth nature of olfactory space, which requires associative processing rather than simple translational symmetries found in visual space, may have been a key driver in the evolution of our associative cortex. The next major milestone in AI, therefore, is not merely achieving high performance on fixed tasks but creating systems that autonomously generate and test hypotheses—an ability Beck terms "active inference."

This push toward autonomous, continually learning systems introduces complex safety considerations. Beck is less concerned with the catastrophic, rogue superintelligence scenario often invoked in popular media, and more focused on the risk of human cognitive enfeeblement. If AI automates all critical thinking, humans risk being reduced to mere "reward function selectors," passively approving or rejecting algorithmic outputs without deep understanding. Beck argues, "I don't really want to have a situation where humans are reduced to like value function selectors. They’re just basically going, oh no, I don't like that outcome, like do this instead." The solution, he suggests, lies not in automation for its own sake, but in building AI that improves human understanding and facilitates scientific discovery, always employing inverse reinforcement learning (IRL) to derive goals from observed human behavior and ensuring any suggested action involves small, controllable perturbations rather than sweeping, naive commands.

Ultimately, the philosophical and technical discussions converge on the idea that intelligence is a measurable degree of computational sophistication, best modeled through architectures that prioritize robust internal representations and the ability to combine specialized knowledge modules dynamically.