Hybrid AI Models Get Orthogonal

The inherent tension between interpretable yet simplistic mechanistic models and powerful but opaque data-driven neural networks has long constrained the modeling of natural phenomena. Hybrid approaches, combining symbolic physics with neural flexibility, promise a synergistic solution, but a critical failure mode exists: neural components can relearn and subsume symbolic structures, leading to redundant and uninterpretable models. This issue is particularly acute when the symbolic framework itself is data-discovered. Existing methods, often relying on $L^2$ regularization, falter when symbolic components are learned sparsely, allowing neural augmentation to bleed into the symbolic domain. According to the researchers, this leads to a breakdown in the desired complementary decomposition.

Unlocking Complementary Decompositions with OrthoReg

To surmount this challenge, the authors introduce OrthoReg (Orthogonal Regularization). This novel technique directly penalizes the overlap between the symbolic and neural components of a hybrid model. By explicitly enforcing orthogonality, OrthoReg prevents the neural residual from absorbing or re-expressing the information already captured by the symbolic structure. This ensures a clear division of labor: the symbolic part captures what can be expressed by the known library of physics, and the neural part learns to model only what remains unexplained or unmodeled by the symbolic component. This orthogonal regularization is key to achieving interpretable and robust hybrid models.

Enhanced Robustness and Discovery in Dynamical Systems

The efficacy of OrthoReg is demonstrated on benchmark dynamical systems. Notably, when faced with partial library mismatches, a scenario where the available symbolic components are incomplete, OrthoReg shows significant improvements in both symbolic recovery and out-of-distribution generalization. This suggests that OrthoReg not only leads to more interpretable hybrid models but also enhances their ability to generalize to unseen conditions, a critical factor for real-world applications. The ability of OrthoReg dynamical systems to maintain distinct symbolic and neural contributions is a significant step forward.