AI in Biology: From Tokens to Cells at Altos Labs

Akram Baharlouei, a machine learning engineer at Altos Labs, delivered a talk titled "From Tokens to Cells: The Engineering Challenges of Foundation Models for Single-Cell Biology." Altos Labs, a biotech startup, aims to restore cell health and resilience through cellular rejuvenation, with the ultimate goal of reversing age-related diseases and disabilities. Baharlouei, approaching the subject from an engineering perspective without a deep biological background, explored the complexities of applying foundation models to single-cell biology.

The Importance of Single-Cell Biology

Baharlouei highlighted the significance of single-cell studies, referencing the groundbreaking work of Shinya Yamanaka and his discovery of the Yamanaka factors in 2006. These four transcription factors can reprogram aged skin cells into a state resembling embryonic stem cells, opening avenues for regenerative medicine, tissue regeneration, and potentially reversing cellular aging. The development of reprogramming medicine, like the OSK therapy, signifies a major step forward, with human trials anticipated soon. The pursuit of a "virtual human" and "digital twins" relies on a holistic understanding of cells, tissues, and organs, making single-cell analysis a foundational element.

Challenges in Drug Development and AI's Role

The talk also touched upon the stark contrast between exponential improvements in computing power (Moore's Law) and the declining output of drug development. Despite technological advancements, the number of new drugs developed annually has decreased, with a high failure rate in the drug development pipeline, which can take up to 10 years and cost billions. Baharlouei suggested that AI and a deeper understanding of cellular processes, such as through virtual cell modeling, could help to reduce these timelines and costs by enabling breakthroughs across the entire pipeline.

Measuring and Modeling Single Cells

Baharlouei detailed various modalities for measuring single-cell data, including genomics, transcriptomics (RNA-seq), proteomics, and morphology. While RNA-seq is currently the most utilized for foundation model training due to its scalability and the availability of large datasets (millions to billions of cells), he emphasized the need for technological advancements in other modalities to gain a more complete biological picture. A key challenge in this field is the heterogeneity of single-cell data; even identical cells can yield different measurements, and much of this variability may be noise rather than signal. This noise stems from both biological processes, such as gene expression bursts, and technical factors like varying experimental conditions.

Foundation Models and Future Directions

The presentation explored the application of transformer-based models like scGPT and Geneformer, which treat genes as tokens within a cell's "sentence." These models use masked gene prediction for self-supervised learning. However, Baharlouei noted that compressing this high-dimensional data into latent vectors can lead to information loss, sometimes making simpler linear models perform comparably or even better. He introduced flow matching-based models, such as PrimeFlow, as a more promising approach. These models aim to directly match data distributions rather than relying on embedding and decoding, potentially preserving more of the valuable signal within the data.

Baharlouei concluded by summarizing three key takeaways: single-cell study is vital for cellular rejuvenation and digital human initiatives; while RNA-seq is dominant, advances in other modalities are crucial; and flow matching models show superiority in capturing the nuances of single-cell data, paving the way for more accurate predictions and a deeper understanding of biological systems.