Survey Notes · AI for Science
From Pixels to Perturbations: A Short Tour of AI for Spatial Biology
Notes from a survey of pathology foundation models, spatial transcriptomics, and the first wave of biological world models.
Biology is organized across scales. Molecules interact, cells settle into states, cells arrange themselves into tissues, and tissues determine patient outcomes. If we want AI to be useful here, it has to do three things: understand the principles and constraints that link those scales, model the multiscale, multimodal, and hierarchical structure of biology, and infer what we cannot measure directly, such as gene expression from morphology, cell-cell communication (CCC) from spatial omics, or the mechanisms behind a disease.
This post condenses a survey I put together this month on where AI for spatial biology stands. It moves from tissue images, to spatially resolved molecular data, to models that try to simulate how cells respond when we perturb them.
Structure is not the whole story
Large models for protein structure and sequence, such as AlphaFold, Evo, and ESM3, are already remarkably capable. But a correct structure does not tell you which target to drug or in which patients to drug it, and therapies still fail because they hit the wrong protein or are tested in the wrong population. Complex diseases rarely reduce to a single gene or protein. They emerge from genetic, cellular, and tissue-level processes that differ across patients. For drug discovery, we need models that understand disease biology at the level of the patient, and that means learning from tissue. Early world models such as OCTO, a multimodal transformer trained on a broad and diverse oncology dataset, point in this direction.
Four windows into tissue
Spatial biology offers several complementary measurements, and each makes a different trade-off between scale and molecular detail.
Cell and tissue morphology. Millions of routine clinical slides with strong hierarchical structure, but weak molecular supervision.
Spatially localized protein signal with explicit marker identity. Usually low- or medium-plex, with differing protocols and antibodies.
Spatially resolved RNA expression that gives dense molecular supervision. Coverage, resolution, and batch effects vary strongly by platform.
Spatial maps of metabolites, lipids, peptides, and drugs, useful for metabolism and drug response. Small datasets, and molecule identity is not always certain.
The pattern is clear: the most abundant data carries the least molecular information, and the most informative data is the smallest and noisiest. Much of the field is about borrowing strength from one modality to compensate for another.
Pathology foundation models: from patches to slides
The goal of pathology foundation models is to learn general-purpose representations from large-scale histology. They vary along two axes: whether they learn from images alone or from images paired with text, and whether they operate on patches and regions of interest or on whole-slide images (WSIs).
PLIP [1] asked whether pathology discussions shared on the internet could supervise a useful vision-language model. They could, but the text is noisy and the model never sees whole-slide context. CONCH [2] showed that stronger language supervision yields a general model for zero-shot classification, retrieval, segmentation, and captioning, though it still works at the patch level.
UNI [3] and Virchow [5] took the image-only route, testing whether scale alone produces a universal morphology encoder. The resulting embeddings are strong and clinically robust, but there is no language and no native slide-level representation. Prov-GigaPath [4] tackled whole-slide context directly, producing contextualized tile and slide embeddings efficiently, with multimodality as a secondary concern. TITAN [6] brings the threads together: a morphology-language model with transferable, zero-shot WSI representations. Its bottlenecks are reliance on synthetic captions and the compute cost of full slides.
The arc runs from patches to slides and from pixels to language, but every one of these models stays within histology. None of them sees the molecules.
Spatial transcriptomics: from spots to neighborhoods
Spatial transcriptomics (ST) fills that gap by measuring gene expression in place. The relevant foundation models fall into roughly three groups.
Cell and gene language models. Geneformer [7] is a rank-based transformer that learns gene network topology and regulatory hierarchy without overfitting to absolute expression values. scGPT [10] learns gene-gene interactions and regulatory syntax, with strong results on cell type annotation and integration. CellPLM [9] moves from gene-level to cell-level tokens, which helps with spatial imputation and modeling communication between cells. Nicheformer [15] learns enough about spatial context to predict, zero-shot, where dissociated cells came from.
Paired datasets. HEST-1k [11] and STimage-1K4M [12] each provide more than a thousand samples pairing histology images with ST profiles. They made the next group possible.
Histology to expression. BLEEP [8] uses bimodal contrastive learning to generate expression profiles directly from H&E images. STEM [14] treats the task as conditional diffusion, and STFlow [13] uses flow matching over the whole slide to model the joint distribution of gene expression, which captures interactions between cells. MCToGene [16] adds many-body attention with hierarchical coupling to model higher-order multi-cell interactions.
Two trends stand out: prediction is becoming generative rather than regressive, and the unit of modeling is moving from independent spots to interacting neighborhoods.
Biological world models: from snapshots to dynamics
Most of the models above describe a tissue as it is. World models aim to predict how a biological system evolves, especially under genetic or chemical perturbation. The current examples span a wide range of designs.
- GeneJEPA [17] applies the Joint-Embedding Predictive Architecture to ST, yielding highly transferable cell embeddings for downstream annotation.
- VCWorld [18] combines knowledge graphs, retrieval-augmented generation, and LLM-based causal reasoning to predict drug-induced expression changes and explain the mechanisms behind them.
- Lingshu-Cell [19] uses masked discrete diffusion, classifier-free guidance, and biological priors to generate transcriptome-wide cell states and simulate responses to genetic and drug perturbations.
- Quine [20], an ongoing effort from Microsoft, learns jointly across genomic, proteomic, chemical, transcriptomic, and imaging data, and pairs that model with reasoning agents and automated wet-lab feedback to predict cell state transitions.
Taken together, these models trace a path from representation, to reasoning, to generation, to closed-loop experimentation.