Nicheverse#

Interpretable modeling of tissues

Every cell and niche mapped to an interpretable codebook.

samples
datasets
cells mapped
tissues
platforms

A clear cell renal cell carcinoma primary tumor core read into cell-state lineages

A whole mouse pup with every cell painted by its learned cell-state code

Interpretable by construction

A discrete code for every cell and niche.

Each cell is assigned to one entry of a learned cell state codebook, and each neighborhood to one entry of a learned niche codebook. The two are coupled by cross attention, so a cell's identity is always read in the context of the tissue around it. The same discrete vocabulary transfers, unchanged, from one cohort and platform to the next.

learned cell state codebook learned niche codebook cross-attention coupled

How it works

Two encoders, two codebooks, one gated bridge.

A cell encoder summarizes each segmented cell, together with the transcripts detected in and around it, and assigns it to the closest entry of a learned cell state codebook. A niche encoder summarizes the local neighborhood, weighting nearby cells more than distant ones, and assigns it to an entry of a learned niche codebook. A gated cross attention step lets each cell draw on its niche, and two decoders are trained to reconstruct the cell's gene expression and the composition of its neighborhood, using count aware losses. Because both codebooks are discrete and shared across the tissue, every code can be compared across samples.

Nicheverse architecture: nuclear transcripts plus segmentation free transcript context feed a cell encoder and a learned cell codebook, distance weighted niche context feeds a niche encoder and a learned niche codebook, a gated cross attention block refines each cell by its niche, and cell and niche decoders reconstruct expression and neighborhood structure, giving spatially aware annotation of cell types and niches

Recovered, not supervised

An interpretable vocabulary of cell states.

When the learned code embeddings are clustered by their similarity, they group into coherent lineages without any labels being supplied: epithelial, stromal, endothelial, and immune codes separate on their own. Each code has a consistent expression signature, so it can be annotated once and recognized across samples and tissues.

Hierarchically clustered correlation of the learned cell state codes, colored by dominant cell type

Explore the atlases mapped in the nicheverse

Explore samples from many independent datasets and imaging platforms, including Xenium, CosMx, MERFISH, RAEFISH, seqFISH, STARmap, osmFISH, RIBOmap, EEL FISH, and BARISTAseq. Each cell is drawn as its nucleus and colored by the cell type of the code Nicheverse assigns it, with tumor cells shown in red.

Browse all samples

Nicheverse

Neighborhood-Inferred Cell type HiErarchical annotation + VEctor-quantized Representations of Spatial Ecotypes

Hierarchical codebooks

Paired codebooks for cell states and spatial niches, linked by cross attention, so each cell is interpreted in the context of the tissue around it.

Swappable components

Encoders and quantizers are modular and interchangeable, so the architecture can be adapted to new data types, panel sizes, and experiments without changing the rest of the pipeline.

Spatial-aware

Neighborhoods are built within each sample, nearby cells are weighted more than distant ones, and several ways of defining spatial neighbors are supported.

Install

pip install nicheverse

Citation

If you use Nicheverse, please cite:

Yarlagadda DVK, Wang Z, Jiang H, Vuong L, López-Sanmiguel A, Yang C-Y, Kotecha RR, Chen Y-B, Hakimi AA, Leslie CS, Massagué J. Developmental reversion underlies resistance to immune checkpoint blockade in kidney cancer. bioRxiv 2026. doi:10.64898/2026.08.05.743137.
Yarlagadda DVK, Massagué J, Leslie C. Discrete Representation Learning for Modeling Imaging-based Spatial Transcriptomics Data. 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 2023, pp. 3848-3857. doi:10.1109/ICCVW60793.2023.00416.