Large‑Scale Unlabeled Microscopy Images Empower a Generalizable Cell Segmentation Foundation Model for Versatile Biological Analysis

Precise cell segmentation across heterogeneous microscopy images is fundamental to a broad range of quantitative biological analysis. Although numerous promising AI models have been developed, their performance and generalizability remain largely constrained by the scarcity of labeled datasets, which demand prohibitive costs for manual annotation. Here we introduce MicroAtlas, a cell segmentation foundation model that pioneers the use of large-scale unlabeled microscopy images to overcome this critical bottleneck. We first curate MicroAtlas-2B, a dataset comprising 5 million microscopy images from 45 diverse sources, encompassing heterogeneous types of cells, tissues, stains, and imaging modalities. By generating over 2 billion cell masks in the MicroAtlas-2B dataset, we transfer segmentation knowledge from limited labeled data to massive unlabeled data via a simple-yet-effective semi-supervised learning framework, thereby yielding a powerful and generalizable cell segmentation foundation model. Scaling training data from the million- to billion-level, MicroAtlas can preserve single-cell gene expression fidelity in spatial transcriptomics analysis, distinguish subtle morphological signatures associated with distinct drug mechanisms, and resolve tumor microenvironment phenotypes in multiplexed imaging data. Our study opens a promising avenue for advancing cell segmentation in microscopy images and establishes a versatile and powerful tool for broad biological applications.

Installation

MicroAtlas is built on top of Cellpose.

Install with conda

Create a new environment:

conda create --name microatlas python=3.12
conda activate microatlas

Install Cellpose (with or without GUI):

# with GUI
python -m pip install cellpose[gui]
# without GUI
python -m pip install cellpose

Install additional dependencies:

python -m pip install segment-anything pandas openpyxl tqdm pillow

Quick start

You can download our model from huggingface and place it at 'src/microatlas/'.

Inference

Python API

from cellpose import models

# Load MicroAtlas model
model = models.CellposeModel(gpu=True, pretrained_model='./microatlas/microatlas')

# Run segmentation
masks = model.eval(img, diameter=None, channels=None, bsize=256)[0]

Cellpose GUI

MicroAtlas can be used directly in the Cellpose GUI for interactive segmentation:

1. Launch the GUI:

   python -m cellpose

2. In the menu bar, go to Models → Add model.
3. Navigate to src/microatlas/ and select the microatlas model file.
4. The model will appear in the user-trained models dropdown. Select it and click run to segment the loaded image.

Note: The GUI requires cellpose[gui] to be installed (see Installation).

napari

Alternatively, you can use MicroAtlas in napari via the cellpose-napari plugin, which provides better multi-dimensional visualization and a richer plugin ecosystem:

python -m pip install napari[all] cellpose-napari

1. Launch napari:

   napari

2. Open Plugins → cellpose-napari.
3. Set Model type to custom and point to src/microatlas/microatlas.
4. Click Run segmentation. The result appears as a labels layer that you can edit directly with napari's brush tools.

Dataset download

The MicroAtlas-2B dataset is available on Hugging Face. It contains two parts:

• Labeled datasets — 18 public cell segmentation datasets with human-annotated masks, used for supervised training.
• Unlabeled IDR datasets — 45 studies from the OpenMicroscopy IDR platform with pseudo segmentation masks, used for semi-supervised training.

Install the Hugging Face CLI

python -m pip install huggingface_hub

Download labeled datasets

The 18 labeled datasets are organized under ./src/data/, each containing .tif images with corresponding _mask.tif and _mask_flows.tif files:

data/
├── Cellpose/
│   └── train/
│       ├── img_001.tif
│       └── img_001_mask.tif
├── livecell/
│   └── train/
├── tissuenet/
│   └── train/
├── Deepbacs/
│   └── train/
├── MoNuSAC/
│   └── train/
├── MoNuSeg/
│   └── train/
├── ...                  # 18 datasets in total
└── all/
    └── test/            # evaluation benchmarks

Download from Hugging Face:

huggingface-cli download MicroAtlas/MicroAtlas-2B --repo-type dataset --include "data/*" --local-dir ./

Download unlabeled IDR datasets

The 45 IDR studies are split into images (./src/IDR_image/) and pseudo masks (./src/IDR_mask/), organized by study ID:

IDR_image/
├── idr0001/
│   ├── img_001.tif
│   └── ...
├── idr0002/
│   └── ...
└── ...                  # 45 IDR studies

IDR_mask/
├── idr0001/
│   ├── img_001_mask.tif
│   └── ...
├── idr0002/
│   └── ...
└── ...

Download from Hugging Face:

huggingface-cli download MicroAtlas/MicroAtlas-2B --repo-type dataset --include "IDR_image/*" --local-dir ./
huggingface-cli download MicroAtlas/MicroAtlas-2B --repo-type dataset --include "IDR_mask/*" --local-dir ./

Note: The full dataset is ~46 TB (after unzip). If you only need a subset, use --include with specific study paths (e.g., --include "IDR_image/idr0001/*").

Training

MicroAtlas uses semi-supervised learning with a teacher–student framework and Gram loss. To start training:

cd src
python microatlas_train.py --root ./microatlas --n_epochs 500 --learning_rate 5e-5 --batch_size 1 --ddp 1 --multi_gpu 1

Key arguments:

Argument	Default	Description
--root	./microatlas	Data and model output root directory
--n_epochs	500	Number of training epochs
--learning_rate	1e-4	Learning rate (with warm-up + cosine decay)
--batch_size	8	Batch size per GPU
--gram_weight	1.0	Weight for Gram consistency loss
--ddp	1	Use DistributedDataParallel (1) or DataParallel (0)
--multi_gpu	1	Enable multi-GPU training
--pretrained_model	None	Path to a pretrained model for resuming training
--add_noise	1	Apply noise augmentation to unlabeled data
--warmup_epochs	3	Linear warm-up epochs before cosine decay

Segmentation evaluation

Segmentation evaluation scripts for all compared models are provided under src/eval/:

cd src
# Evaluate MicroAtlas
python eval/eval_microatlas.py
# Evaluate Cellpose3
python eval/eval_cellpose3.py
# Evaluate Cellpose4 / Cellpose-SAM
python eval/eval_cellpose4.py
# Evaluate CellSAM
python eval/eval_cellsam.py
# Evaluate MicroSAM
python eval/eval_microsam.py

SPATCH benchmark

Cell segmentation evaluation on 12 spatial transcriptomics datasets (4 platforms × 3 cancer types), measured by AP@0.5. Data is organized under src/spatch/ with tile*.png images and mask*.json annotations. The datasets are from SPATCH.

cd src

# Evaluate all models (microatlas, cellpose4, cellpose3, cellsam, microsam)
python spatch/eval_spatch.py

# Evaluate specific models
python spatch/eval_spatch.py --models microatlas cellpose4

Spatial transcriptomics evaluation

Spatial transcriptomics evaluation uses three Xenium datasets from 10x Genomics. Due to their large size, the data is not included in the repository and must be downloaded separately.

Dataset	License	Link	Size
Human-pancreas	CC BY 4.0	ffpe-human-pancreas-with-xenium-multimodal-cell-segmentation-1-standard	~6.5 GB
Human-lung	CC BY 4.0	preview-data-ffpe-human-lung-cancer-with-xenium-multimodal-cell-segmentation-1-standard	~19 GB
Mouse-colon	CC BY 4.0	fresh-frozen-mouse-colon-with-xenium-multimodal-cell-segmentation-1-standard	~24 GB

Download spatial transcriptomics data

Download the *_outs directory from 10x Genomics and place it under the corresponding dataset folder:

cd src/spatial_analysis/Xenium

# Human-pancreas: ffpe-human-pancreas-with-xenium-multimodal-cell-segmentation-1-standard
#   Extract to: Human_pancreas/Xenium_V1_human_Pancreas_FFPE_outs/

# Human-lung: preview-data-ffpe-human-lung-cancer-with-xenium-multimodal-cell-segmentation-1-standard
#   Extract to: Human_lung/Xenium_V1_humanLung_Cancer_FFPE_outs/

# Mouse-colon: fresh-frozen-mouse-colon-with-xenium-multimodal-cell-segmentation-1-standard
#   Extract to: Mouse_colon/Xenium_V1_mouse_Colon_FF_outs/

Run segmentation

Each model must be run separately for each dataset. The working directory should be the repository root:

# MicroAtlas
python -u src/spatial_analysis/Xenium/Human_lung/microatlas_xenium_analysis/run_pipeline.py --model microatlas

# Cellpose4 / Cellpose-SAM (default)
python -u src/spatial_analysis/Xenium/Human_lung/microatlas_xenium_analysis/run_pipeline.py

# Cellpose3 (cyto3)
python -u src/spatial_analysis/Xenium/Human_lung/microatlas_xenium_analysis/run_pipeline.py --model cellpose3

# CellSAM
python -u src/spatial_analysis/Xenium/Human_lung/microatlas_xenium_analysis/run_pipeline.py --model cellsam

# MicroSAM (vit_l_lm)
python -u src/spatial_analysis/Xenium/Human_lung/microatlas_xenium_analysis/run_pipeline.py --model microsam

Each model takes ~1-2h on a full morphology image. Use --crop to test on a small ROI first.

Parameter	Default	Description
--model	cellpose4	microatlas / cellpose4 / cellpose3 / cellsam / microsam
--crop	off	Use cropped ROI for quick testing
--gpu	0	GPU device index (-1 for CPU)
--output	output/	Output directory

Output per model (saved under output/):

Model	Mask file	Overlay
microatlas	masks_microatlas_morphology.npy	seg_overlay_microatlas_morphology.png
cellpose4	masks_cellpose4_morphology.npy	seg_overlay_cellpose4_morphology.png
cellpose3	masks_cellpose3_morphology.npy	seg_overlay_cellpose3_morphology.png
cellsam	masks_cellsam_morphology.npy	seg_overlay_cellsam_morphology.png
microsam	masks_microsam_morphology.npy	seg_overlay_microsam_morphology.png

Repeat the above for all three datasets (Human_lung, Human_pancreas, Mouse_colon).

After all models finish segmentation, run eval.py to compute the unassigned transcript fraction (spatial bleeding) — the proportion of transcripts that fall outside mask boundaries:

python spatial_analysis/Xenium/eval.py --project all --models all

Morphological profiling

Morphological profiling evaluation uses the U2OS-Cell-Painting dataset (figshare 21378906, Uppsala University, CC BY 4.0), a 5-channel Cell Painting screen of human U2OS osteosarcoma cells treated with 231 compounds at a single 10 µM dose, plus DMSO negative controls. The compounds span 10 balanced Mechanism of Action (MoA) classes across 18 plates × 5 sites = 7,710 fluorescence fields (2160×2160, uint16; channels: DNA, Mito, AGP, RNA, ER). The two original papers using this data perform supervised MoA classification; this pipeline instead performs unsupervised morphological clustering, so the numbers here are not comparable to the papers' supervised scores. The pipeline benchmarks five segmentation models for their ability to preserve morphological signal, measured by how well downstream unsupervised clustering separates MoA classes.

References (cite when using this data):

1. Gupta, Harrison, et al. "Is brightfield all you need for mechanism of action prediction?" bioRxiv 2022. DOI 10.1101/2022.10.12.511869
2. Tian, Harrison, et al. "Combining molecular and cell painting image data for mechanism of action prediction." Artificial Intelligence in the Life Sciences, 2023. DOI 10.1016/j.ailsci.2023.100060

Download data

Download metadata and plate images from figshare. The full record is ~634 GB; each per-plate tar.gz bundles both fluorescence (FL) and brightfield (BF) data, and only the 5 FL channels are extracted.

# Download all 18 plates (metadata + per-plate FL images)
python src/morphology_profiling/download.py --all

Data is stored under src/morphology_profiling/data/: fl_data.csv in metadata/, plate images organized as images/{plate}/DNA|Mito|AGP|RNA|ER/*.tif.

Pipeline

All scripts support --models to specify one or more models (cellpose4, cellpose3, microatlas, microsam, cellsam, or all).

Step 1 — Preprocess metadata: Build the unified image table from fl_data.csv (plate/well/site, compound, per-channel filenames and the single MoA label per field; no external platemap join needed).

python src/morphology_profiling/preprocess.py

Step 2 — Segmentation: Run whole-cell instance segmentation for all fields (cytoplasm/membrane marker AGP paired with the DNA nucleus marker; single AGP marker for microsam).

python src/morphology_profiling/segment.py --models microatlas --all

Output: results/masks/{model}/{plate}/{field}_mask.tif.

Step 3 — Feature extraction: Extract morphological features per cell across 6 categories:

Category	Description
AreaShape	regionprops shape descriptors
Intensity	batched ndimage stats + percentiles × 5ch
Texture	Haralick GLCM on DNA + RNA
Granularity	multi-scale opening on DNA
RadialDistribution	binned radial intensity × 5ch
Correlation	Pearson + Manders inter-channel

python src/morphology_profiling/feature_extraction.py --models microatlas --all

Step 4 — Aggregation & normalization: Aggregate single-cell features to field-level (median + MAD), robust z-score against per-plate DMSO controls, remove low-variance (< 0.01) and highly correlated (> 0.95) features.

python src/morphology_profiling/feature_aggregation.py --models microatlas

Step 5 — Unsupervised clustering & evaluation: Field-level profiles are reduced via PCA (50 dims) then UMAP (5D, cosine distance), clustered via Agglomerative Clustering (ward linkage, k=10). Quality is measured by NMI and ARI.

python src/morphology_profiling/biomarker/unsupervised.py --models microatlas

Counting evaluation

Cell counting evaluation uses three BBBC datasets (BBBC001, BBBC039, BBBC041). Dataset zip files are included under src/counting/BBBC/:

cd src/counting

# Batch-run all five models on BBBC001 and BBBC041
python batch_counting.py
# Or run a single model or dataset
python batch_counting.py --datasets BBBC001 --models microatlas
# Available models: cellpose4, cellpose3, microatlas, microsam, cellsam

Multiplexed imaging cell phenotyping

Cell phenotyping evaluation uses the public TNBC-MIBI dataset Keren et al., 2018. The evaluation consists of three phases: (1) Segmentation & AP Assessment — generating composite RGB images from raw marker TIFs, running model inference; (2) Expression Extraction & Comparison — extracting per-cell mean intensity via regionprops, arcsinh transforming, per-patient z-score normalizing, and matching predicted to GT cells via IoU>0.5 for per-marker comparison; (3) FlowSOM Hierarchical Clustering — performing three-level immune phenotyping (immune vs non-immune, non-immune subtypes, immune subtypes) and evaluating via Hungarian-matched accuracy.

Download data

The dataset is publicly available at https://www.angelolab.com/mibi-data. Download the following and place them under src/phenotyping/:

Data	Path	Description
Raw marker TIFs	TNBC/TNBCShareData/Point{pid}/*.tif	40 protein markers per patient, 1024×1024 float32
GT segmentation masks	TNBC_shareCellData/p{pid}_labeledcellData.tiff	41 uint16 label maps (~250K cells total)
Single-cell expression matrix	TNBC_shareCellData/cellData.csv	~85 MB, arcsinh-transformed, with Group (6 classes) and immuneGroup (12 classes) labels
Patient classification	TNBC_shareCellData/patient_class.csv	Molecular subtype of each patient

Generate composite input images

For models requiring RGB input, raw marker TIFs are synthesized into 3-channel composites:

Channel	Markers summed	Biological meaning
R	Pan-Keratin + Beta-catenin	Tumor membrane/cytoplasm
G	CD45 + HLA-DR	Immune membrane
B	dsDNA + H3K27me3 + H3K9ac	Nucleus

python src/phenotyping/analysis/gen_composite_preview.py

Output: src/phenotyping/composite_preview/PointXX_composite.tif (float32) and .png (preview).

Run evaluation

Each model runs under the same 3-phase framework controlled by --phase:

--phase	Stages	Description
all (default)	1+2+3	Full pipeline: segment → express → cluster
segment	1 only	Segmentation
express	2 only	Expression extraction + GT comparison
cluster	3 only	FlowSOM hierarchical clustering (requires pred_expr/*.parquet from phase 2)

Note: FlowSOM clustering (phase 3) requires Python 3.10+. If your environment is Python 3.9, run --phase all or --phase express first, then run --phase cluster in a Python 3.10+ environment.

# MicroAtlas
python src/phenotyping/analysis/eval_microatlas_TNBC.py --gpu --weights ./microatlas/microatlas

Output per model saved under src/phenotyping/eval_results/{model_name}/:

Phase	Output files
Segment	ap_results.txt, ap_per_image.xlsx, masks/p{pid}_pred.tif
Express	expression_comparison.txt, expression_comparison.xlsx, pred_expr/p{pid}.parquet
Cluster	clustering_results.txt, clustering_results.xlsx

Acknowledgement

Our dataset is collected from the OpenMicroscopy platform. Our model is developed on the prestigious Cellpose. We highly appreciate their great efforts.