How should cell prediction models be validated?

Prepared by:AI Virtual Cell · Published / updated:2026-10-31

Key takeaway

Validate cell prediction models against their intended use, with appropriate data separation, simple baselines, multiple readouts and independent tests. State the conditions where the model is not applicable.

Define the generalization problem

Performance on new cells and performance on new perturbations are different questions. Before splitting data, define the intended setting: new donors, cell types, compounds or multiple changes at once.

Prevent leakage before evaluation

Sharing samples, donors or highly similar conditions between training and test sets can inflate performance. Splits must match the task, and preprocessing and parameter selection must not use test-set information.

Always compare simple baselines

Compare complex models with control means, condition means or other suitable simple methods. Additional complexity should provide consistent value before it becomes a basis for further research.

Match metrics to the biological question

Global expression similarity, differential-gene recovery, pathway direction and phenotype readouts measure different things. For intervention tasks, global similarity alone can obscure weak prediction of the actual response signal.

Report external validation and limitations

Retain data provenance, model versions, condition ranges and known failure cases. Cross-laboratory, cross-batch or new-context validation tests robustness. Report uncovered scenarios rather than presenting uncertain outputs as established conclusions.

This article explains research methods; it does not report a model release or experimental results from this website. Research decisions require task-specific data and validation evidence.

Sources and evidence

Assess uncertainty at the biological sample or replicate level. Report variation across splits and cell contexts; do not treat individual cells from one sample as independent biological replicates. A non-significant difference does not establish equivalence.

In the tested datasets and tasks, evaluated deep learning models did not outperform simple baselines; this does not establish failure of all models on all tasks. Nature Methods · Perturbation prediction and simple baselines ↗

Evaluation across datasets, perturbations and cellular contexts supports task-specific model selection. Nature Methods · Generalizable perturbation prediction ↗