Single cell RNA sequencing allows scientists to study gene expression in individual cells, helping reveal how different cell types function and interact. However, combining datasets from different experiments can be difficult because of technical differences known as batch effects, which can make similar cells appear different.

Researchers at Yale University conducted a large scale comparison of methods designed to integrate single cell RNA sequencing datasets. In particular, they focused on semi supervised approaches, which use known cell type labels to help guide the integration process. These methods are often expected to improve results by preserving biological signals while correcting technical variation.

To test how well these approaches perform in real world conditions, the team evaluated several leading methods across six different datasets. Importantly, they did not assume perfect data. Instead, they introduced realistic challenges such as missing labels, incorrect annotations, mixed cell populations, and labels with varying levels of detail.

Schematic diagram of the benchmarking workflow

In this study, ten data integration methods, including five semi-supervised algorithms and five unsupervised baselines, are evaluated across six integration datasets. Five label scenarios are designed to reflect diverse real-world conditions, including randomly missing, randomly wrong, missing and mixing at edge, partially annotated batches, automatically generated labels and labels with varied granularity, together with the baseline setting where five unsupervised methods are used and five semi-supervised method are used with full labels presented. Integration results are assessed using 9 metrics that evaluate batch effect removal and conservation of biological variance from true cell type labels (label conservation).

The results showed that semi supervised methods can perform well when cell type labels are accurate and complete. However, their performance often declined when labels were imperfect, which is common in real datasets. In many cases, these methods did not outperform traditional unsupervised approaches that do not rely on labels.

Among the tested tools, scANVI and ssSTACAS showed relatively stable performance even when labels were not perfect, although their improvements were modest. Other methods, such as scDREAMER, performed very well under ideal conditions but were more sensitive to errors in labeling.

The findings suggest that while adding cell type labels can be helpful, it does not always lead to better results. In fact, relying too heavily on imperfect labels can introduce new biases and reduce the quality of data integration.

For researchers working with single cell RNA sequencing data, this work provides practical guidance. When high quality annotations are available, semi supervised methods may offer benefits. However, when labels are incomplete or uncertain, robust unsupervised methods may be the safer and more reliable choice.

Overall, the study highlights the importance of evaluating computational tools under realistic conditions and choosing methods based on the quality of available data, rather than assuming that more information will always improve results.