University of Turku researchers compared five statistical methods to detect differentially expressed genes between two distinct single-cell populations. Currently, it remains unclear whether differential expression methods developed originally for conventional bulk RNA-seq data can also be applied to single-cell RNA-seq data analysis. Their results in three diverse comparison settings showed marked differences between the different methods in terms of the number of detections as well as their sensitivity and specificity. They, however, did not reveal systematic benefits of the currently available single-cell-specific methods. Instead, a previously introduced reproducibility-optimization method showed good performance in all comparison settings without any single-cell-specific modifications.
- SCDE (v. 1.99.0) was downloaded fromhttp://hms-dbmi.github.io/scde/package.html
- MAST (v. 0.931) was downloaded fromhttps://github.com/RGLab/MAST
- DESeq (v. 1.20.0) is available in Bioconductor (http://bioconductor.org/packages/release/bioc/html/DESeq.html)
- Limma (v. 3.24.15) is available in Bioconductor (http://bioconductor.org/packages/release/bioc/html/limma.html)
- ROTS (v. 0.99.9) is available in Bioconductor (http://bioconductor.org/packages/release/bioc/html/ROTS.html)
ROC curves of the five differential expression methods using a bulk benchmark
Sensitivity is shown as a function of specificity with the five statistical methods in the scRNA-seq data by Islam et al. . Bulk measurements from the study by Moliner et al.  were used as the gold standard, following the comparison design by Kharchenko et al. . Areas under the ROC curves (AUC) are shown in the parentheses.
- Selection of the method had a large impact on the results.
- Single-cell-specific methods did not have systematically better performance than other methods.
- Methods developed originally for bulk RNA-seq, DESeq and Limma, were not suitable for analyzing scRNA-seq data.
- ROTS performed well in all the comparisons and MAST had good statistical properties (false positives, precision and recall) as well.