Performance of RNA-seq quantification pipelines is “generally poor”

Obtaining RNA-seq measurements involves a complex data analytical process with a large number of competing algorithms as options. There is much debate about which of these methods provides the best approach. Unfortunately, it is currently difficult to evaluate their performance due in part to a lack of sensitive assessment metrics.

Using two independent datasets, a team led by researchers at the Dana-Farber Cancer Institute assessed seven competing pipelines. Performance was generally poor, with two methods clearly underperforming and RSEM slightly outperforming the rest.

ROC curves indicating performance of quantification methods based on differential expression analysis of a an experimental dataset and b a simulation dataset. Seven quantification methods are shown. FP false positive, TP true positive

Availability – A series of statistical summaries and plots to evaluate the performance in terms of specificity and sensitivity is available as a R/Bioconductor package: http://bioconductor.org/packages/rnaseqcomp

A webtool is available that permits users to submit other competing methods: http://rafalab.rc.fas.harvard.edu/rnaseqbenchmark