Benchmarking of RNA-Seq analysis workflows

RNA-sequencing has become the gold standard for whole-transcriptome gene expression quantification. Multiple algorithms have been developed to derive gene counts from sequencing reads. While a number of benchmarking studies have been conducted, the question remains how individual methods perform at accurately quantifying gene expression levels from RNA-sequencing reads. Ghent University researchers performed an independent benchmarking study using RNA-sequencing data from the well established MAQCA and MAQCB reference samples. RNA-sequencing reads were processed using five workflows (Tophat-HTSeq, Tophat-Cufflinks, STAR-HTSeq, Kallisto and Salmon) and resulting gene expression measurements were compared to expression data generated by wet-lab validated qPCR assays for all protein coding genes. All methods showed high gene expression correlations with qPCR data. When comparing gene expression fold changes between MAQCA and MAQCB samples, about 85% of the genes showed consistent results between RNA-sequencing and qPCR data. Of note, each method revealed a small but specific gene set with inconsistent expression measurements. A significant proportion of these method-specific inconsistent genes were reproducibly identified in independent datasets. These genes were typically smaller, had fewer exons, and were lower expressed compared to genes with consistent expression measurements. The researchers propose that careful validation is warranted when evaluating RNA-seq based expression profiles for this specific gene set.

Each workflow (or workflow group) has specific non-concordant genes,
which are reproducible identified in independent datasets


(A) Venn diagrams showing the overlap between the non-concordant genes with ∆FC < 2, non-concordant genes with ∆FC > 2 and non-concordant genes with opposite direction. (B) Examples of workflow-specific non-concordant genes. (C) Overlap of the non concordant genes with a ∆FC > 2 between two independent datasets. The p-values (Fisher Exact test) represent the significance of the overlap.

Everaert C, Luypaert M, Maag JLV, Cheng QX, Dinger ME, Hellemans J, Mestdagh P. (2017) Benchmarking of RNA-sequencing analysis workflows using whole-transcriptome RT-qPCR expression data. Scientific Reports 7(1): 1559. [article]

Leave a Reply

Your email address will not be published. Required fields are marked *


Time limit is exhausted. Please reload CAPTCHA.