Group reports the results of phase 1 of the ABRF-NGS study at ABRF 2014

The ABRF Next Generation Sequencing Study: Multi-platform and Cross-methodological Reproducibility of Transcriptome Profiling by RNA-seq

Next generation sequencing (NGS) has dramatically expanded the potential for novel genomics discoveries, but the wide variety of platforms, protocols, and performance has created the need for reference data sets to understand the sources of variation in results.

The goals of the ABRF-NGS Study are to use standard references to evaluate the performance of NGS platforms and to identify optimal methods and best practices. For the first phase of this study, over 20 core facility laboratories performed replicate RNA-seq experiments, using titrated reference RNA standards and a set of synthetic RNA spike-ins, evaluated over a wide range of methods: polyA-enriched, ribo-depleted, size-specific fractionations, and degraded RNA, on six NGS platforms (Illumina HiSeq 2000/2500 and MiSeq, Life Technologies PGM and Proton, Roche 454 GS FLX+, and PacBio RS). Two RT-qPCR data sets were used as orthogonal tools to gauge the RNA-seq results.

The ABRF NGS group reports:

  • High intra-platform consistency and inter-platform concordance for expression measures.
  • Highly variable rates of efficiency and costs for splice isoform detection between platforms.
  • Ribosomal RNA depletion can both salvage degraded RNA samples and be readily compared to polyA-enriched fractions.
  • Comparisons of alternative aligners for each platform show that algorithm choice affects mapping rates and transcript coverage more than gene quantification.
  • Surrogate variable analysis (SVA) proved to be an optimal method to combine data within and between platforms, increasing sensitivity and reducing false positives by over 90%.

Taken together, these data represent a broad cross-platform characterization of RNA standards and provide a comprehensive comparison of results from degraded, full-length, and size-selected RNA across the latest NGS platforms. The next phase of this study is focusing on use of DNA reference standards. Results of the ABRF-NGS Study provide a broad foundation for cross-platform standardization, evaluation, and improvement of NGS applications.

The ABRF Next Generation Sequencing Study: Multi-platform and Cross-methodological Reproducibility of Transcriptome Profiling by RNA-seq

Christopher E. Mason1, Sheng Li1, Scott Tighe2, Charles Nicolet3, Don Baldwin4, George Grills4, the ABRF-NGS Consortium1

  1. Weill Cornell Medical College
  2. University of Vermont
  3. University of South California
  4. Pathonomics, LLC
A complete version of the findings of this study will be published in Nature Biotechnology shortly.
The ABRF-NGS group has created an online community resource for self-evaluation and self-improvement that allows users of NGS technologies to readily compare and improve their own performance data and to evaluate existing and emerging protocols and platforms. You can access the resource at:

Find out more at: