With the goal of ensuring that single-cell RNA sequencing, a current focus of intense research, makes use of the best possible methods, an international group has benchmarked 13 different methods. The group, led by Holger Heyn of the Centro Nacional de Análisis Genómico (CNAG-CRG) in Spain, found that the Quartz-seq2 method, developed by a team in the RIKEN Center for Biosystems Dynamics Research, was overall the best method developed to sequence single-cell RNA. The research was published in Nature Biotechnology.

The group began the project based on the concern that in the past, lack of benchmarking on methods used for genomic analysis has given rise to problems later in the process, as different groups were using different methods that had varying standards and came up with different results. With this in mind, a number of groups working on the analysis of single cell RNA came together to evaluate different methods to ensure that there is good reproducibility.

Single-cell DNA sequencing is seen as the next major project in genomic research. Initially, genomic research, exemplified by the Human Genome Project, sought to determine the DNA sequence that is found in all the cells in any organism. But what makes things complicated is that although the cells in an organism share the same DNA code, in fact the cells are all phenotypically different because different genes are expressed or not expressed based on epigenetic factors. There are enormous differences in the expression of genetic regions known as promoters and enhancers, which do not code proteins directly but act on other genetic regions. Understanding the genetic makeup of individual cells would make it possible to identify how individual cells differ in conditions such as cancer and how cells change during the development process. Currently, researchers participating in the Human Cell Atlas are working to develop a comprehensive atlas of gene expression in different cell types.

To make the comparison, the group used the 13 methods to analyze a set of approximately 3,000 cells selected to fulfill four conditions: it included a variety of cell types, some of the cells were very similar, with only subtle differences in gene expression, the cells had trackable markers, and they included cells from different species. The cells were mostly human peripheral blood cells and mouse colon cells, but also included a small set of dog cells.

The reference sample consists of human PBMCs (60%), and HEK293T (6%), mouse colon (30%), NIH3T3 (3%) and dog MDCK cells (1%). The sample was prepared in one single batch, cryopreserved and sequenced by 13 different sc/snRNA-seq methods. Sequences were uniformly mapped to a joint human, mouse and canine reference, and then separately to produce gene expression counts for each sequencing method.

The methods were evaluated based on how precisely they could detect cell profiles and marker expression. The group evaluated the methods using six key metrics: gene detection, overall level of expression in transcriptional signatures, cluster accuracy, classification probability, cluster accuracy after integration, and mixability. These metrics were selected to compare the methods in terms of their accuracy, applicability to various cell types, ability to distinguish between closely related cell types, ability to produce reproducible profiles, ability to detect population markers, compatibility with other methods, and have good predictive value for cell mapping.

As a result, they found that the Quartz-seq2 method, developed by researchers at RIKEN in Japan, was particularly accurate, scoring highest on the benchmark. According to Itoshi Nikaido, leader of the group that developed the method, “We were very happy that our method was selected as overall best, and plan to further improve it so that we can achieve the best results in projects such as the Human Atlas Project.”

According to Dr. Heyn, “The protocols showed profound performance differences and we hope that our work contributes to developing standards and guidelines towards the production of high-quality datasets for the Human Cell Atlas and broader single-cell community.”

Source – RIKEN