Benchmarking algorithms for pathway activity transformation of single-cell RNA-seq data

Biological pathway analysis provides new insights for cell clustering and functional annotation from single-cell RNA sequencing (scRNA-seq) data. Many pathway analysis algorithms have been developed to transform gene-level scRNA-seq data into functional gene sets representing pathways or biological processes. Researchers from Wenzhou Medical University collected seven widely-used pathway activity transformation algorithms and 32 available datasets based on 16 scRNA-seq techniques. They proposed a comprehensive framework to evaluate their accuracy, stability and scalability. The assessment of scRNA-seq preprocessing showed that cell filtering had the less impact on scRNA-seq pathway analysis, while data normalization of sctransform and scran had a consistent well impact across all tools. The researchers found that Pagoda2 yielded the best overall performance with the highest accuracy, scalability, and stability. Meanwhile, the tool PLAGE exhibited the highest stability, as well as moderate accuracy and scalability.

An evaluation framework for benchmarking pathway activity score (PAS) calculators

The seven widely applied PAS inference algorithms were assessed on 32 well-defined benchmark data sets. These algorithms combined prior knowledge (biological pathways or functional gene sets) with a  statistic  method  to  aggregate  gene-level  matrix  into  PAS-level  matrix.  The  accuracy  (take  into account  three  downstream  applications),  stability  of  results  (in  the  presence  of  dropout  events  and across  technologies),  and  scalability  (running  time  and  memory  usage)  were  used  to  systematically evaluate these algorithms.