Understanding the complex mix of cell types in tissues is a major challenge in biology, especially when working with single-cell RNA sequencing (scRNA-seq) data. This type of data gives scientists detailed insight into gene expression at the single-cell level, but analyzing it is no easy task. One of the toughest parts is figuring out what type of cell each data point represents—a process known as cell type annotation.

Researchers at the University of Toronto have developed a new tool called scTrans to make this process faster and more accurate. Traditional annotation tools reduce the complexity of scRNA-seq data by focusing only on the most active (highly variable) genes. While this simplifies computation, it can also leave out valuable information.

scTrans takes a smarter approach. It uses a machine learning method called a Transformer model—a type of AI often used in language processing—and adapts it for biology. Instead of ignoring less variable genes, scTrans uses “sparse attention” to focus only on genes that are actually expressed in each cell, which helps preserve important biological details while reducing computational load.

Overview of scTrans

(a) Sparse attention aggregates cellular representation: scTrans leverages a sparse attention mechanism to efficiently encode non-zero genes into cellular representations. We assign each gene an embedding based on its gene symbol and only use the embeddings corresponding to genes with non-zero expression values, aggregating these embeddings through attention weights, enabling focused learning on informative genes. (b) Contrastive pre-training strategy: During pre-training, scTrans generates augmented cells through random masking, creating positive pairs with the original cells and negative pairs with other cells in the batch. Features for contrastive learning are extracted via an encoder-projection architecture. This process pulls similar positive pairs closer and pushes negative pairs apart in the latent space, facilitating unsupervised pre-training. (c) Fine-tuning for cell type classification: In the fine-tuning phase, a classification layer is appended after the latent representation layer, enabling supervised learning for cell type classification using labeled data. Model parameters are optimized accordingly. This optimization is achieved through supervised learning using labeled data. (d) Applications in downstream tasks: Trained scTrans can be deployed for cell type annotation on novel datasets, as well as for downstream tasks such as gene expression analysis, clustering or cell trajectory inference.

When tested on large-scale datasets from the Mouse Cell Atlas, scTrans was able to efficiently analyze and correctly identify cell types, even in datasets with close to a million cells. Not only was it fast and accurate, but it also worked well on new, unseen data, showing strong generalization power. The model even produced high-quality representations of the data that are useful for deeper analyses like clustering and mapping how cells change over time.

This innovative tool could be a game changer for researchers studying complex tissues or diseases using scRNA-seq, helping them get faster and more reliable results—without needing high-end computing resources.

Availability – the source code of scTrans is available at Github https://github.com/zou-zhiyi/scTrans.