A team led by researchers from Tsinghua University has developed a novel tool for understanding RNA sequences. The team recognised that most existing computational models treat an RNA molecule simply as a string of letters (A, U, G, C), but they often ignore the way the molecule folds and forms base-pairs, and yet structure is key to how RNA works in cells.To address this, the researchers developed a new model called ERNIE‑RNA (Enhanced Representations with Base-Pairing Restriction for RNA Modeling) built on a familiar framework (BERT-style language modelling). What makes ERNIE-RNA special is that it explicitly incorporates base-pairing biases during its training, so that the “attention” parts of the model are nudged to respect which bases might pair or interact in folded RNA. The result: even before the model is tuned for any specific task, its internal attention maps already show strong ability to detect structural features of RNA, essentially it “learns” aspects of RNA architecture from sequences alone, in a zero-shot fashion. After further fine-tuning, ERNIE-RNA achieves top-tier performance across a variety of downstream tasks including predicting RNA secondary structure, contact maps, and functional annotations.

Overview of the ERNIE-RNA model architecture and application

ERNIE-RNA incorporates RNA structural information into the self-attention mechanism. a In the pre-training stage, ERNIE-RNA, consisting of 12 transformer layers, was pre-trained with 20.4 million non-coding RNA sequences from RNAcentral via self-supervised learning. b In the fine-tuning stage, ERNIE-RNA provides attention maps and token embeddings that encode rich structural and semantic RNA features, achieving state-of-the-art performance on diverse downstream tasks spanning structure prediction and functional annotation.

In plain terms: think of an RNA molecule like a piece of origami paper with folds. Many models look just at the flat pattern printed on the paper (the sequence) and ignore folds. What the research team has done is build a model that also has a sense of how the paper would fold, giving it more insight into how the final shape (structure) might influence function. For anyone working in RNA biology, computational RNA analysis, or therapeutics based on RNA, even at a basic level, this is exciting, because models that capture both sequence and structure are more likely to capture how RNA really works in the cell.

Importance

Many RNAs (especially non-coding ones) carry out their roles in part through their structure rather than just their sequence.

As RNA therapies, mRNA vaccines, and RNA-based diagnostics continue to expand, we’ll need better computational models that predict what an RNA will do, given its sequence and shape.

By showing that you can “teach” a model to implicitly capture structure through clever architectural bias, this study hints at a more general strategy: build biological prior knowledge into machine learning models rather than treating everything as pure sequence.

In short, this work adds a powerful new tool to the RNA-bioinformatics toolbox and suggests that the next generation of RNA modelling will put structure and sequence on more equal footing.

Availability – The ERNIE-RNA scripts are available at (https://github.com/Bruce-ywj/ERNIE-RNA.