Predicting how genes are turned on or off—also known as gene expression—has been one of the toughest challenges in genetics. That’s because gene expression is controlled by many DNA elements (like promoters and enhancers) and a host of transcription factors, all interacting in complex ways. But thanks to cutting-edge deep learning technologies, scientists are finally making serious progress.

Researchers at UMC Utrecht explore how deep learning is helping researchers decode this regulatory puzzle. These advanced models can analyze massive datasets, such as those from epigenome mapping or high-throughput reporter assays, to learn the “grammar” of how genes are regulated.

Sequence-to-expression architectures and training schemes

a, A convolutional neural network architecture consists of a stack of convolutional layers: initial layers learn simple patterns (such as transcription factor (TF) motifs), whereas subsequent or deeper layers learn more complex patterns. The DNA sequence is first transformed into a one-hot encoded matrix. In the first convolution layer, a dot product is performed between a subregion of the sequence (highlighted in yellow) and a kernel. The kernel acts as a motif detector. The resulting scores are stored in a vector: higher values indicate stronger matches between the motif and the subsequence at a given position. Multiple kernels can be applied in a single layer, each learning different (partial) motifs. The second layer learns relationships between the motifs identified in the previous layer. Several convolutional layers can be stacked, ultimately combining their output with a dense layer to produce the final output. The number of convolutional layers, the size of the kernels or the number of kernels applied in each layer are hyperparameters that can be tuned. However, some of these hyperparameters align with known biological priors (for example, the size of kernels tends to be in the length range of TF binding sites). b, Sequence-to-expression models that use a transformer architecture usually start by using convolutional layers. The convolutional layers initially learn local patterns, compressing the sequence, and the self-attention layers learn more distal patterns. The output of the convolutional layers feeds into the attention layer. The attention mechanism can be thought of as a similarity matrix, whereby each element computes how relevant it is for the other elements. In this example, the interaction of the promoter and its closest enhancers is highly important for the prediction, whereas the furthest enhancer does not interact with the promoter owing to the presence of an insulator.

These models don’t just predict which genes will be active—they can also help pinpoint how specific non-coding DNA variants might affect gene function. That has big implications for understanding diseases caused by mutations outside of traditional genes. The article also discusses how these tools might be used to design synthetic DNA sequences that control gene expression in predictable ways—an exciting development for synthetic biology and biotechnology.

Of course, not all deep learning methods are equal. Their review outlines the different types of training data, the strengths of various modeling approaches, and what challenges still remain. But overall, it paints a promising picture of how artificial intelligence is transforming our ability to understand and engineer biology.