Researchers are constantly looking for better ways to understand how genes function in the context of actual tissue structure. A new AI-powered tool developed by researchers at the Houston Methodist Research Institute aims to do just that. The tool, called OmiCLIP, connects gene expression data from RNA sequencing with histology images of tissues stained using the common hematoxylin and eosin (H&E) method. This fusion of data helps scientists gain a much more complete picture of how genes are being used across different regions of tissue and how this relates to disease processes.

To train OmiCLIP, the researchers used a massive dataset of 2.2 million tissue image and gene expression pairs from 32 different organs. Instead of just feeding raw numbers into the model, they converted the transcriptomic data into “sentences” by listing the most highly expressed genes in each small patch of tissue. This made it easier for the AI to link the visual features of each tissue section with the underlying gene activity. The model essentially learns how different tissues “look” and what gene expression patterns are associated with those appearances.

Overview of the study

a, The workflow of pretraining the OmiCLIP model with paired image–transcriptomics dataset via contrastive learning. b, Workflow of the Loki platform using the OmiCLIP foundation model as an engine. Left diagram illustrates the size of the training data in different organs. Right diagram lists the existing modules of the Loki platform, including tissue alignment, cell-type decomposition, tissue annotation, ST gene expression prediction and histology image–transcriptomics retrieval. c, The heat map represents image embeddings and transcriptomic embeddings similarity across various organs and disease conditions. The color of the heat map reflects the OmiCLIP’s embedding similarities, with red indicating high similarity and blue indicating low similarity. HCM, hypertrophic cardiomyopathy; HBV, hepatitis B virus infection. d, Schematic illustration of Loki platform with transfer learning for 3D tissue analysis. 

The researchers didn’t stop with OmiCLIP. They built a full platform on top of it, called Loki, which performs five key functions that researchers and clinicians can use:

• Tissue alignment – matching tissue images to corresponding transcriptomic data.
• Annotation – labeling tissue types or structures using bulk RNA sequencing or known marker genes.
• Cell-type decomposition – figuring out what types of cells are present in each part of the tissue.
• Image–transcriptomics retrieval – searching for gene expression profiles based on tissue image features.
• Prediction of spatial gene expression – estimating gene activity directly from an H&E-stained image without needing additional molecular analysis.

When tested against 22 state-of-the-art models across five simulation datasets, 19 public datasets, and four in-house experiments, Loki consistently showed strong performance and reliability. These tools are especially useful in areas like cancer diagnostics, where understanding both gene expression and tissue architecture is critical for accurate diagnosis and treatment planning.

By combining visual and molecular data in a single framework, OmiCLIP and Loki mark a major advancement in how scientists and pathologists can study tissue samples. They not only streamline complex analyses but also make it possible to extract rich gene expression data from routine pathology images, potentially reducing the need for extra laboratory testing.

Availability – Loki is implemented in Python and is available via https://github.com/GuangyuWangLab2021/Loki/. The pretrained OmiCLIP weights are available via https://huggingface.co/WangGuangyuLab/Loki/.