FEATURE | Shortening the Diagnostic Odyssey: Utilizing ML and Attention-Based AI for the Advancement of Genetic Disease Diagnostics

Families who are trying to solve the mystery of rare genetic disorders often have to endure a long journey before the diagnosis: a journey that doctors refer to as the “diagnostic odyssey.” The patients visit different doctors and must perform repeated tests before waiting for the genetic sequencing result, which often fails to explain the disorder despite the long search. Recently, however, a potential bypass into this labyrinth has come via the use of artificial intelligence. This, however, relies not on mere automation but on a mathematical toolbox full of statistics and matrices that looks for clues hidden within vast datasets.

Behind the success of most current diagnosis platforms lies a learning architecture from the “transformer” family of models launched in the research paper “Attention Is All You Need” in 2017. Unlike the architecture used by earlier models of artificial intelligence, the “transformer” architecture uses a mechanism known as “self-attention” to determine the relative importance of a piece of information in relation to the rest of the data a customer would be dealing with. In essence, attention gives the system the ability to determine the importance of symptoms, genes, or the interaction of different symptoms in a way not very different from what a human expert would do.

Two recent scientific articles evaluate how effectively the mathematical framework described here can be applied in real-world gene diagnosis. In the first study, the authors questioned whether the use of large language models (LLMs) composed exclusively of transformer models can ensure the prioritization of genes responsible for the disease based solely upon clinical descriptions. The second article proposes GestaltMML, a multimodal transformer model capable of integrating facial characteristics, demographic information, and clinical information into a unified predictive framework. Both articles, it seems, investigate not whether AI models can chat about medicine but rather whether attention-based models can reason.

Rare genetic disorders in total affect more than 6% of the global population, while an individual disease might be rare. Usually, there are only a few symptom characteristics to be matched with thousands of possible mutations of genes. The existing computer programs, such as Phenomizer, Exomiser, and AMELIE, would solve this task by comparing patient symptom characteristics with graphs of genes to diseases. This worked quite well, but was not very flexible if the data was not complete or not clear.

The first study, conducted by the Weng Lab of Columbia University, examined whether a transformer-based language model could look past these limitations. To test this, researchers fed models, such as GPT-4 and LLaMA, free-text descriptions of patients’ conditions and asked them to rank genes for potential diseases in order of probability. From a technical level, one could describe this as taking text input and translating it into high-dimensional vectors and using attention matrices that look at statistical patterns in the relationships between symptoms and genes from past models. What came out, however, highlighted an obvious flaw in the system. GPT-4 correctly identified the disease gene in its top 50 guesses a mere 17% of the time, which is much lower than what current systems are capable of, and even demonstrated an obvious bias towards highly mentioned genes in texts, consistently picking widely-known genes like BRCA1.

The second study, performed by the Wang Lab of the University of Pennsylvania, illustrates the effect of attention mechanisms when trained on a variety of data sources. In this case, GestaltMML can combine the transformer model with images of faces, as well as other clinical variables, in a single mathematical framework. The attention mechanisms are then used to understand how images of faces affect clinical text, or how age and ethnicity affect a particular disease. The multimodal model performed much better in diagnoses, particularly when the faces were subtle or ambiguous. Notably, this clinical text input helped to mitigate a common concern regarding imbalanced datasets used in image-recognition models, in which diagnostic accuracy in some ethnic groups was depressed.

Nonetheless, these developments also underscore the problems mentioned above. Transformer models are indeed powerful, but data-intensive and dependent on the quality of the output probability distributions they are trained and tested on. Moreover, the data may contain unbalanced demographics that are directly encoded within the model’s attention weight, and the model's ensuing mechanisms are also opaque. Finally, the clinician's goal of speed is important within the healthcare domain because the consequences of the model's opacity are potentially irreversible.

For these reasons, many people have argued that the role of AI should be to assist with the mathematical calculations rather than provide a medical opinion. The geneticist would remain far superior to computers regarding the challenging situations that are unclear and the new syndromes that involve making inferences. The benefit of machine learning comes into place while navigating massive solution spaces, pointing the computer to the variables that need examination.

However, if leveraged properly, the potential is profound. Attention-based models might decrease the time spent on diagnosis, focus on genetic analysis, or enable doctors to think more systematically. This move towards multimodal transformers indicates the growth of medical AI, which is driven not by the ease of chatbot interactions but by mathematical expression.

The time is not yet ripe for artificial intelligence to make a diagnosis of rare genetically transmitted disorders. However, as attention-based systems develop vision, language, and biological knowledge, it may soon be time to embark on a diagnostic odyssey. The most important innovations in medicine will not come from a machine replacing a physician, but rather a marriage between mathematics and medical expertise.