Transforming Attention: AI Architecture That Can Help Reduce Diagnostic Bias
#Artificial Intelligence (#AI) has advanced very rapidly, and so has its popularity. We are in the Wild West period of innovation, with its unregulated chaos. Though dangerous, this is the most critical period of growth for any new technology. This freedom and popularity have created an environment with vast amounts of feedback, which will influence further development and shed light on the product's perception.
The more I observe #AI's evolution, the more I believe its proper place is within the science realm, specifically #healthcare. The potential here is remarkable, but there are a few roadblocks regarding ethics that we must overcome.
My personal goal for ethical AI isn't just to create an environment that we already have, but to use this opportunity to fix humanity's mistakes that currently hinder healthcare.
To establish a usable model that can be trusted in diagnosing, the issue of bias must be addressed. Bias has many facets, but typically, when most people see the word, they associate it with gender, race, or age.
When a company wants to hire employees ethically, they eliminate gender, race, and age from the decision equation and try to let the candidate's experience speak on their behalf.
Unfortunately, this is not a viable method in healthcare because, in order to have an accurate diagnosis and overall view of a patient's health, all of the patient's data must be included. The more complete the patient profile is, the easier it is to create an ideal testing environment for a provider to use. (This is the basis of a digital twin.)
What causes bias is also what defines a person.
Let's look at an example of how bias in statistics could lead to an incorrect diagnosis.
Let's say there is a disease (Disease One) that affects more people of a certain race (Race One).
Disease One affects about 100,000 people in the United States; more than 90% of those affected are Race One, and 3%-9% are Race Two.
So what if Patient One is undiagnosed and has Disease One but is Race Two?
Patient One goes to her doctor to present her symptoms, but because of the low probability of her race having Disease One, there is a lower chance that she will be diagnosed correctly. The process result would be the same in an AI model.
How Do We Fix This?
Removing race would allow the symptoms to stand alone, and a correct diagnosis would be closer to the top of the probability list, but that only works in this case and is not a sustainable solution.
If we create an AI environment where probability is lessened and replaced with a creative output for more challenging and rare diagnoses, this process would allow models to act like humans and use critical thinking and creativity to solve complex cases.
Can AI think outside the box?
Yes.
AI models can use hyperparameters within the Transformer architecture to control the creativity of the output, which is called Transformer Temperature.
When a standard model containing a transformer generates data, the AI assigns probability to the potential output, and the temperature parameter can change how the probability is weighed.
Let's assume we have a decent AI model tailored for assisting providers with diagnosing, which only pulls its data from PubMed.
The doctor inputs Patient One's symptoms and history; the results will be the most probable, statistically, given the information. That is defined as having a low temperature.
As the temperature increases, the probability evens out.
If the doctor wants a more creative but less probable diagnosis suggestion, they can increase the transformer temperature.
How is this possible?
A breakdown of the AI transformer
First represented in Attention Is All You Need, a groundbreaking paper that introduced the Transformer. The Transformer enhances the Large Language Model’s (LLMs) Self-Attention mechanism. It removes the need for recurrent or Convolutional Neural Networks and only uses the Attention Mechanism for input and output dependencies.
The Transformer revolves around the Attention Mechanism. The Attention Mechanism allows the model to determine what data is the most relevant when processing an answer. It prioritizes information through prediction, and it assigns relevance by placing different weights to different portions of the input.
A simplified explanation of the Transformer's function:
Preprocessing
The input text is preprocessed through a method called Tokenization. Tokenization breaks down the text into smaller sections of words or subwords, which are called tokens.
Each word (token) is given a numerical value so the model can process the words as numerical representations, essentially a transliteration, for a mechanism that can only process numbers.
The numbers assigned to the Tokens are called Token IDs.
The Token IDs are passed to the encoder, and vector embedding occurs.
Vector embedding assigns the relevance and context representations to the tokenized sentence (semantics).
But what if the same word has a different position or meaning based on context?
That’s where Positional Encoding comes into play.
Positional Encoding adds a sequence of predefined vectors to the embedded vectors of each word. This provides more numerical representations of the words that establish their positions and context in the sentence. Then the numbers are taken and converted into a probability distribution or Softmax.
If the probability distribution needs to be changed, this is done through the temperature value (T) before it goes through the Softmax function.
When the temperature increases, the probability becomes more uniform, and as the temperature decreases, the distribution is based on relevance to the input, so it is the more traditionally accurate option.
Graph Source
As you can see, temperature could be used effectively in healthcare without compromising the accuracy of the patient's profile. Temperature can also be used in calculating treatment outcomes and all the potential possibilities.
AI transformers can help revolutionize healthcare by processing complex medical data, generating new insights, and streamlining operations. There is even a type of transformer that processes images.
Temperature is just the beginning of tailoring AI models to assist in more intricate healthcare tasks. Transformers have even been adapted to analyze images, through the Vision Transformer (ViT), which I will get into on another day. Though these processes aren't without their flaws, they are showing more potential and accuracy than humans ever could.
