AI Bias #1 : Healthcare

July 19, 2022


Sharon Cohen

It seems that all AI is biased nowadays. This problem does not spare the healthcare industry. In fact, it is even a particular case as the consequences of AI errors are far greater than rejected loans, inaccurate polls or unfair insurance policies. Not that these are to be overlooked, we are talking about urgent issues to be solved, but, in comparison, a misdiagnosis due to a biased algorithm can have actual dramatic or even fatal consequences.

Algorithmic biases

The prediction is that AI in Healthcare will represent $36.15 billion by 2025.

Out of these AI applications to health, learning algorithms are our main concern. Being trained on existing data, these algorithms integrate existing societal biases in their decision-making. This can then take the shape of disadvantaging minorities based on gender, age or race, for example.

The three main challenges of the industry can be summarised in the following graphic:

  1. Lack of data on certain groups
    algorithms are trained on data that includes small sample sizes of certain groups which leads them to be less accurate when applied to these groups.
  2. Health data is biased
    datasets lead to lower performance due to past human mistakes / disadvantages.
  3. Each mistake is extremely costly
    an algorithm providing improper diagnosis can lead to serious care errors / fatal outcomes.

"Fairgen's aim is to provide fairer and more accurate datasets to avoid mistakes and biased treatment on underrepresented groups."

Biases in practice

Example 1: Cancer diagnosis - Lack of data on certain groups

AI algorithms have started to make a real difference in cancer diagnosis. They can for example diagnose skin cancer, catch a stroke on a CT scan or detect potential cancers on a colonoscopy. However, when the data that trains the algorithm does not represent the whole population accurately, the algorithm will necessarily make mistakes when applied to the misrepresented categories.

This has been the case in the detection of skin cancer. The algorithm in question was 34% less accurate on darker skinned individuals. In fact, since more people with white skin suffer from skin cancer, less data is available for other skin types and the algorithms are not well trained enough to provide fair diagnosis and treatment of all individuals.

Example 2: Optum -  Health data is biased

A recent study published in the American Association for the Advancement of Science found racial biases in an Optum algorithm used by US hospitals to identify patients with complex health needs.

The study found that black individuals allocated the same degree of risk by the algorithm are actually sicker than white patients and that bias occurs because the algorithm uses health costs as the gauge for healthcare needs.

The algorithm is trained on medical histories and the amount of money spent on care. It incorrectly assumes that because black patients receive less funding despite having the same amount of need that they are healthier than equally ill white patients. This is due to disparities in cost that result from low-income classes still facing significant obstacles to receiving medical care and from direct discrimination still occurring.

What Fairgen does‍

Fairgen's aim is to provide fairer and more accurate datasets to avoid mistakes and biased treatments on underrepresented groups.

  1. We improve diagnostic accuracy: we can simulate accurate synthetic datasets to train AI to come up with fair & transparent decisions.
  2. We better reflect the overall population: our rebalancing tool generates synthetic data without unfair advantages to a certain group/demographic.
  3. We provide empirical & theoretical proofs for the effectiveness of our approach through margins of error and accuracy benchmarking.

How does it work?

We input real data that is biased into a generator to which we add fairness constraints. We obtain synthetic realistic data which then goes through an AI that will make decisions based on this new fair synthetic data.

All in all, AI biases are very present in healthcare. Either there is a lack of data or the existing data is biased due to past human mistakes and discriminations or economic disparities. The various examples of biases testify to the seriousness and urgency of the situation. Fairgen's solution of providing synthetic data that is fair to replace the biased one is the optimal option to reduce these erroneous diagnoses and care recommendations.


Health It Analytics - AI bias in cancer treatment

WIPRO - the pitfalls of AI bias in healthcare

PC mag - MIT AI predicts breast cancer risk 5 years in advance

Scientific American - healthcare AI systems are biased - Dermatologist-level classification of skin cancer with deep neural networks

Jama Dermatology - Machine Learning and Health Care Disparities in Dermatology