Researchers from the University of Maryland School of Medicine (UMSOM) found that after analysing crowd-sourced sets of data used to create AI algorithms from medical images, most did not contain demographic data. In addition, they found the algorithms were not evaluated for inherent biases.
This implies that researchers or clinicians who may use one of these algorithms have no way of being certain whether images include representative samples of the population such as Asians, Blacks, and Indigenous Americans.
The team noted that biases are a significant issue in medicine, with more collected data tending to skew toward people with easy access to healthcare, toward men versus women, and toward people who are white rather than other races. As a result, certain races, ages, genders and sexual orientations are not represented.
Within this study the team evaluated the datasets used in data science competitions which used crowdsourced data from around the world. The goal is to develop the most accurate medical imaging algorithm. 61% of the data did not contain demographic data including race, age and sex. None of the competitions in the analysis had evaluations for biases against underrepresented populations.
Lead author, Sean Garin, said, “We hope that by bringing awareness to this issue in these data competitions – and if applied in an appropriate way – that there is tremendous potential to solve these biases,”
Mark T. Gladwin, MD, Vice President for Medical Affairs, University of Maryland, Baltimore, added, “As AI models become more prevalent in medical imaging and other fields of medicine, it is important to identify and address potential biases that may exacerbate existing health inequities in clinical care”.
Source: University of Maryland
Image Credit: iStock
