WHEN THE ALGORITHM BECOMES DISCRIMINANT
“Artificial Intelligence is potentially more dangerous than nuclear weapons“. Elon Musk. Dangerous because it is capable of affecting all aspects of life: rights, freedoms, privacy, security, safety, defense and many others.
AI is used through algorithms that retrieve and process a mass of personal data, this system can sometimes cause violations of various fundamental human rights, safety and security. The data retrieved by algorithms are the facts of individuals, some adopt discriminating behaviors, the algorithm will therefore, like a mirror, reflect this discrimination.
This is the case of the chatbot Tay, who had learned certain discriminatory and misogynistic remarks, considering feminism as a “cancer” and spreading racist remarks. The tweeting algorithm is also pointed out when it starts cropping the photo of a black man to leave only the photo of a white man on a post. Worse still, Google photos have confused black people with gorillas, which poses a real problem of exclusion and discrimination that needs to be addressed.
The discrimination is also reflected in road safety, in fact, a study showed that black people were more likely to be ran over by an autonomous car than white people, the detection of black pedestrians is less accurate, because the learning database contains almost 4 times more images of white people. The lack of diversity in the learning data processed by the algorithms leads to this kind of discrimination, the proof of which is that Amazon’s algorithm discriminated against women in its automated recruitment process in 2014. The goal was to give a score out of 5 to each candidate, based on the profiles of predominantly male candidates. The software has therefore better-rated men at the expense of women.
The algorithms are also used by judges in the United States. COMPAS is an algorithm used to predict a person’s risk of recidivism. However, a survey showed that the data used was biased against ethnic minorities. This is a violation of a fundamental principle, which is the right to a fair trial and the right to due process guaranteed in Article 6 of the ECHR (European Convention on Human Rights).