back to article Facial recognition software easily IDs white men, but error rates soar for black women

Commercial AI is great at recognising the gender of white men, but not so good at doing the same job for black women. That's the conclusion of a new study, "Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification", that compared gender classifiers developed by Microsoft, IBM, and Chinese startup …

  1. Anonymous Coward
    Anonymous Coward

    BBC - The Last Enemy (2008)

    We're almost there, but with even less accountability... Happy days!

    https://en.wikipedia.org/wiki/The_Last_Enemy_(TV_series)

  2. This post has been deleted by a moderator

  3. Pete4000uk

    Is spreading

    We now have racist computer systems. When will it end!

  4. Dodgy Geezer Silver badge

    Re: Is spreading

    As the downvotes indicate, this is something you're not allowed to say. Or think...

  5. Anonymous Coward
    Joke

    Re: Is spreading

    It must be made by Microsoft.

  6. Anonymous Coward
    Anonymous Coward

    Re: Is spreading

    Surely it's absolutely not racist since the one thing it can't do is discriminate?

  7. Voland's right hand Silver badge

    Re: Is spreading

    Surely it's absolutely not racist since the one thing it can't do is discriminate?

    It is not the computer's fault to start off with. Basic photography 101. With the same exposure levels for a colour photograph you will get less contrast and less feature differentiation for darker skin colours.

    If you want to make the computer job easy, change the spectrum band in which you take pictures. I suspect that you can get significantly lower error rate going into near IR. This is already being done for number plates by the way.

  8. Anonymous Coward
    Anonymous Coward

    Re: Is spreading

    Which is odd considering most programming languages ignore white space.

  9. ibmalone Silver badge

    Re: Is spreading

    Well, this is my first thought, poorer contrast. But it is an assumption. Even with a balanced training set tuning of parameters is often already done first on a separate set, so it could be at that stage too (and coming back the the contrast/IR issue, evaluation against other methods is often on smaller sets before being applied to larger ones, so the need for an appropriate imaging method can get missed).

  10. AMBxx Silver badge
    Coat

    Re: Is spreading

    The problem with contrast has been known for a long time. anyone else remember the racist webcam that only followed white people?

    That said, the higher error rate for women vs men implies that the problem is more to do with a smaller sample set than methodology.

  11. MrXavia
    Coat

    Re: Is spreading

    but also women usually have one less differing facial feature to help.. Beards..

  12. Anonymous Coward
    Anonymous Coward

    Re: Is spreading

    OOO upset the MS fans. So many downvotes, so little memory.

    http://uk.businessinsider.com/microsoft-deletes-racist-genocidal-tweets-from-ai-chatbot-tay-2016-3

  13. This post has been deleted by its author

  14. AMBxx Silver badge

    Re: Is spreading

    >>> but also women usually have one less differing facial feature to help.. Beards..

    I'm pretty sure a beard makes it much harder to identify. Women have greater variety of hair styles too. Unless they're looking a Male Pattern Baldness!

  15. David Nash Silver badge

    Re: Is spreading

    No icon so not sure how serious you are but it seems to me that it's a technical problem rather than one of racism.

  16. Fruit and Nutcase Silver badge
    Alert

    Re: Is spreading

    @AC

    Which is odd considering most programming languages ignore white space.

    These must be written in Python then!

  17. spold Bronze badge

    Re: Is spreading

    While the black/white thing may be due to technical factors such as contrast, perhaps the female thing is that it is software written by men and it keeps getting distracted by boobs?

  18. The Nazz Silver badge

    Re: Is spreading

    re MrXavia.

    Subtle, i like the way you left moustaches in there.

  19. Anonymous Coward
    Anonymous Coward

    Re: Is spreading

    Women tend to have more varying hairstyles, make up, glasses (if they wear them) than men, across cultures. But I agree moving the cameras into a part of the spectrum where melanin is not seen would probably help.

  20. John Sanders
    Alien

    Re: Is spreading

    All human beings are the same and that there are no differences between men and whamen.

  21. Warm Braw Silver badge

    Re: Is spreading

    the one thing it can't do is discriminate

    Given the increasing fluidity of gender identity, I can't really see why attempting to form two tidy distinct subsets is even considered a worthwhile goal - unless they're trying to maximise the revenue from Valentine cards.

  22. Anonymous Coward
    Anonymous Coward

    Re: Is spreading

    "Given the increasing fluidity of gender identity"

    There are only a very few people with that condition though.

  23. kain preacher Silver badge

    Re: Is spreading

    Voland's right hand you wrong very wrong this is the reason why,

    https://petapixel.com/2015/09/19/heres-a-look-at-how-color-film-was-originally-biased-toward-white-people/

  24. This post has been deleted by its author

  25. Orv Silver badge

    Surely it's absolutely not racist since the one thing it can't do is discriminate?

    That's a bit like saying that there's no problem with racial minorities being shot by police, because guns can't see skin color.

    Computer software can't help but reflect the biases present in the data sets it's trained with (AI), or validated against (manually coded algorithms.) One of the dangers here is that computers will become a way to codify bias in a socially acceptable, plausibly deniable way. "It's not me, it's the computer."

  26. onefang

    Re: Is spreading

    "Which is odd considering most programming languages ignore white space."

    While that is true, Python is popular with AI coders. Maybe we need a language where black space is significant to balance things out? Followed by all the other colours.

  27. AMBxx Silver badge
    Joke

    Re: Is spreading

    What do we do with non-binary stuff?

  28. John Brown (no body) Silver badge

    Re: Is spreading

    "We now have racist computer systems. When will it end!"

    On the bright side, we are, even here in the 21st century, still seeing regular news stories of black people being stopped and searched or assumed to be up to no good by white police officers. If the facial recognition is so much more poor with darker skins, then mainly white "persons of interest" will get flagged up by facial recog.

  29. TheMeerkat

    Re: Is spreading

    Women use cosmetics which might hide some specific features. Could this be a reason?

  30. Orv Silver badge

    Re: Is spreading

    If the facial recognition is so much more poor with darker skins, then mainly white "persons of interest" will get flagged up by facial recog.

    That depends on whether the failures are false positives or false negatives. It could be the software will decide all black people look suspiciously like its database of perps.

  31. Daniel 18

    Re: Is spreading

    I wonder if using more cosmetics more often is a factor?

  32. Phil 54

    "Rwanda, Senegal, South Africa, Iceland, Finland, Sweden"

    There seem to be a few areas of the world with relatively distinctive features missing...

    I wouldn't call it representative if there are no or very few people from the Indian subcontinent or east Asia, not to mention South America.

  33. Adam 52 Silver badge

    Not only that but "individuals

    from three African countries (Rwanda, Senegal, South Africa) and three European countries (Iceland, Finland, Sweden) selected for gender parity in the national parliaments"

    The authors are so blinded by their desire to find a bias on gender and skin colour that they've picked a test dataset that is massively skewed on ethnicity, age, health and social class.

    They also tie themselves in knots trying to reconcile biological gender with gender identity, so their benchmark gender classification is suspect in the first place (it's based on Mr, Mrs and the "appearance of the photo").

  34. Richard 81

    Agreed. To be representative you really need to sample everywhere. Otherwise there are always going to be big gaps in its applicability domain. In this case Scandinavia is over represented, so it's going to be over trained on Vikings, while they've got just three really distant countries representing all of Africa.

  35. Lysenko Silver badge

    The word "Intersectional" is the giveaway. A study declaring itself Marxist/Leninist would have identified that the discrepancy was class based and constituted oppression of the proletariat using exactly the same data and for exactly the same reasons (confirmation bias).

    That doesn't mean they are wrong about the differential accuracy of course. It just means they have pointlessly poisoned the well regarding the integrity of the study.

  36. Voland's right hand Silver badge

    A) Rwanda, Senegal, South Africa - three very distinct and different black race subtypes.

    B) Iceland, Finland, Sweden - same Caucasian subtype across the board

    How about trying an equally diverse Caucasian set. Let's say: Sweden, France and Bulgaria (*).

    (*)I know I am being mean to the poor AI - Bulgarians vary from a nordic sand blond to outright Mongoloid nearly Genghis Khan look alike. However, officially - they are all Caucasian.

  37. katrinab Silver badge

    You won't find an equally diverse white set because there is far more diversity among black people than any other group - it is a small group of black people who moved north and became white.

  38. Anonymous Coward
    Anonymous Coward

    I'm not sure if serious.

    Many times it has been stated, there is more diversity and difference inside groups than there is between groups. So while we notice the differences of vast distant populations, we ignore through confirmation bias or social normality, the massive differences locally.

  39. Cuddles Silver badge

    "I wouldn't call it representative if there are no or very few people from the Indian subcontinent or east Asia, not to mention South America."

    Beat me to it. Between them China and India cover over a third of the world's population. It looks more as though they deliberately took the extremes of some of the whitest and blackest populations they could find. Which is fair enough for a certain kind of testing, but can hardly be considered representative and is likely to paint things in a particularly bad light.

  40. Lysenko Silver badge

    Re: I'm not sure if serious.

    Many times it has been stated, there is more diversity and difference inside groups than there is between groups.

    That may be true for genotypes (it's actually an oversimplification), but it doesn't hold for phenotypes, particularly not when you're only considering a tiny subset of phenotypical data which the human brain has specifically evolved to evaluate (faces).

  41. asdf Silver badge

    Genetic diversity

    Isn't 85% of all genetic diversity in humans found in Sub Sahara Africa (phenotype as well)? Remembering hearing that. Too lazy to get real reference but this wikipedia snippet will do for now.

    "Sub-Saharan Africa has the most human genetic diversity and the same has been shown to hold true for phenotypic diversity.[36] Phenotype is connected to genotype through gene expression. Genetic diversity decreases smoothly with migratory distance from that region, which many scientists believe to be the origin of modern humans, and that decrease is mirrored by a decrease in phenotypic variation. Skull measurements are an example of a physical attribute whose within-population variation decreases with distance from Africa."

  42. Lysenko Silver badge

    Re: Genetic diversity

    Isn't 85% of all genetic diversity in humans found in Sub Sahara Africa (phenotype as well)?

    Most of the genome has no external phenotypical expression and whole-genome genetic distance does not necessarily correlate with phenotype as the variation in dog breeds and the quasi-canine appearance of hyaenas illustrates. Conversely, sub-saharan Africans and aboriginal Australasians have the greatest genetic distance but are often considered to be phenotypically similar.

    Sub-saharan Africans do not have any Neanderthal or Denisovan genetic contribution (3-5% in everyone else) besides some rare instances of genetic backflow from the Levant. This impacts externally observable phenotype, particularly in terms of eye and probably hair colouration (plus the immune system, hair texture, respiratory metabolism and a number of other areas).

    Human perceptions of phenotype are evolved to assess ingroup membership rather than genetic distance per se. That means the environmental (in human terms, cultural) aspect of phenotype is often of greater importance. For a human, how an individual dresses and behaves (e.g. a military uniform, prayer rituals) is part of the phenotype just as the exact design of a nest is for a bird.

  43. onefang

    "I wouldn't call it representative if there are no or very few people from the Indian subcontinent or east Asia, not to mention South America."

    Or Australian aborigines, or several other indigenous populations.

  44. Scroticus Canis Silver badge
    Boffin

    "officially - they are all Caucasian"

    So are those from the middle east and Indian sub-continent; well at least back when I was in uni.

  45. ProgrammerForHire

    Re: Genetic diversity

    A lot of genetic variation can arise within hybrids of two different species.

    It seems it happened in Sub Saharan Affie as well as in Europe, only with an older more primitive humanoid :

    https://newatlas.com/ancient-ghost-species-human/50591/

    Looks like Europe and Asia were subjected to high selectivity, because only a bit of the variation derived from Neanderthal / Denisovan mixing has survived

  46. John Brown (no body) Silver badge

    "it is a small group of black people who moved north and became white."

    I think you struck a nerve in some readers. 4 downvotes for a factually correct statement.

  47. ProgrammerForHire

    The Out of Africa theory is controversial. As of today, the evidence for modern HS originating there is just the age of the oldest HS fossils

  48. Anonymous Coward
    Anonymous Coward

    "B) Iceland, Finland, Sweden - same Caucasian subtype across the board"

    If linguistic patterns dating back a long long long time - several thousand years at a minimum - are a clue, Finns and Swedes are very different groups... one would expect the Finns are more closely related to Siberians than Scandinavians.

  49. Anonymous Coward
    Anonymous Coward

    "The Out of Africa theory is controversial. As of today, the evidence for modern HS originating there is just the age of the oldest HS fossils"

    AFAIK, the only part of OoA that is controversial is the number and timings of the migrations.

    A recent study showed that a second wave of OoA migration is sufficient to explain the replacement of the Neanderthals in Europe with Homo sap.

    And yes, you are right, there seems to be absolutely no evidence for humans or protohumans in Europe before the OoA migrations.

  50. Ledswinger Silver badge

    Skin tone is not binary with the meta groups

    So was the correlation purely on darker skin tone as such, or was it actually caused by a correlation between darker skin tone and different facial characteristics that react less well to the identifying parameters used by the algo?

    I suppose I could read the article, but that's too much of my time.

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2018