back to article Is your computer doctor secretly a racist? Two US senators want to find out the truth

US Senators Ron Wyden (D-OR) and Cory Booker (D-NJ) are examining how federal agencies and healthcare companies are tackling algorithmic biases – after a recent study found that black patients were less likely to be referred to care programs by software than white patients, despite being sicker. On Tuesday, Wyden (D-OR) and …

  1. Marketing Hack Silver badge
    Stop

    You need actual data on the incidence of medical conditions and risk factors

    If you start allocating and recommending treatment based on past treatment of, let's say, the African-American population, then you are begging for inaccuracies in the future. African-Americans may have received fewer medical appointments and procedures, due to racial animus against treating them, lack of doctors in areas with larger African-American populations, based on lower African-American incomes and employment restricting access to medical services and due to African-American populations being anxious about seeking help from the largely white medical profession.

    So if you bake all that biased past data into current coverage models, then you can definitely carry forward past racial and racist disparities.

    1. Nunyabiznes Silver badge

      Re: You need actual data on the incidence of medical conditions and risk factors

      I am not a Doctor, so honest question:

      Why would race be a necessary factor in an algorithm for treatment? I know some genetic groups have pre-dispositions to certain diseases (diabetes and kidney disease come to mind) but with more homogenization of populations (in the States at least) genetic testing should be done as a matter of course because visual indicators will lead to false assumptions. The results of preliminary testing could then be factored into the algorithm.

      1. Carpet Deal 'em Bronze badge

        Re: You need actual data on the incidence of medical conditions and risk factors

        That's partially why genetic testing will never fly: it reveals a little too much about our susceptibilities, which you'll never keep away from insurance companies(you can try to mask who's who all you want, but there's no such thing as anonymization against someone with any determination to undo it).

      2. FriendInMiami

        Re: You need actual data on the incidence of medical conditions and risk factors

        The United States has created and maintained divided populations sufficiently that in certain populations some diseases are much more prevelant. For instance, "sickle cell" blood disease. From the U.S. National Institutes of Health website:

        "Sickle cell disease affects millions of people worldwide. It is most common among people whose ancestors come from Africa; Mediterranean countries such as Greece, Turkey, and Italy; the Arabian Peninsula; India; and Spanish-speaking regions in South America, Central America, and parts of the Caribbean.

        Sickle cell disease is the most common inherited blood disorder in the United States, affecting 70,000 to 80,000 Americans. The disease is estimated to occur in 1 in 500 African Americans and 1 in 1,000 to 1,400 Hispanic Americans."

    2. Psmo Bronze badge

      Re: You need actual data on the incidence of medical conditions and risk factors

      Even given all that, the data is still biased; the medical condition must be found and treated.

      No data for undetected conditions.

  2. Anonymous Coward
    Anonymous Coward

    Health care costs predict standard of care in the US?

    “black patients were less likely to be referred to care programs by AI systems than white patients, despite being sicker .. Poor performance was due to the algorithm predicting health care costs rather than potential illnesses, the study claimed.”

    The US Senators should examine the nature of the environment and historical difference in the standard-of-living as experienced by the two groups. Perhaps that is the source of the discrepancy.

    1. Marketing Hack Silver badge

      Re: Health care costs predict standard of care in the US?

      All health care plans, including the NHS, ration care to some extent, based on costs and measured effectiveness per unit of expenditure. They set what procedures, devices and drugs are covered for patients experiencing what conditions and symptoms to which degrees, in an effort to make sure that someone with acute backache does not receive expensive drugs and procedures when perhaps pain relievers and therapy are a more cost-effective answer.

  3. redpawn Silver badge

    Fortunatly

    the rich can buy their way into good health care and also benefit through their portfolios from the money saved by the algorithms. It's a win win proposition.

    1. LucreLout Silver badge

      Re: Fortunatly

      Thankfully in the UK you don't have to be rich to afford private medical insurance. I'm pretty sure my family premiums are about £150, which is latte/pub money. I'm not suggesting it's universally affordable, only that you can be a long way from rich and still have private cover.

      1. Is It Me Bronze badge

        Re: Fortunatly

        And one of the main reason the premiums are so low is the NHS and universal healthcare.

        1. LucreLout Silver badge

          Re: Fortunatly

          And one of the main reason the premiums are so low is the NHS and universal healthcare.

          No it isn't. The policy covers what the policy covers entirely absent the NHS. The reason its cheap is you don't have to pay horrendous unearned pensions from peoples mid fifties, and because inefficient providers go bust, quite unlike the bloated NHS.

  4. Suricou Raven Silver badge

    So, no

    Catchy headline, but on further investigation the algorithm doesn't want to kill black people. It just wants to kill poor people.

    I guess that's ok then.

    1. Anonymous Coward
      Anonymous Coward

      Re: So, no

      Catchy headline, but on further investigation the algorithm doesn't want to kill black people. It just wants to kill all people differently.

      FTFY

    2. ibmalone Silver badge

      Re: So, no

      Not entirely clear. It's generating its predictions on cost, but my reading of the article is that those costs are being factored into racial group risk levels and then coming out the other side according to the "race" (however you want to slice that) of the person whose risk you're attempting to predict. Healthcare is one of the areas that you might carefully use race as an input for certain things (for example, people of African American descent are more likely to develop glaucoma, as are those with an Irish background), carefully because decisions based on such predictions can become systematic discrimination.

  5. SonofRojBlake

    Betteridge's law of headlines strikes again

    Is the computer doctor secretly racist?

    No.

    It's classist. (And black people are disproportionately in lower social classes than white because racism, yes, but that wasn't the question).

    1. Anonymous Coward
      Anonymous Coward

      Re: Betteridge's law of headlines strikes again

      It's the type of headline you'd expect from one of the brainless tabloids, not a specialist IT site - very poor.

      However, everyone seems to be eager to jump onboard the social justice bus lately, to the detriment of logical analysis and sound reasoned argument. Virtue signalling is the order of the day unfortunately.

      1. Androgynous Cupboard Silver badge

        Re: Betteridge's law of headlines strikes again

        Bless. Perhaps you're new here?

        Or maybe, as you've managed to shoehorn "social justice" and "virtue signalling" into a response on a decent article into what is very real and well known issue in IT (training bias), you've failed to acknowledge the humour in the headline AND the logic in the article because you're a wee bit of a racist yourself? Hmm?

  6. herman Silver badge

    Some illnesses and treatments are race specific

    I'm sorry, but races do exist and some illnesses and treatments are race specific. For example, some heart medications that work on Caucasians will kill Africans. So, AI Doctors cannot afford to ignore race.

    1. Doctor Syntax Silver badge

      Re: Some illnesses and treatments are race specific

      "For example, some heart medications that work on Caucasians will kill Africans."

      Can you be more specific, especially as to mechanism? If this is the case it's more likely to be a consequence of some specific allele. The risk would be correlated with some specific aspect of the patient's ancestry but a more sensible approach would be to test for that allele rather than make some judgement based on what the patient looks like.

    2. Len
      Holmes

      Re: Some illnesses and treatments are race specific

      Races exist culturally, they do not exist biologically. The majority of illnesses are biological, cultural illnesses do exist but are mainly down to how symptoms are explained and perceived culturally. I have been told the German condition of Herzinsuffizienz does not exist elsewhere for instance. What we do have is clusters of genes that are more typically found in certain geographic areas.

      That's why we have issues that have a higher prevalence around the Mediterranean than elsewhere, issues that are common in people of West-African/European origin but virtually non-existent in East Africa (not surprising as West Africans and Europeans are genetically extremely similar whereas East-Africans couldn't be more different), issues that are common in the Caribbean but less common elsewhere etc. etc.

      You some times see blanket suggestions for people with certain backgrounds to do further screening for specific issues, screening for Thalassaemia for people with "a Mediterranean background" or Sickle-Cell for people with "a West-African background" for instance but any decisions will be based on biologically proven situations detected in testing and not on whether someone "looks West-African" or "looks Mediterranean".

      Any medical professional that bases their decisions on cultural constructs like character, race or class should be on the receiving end of an investigation.

      1. Tigra 07 Silver badge

        Re: Some illnesses and treatments are race specific

        "Races exist culturally, they do not exist biologically"

        A popular opinion, but there's a bit more to it. If 15 people die in a house fire the makeup and density of bones, teeth, etc can tell which part of the world those people and their ancestors originated from to a fine degree. Providing the effort to do that is not usually necessary, unless you've discovered a mummified body of an ancient king while excavating a Tesco car park.

        So in some cases and with the correct funding and equipment you can tell which nationality someone was when they were alive.

        1. Len
          Holmes

          Re: Some illnesses and treatments are race specific

          You're making a few strange jumps there.

          You can indeed detect certain things from teeth and bones, ancestry, where someone lived, diet etc. Isotopes in teeth are particularly helpful to detect where someone grew up, Haplogroup Q could give an indication that someone has North-African or European ancestors, etc. You can not detect "race" however as that is a cultural construct that is not stored in cells. It depends on who you ask, where and when. The definitions of "race" vary too much to be of any use.

          An average European will have an entirely different grouping of "races" than an average American (Some Americans call Barack Obama "black" despite having a "white" mother, consider Kim Kardashian not to be "white", Marco Rubio "white" but Alexandria Ocasio-Cortez "Hispanic" and think it's possible to be "Hispanic" while not "white". Those are constructs that no biologist could endorse). If you are able to find a forensic investigator willing to suggest a "race" the chances are that a European investigator will find a different "race" than an American investigator.

          All of these constructs vary through time as well. Many Indians have darker skin tones than many Africans, does that mean Indians are "black"? Not many people would argue so now but perhaps in fifty years people do. Are people of Jewish descent "white"? We tend to think so now but not a hundred years ago. Then again, some people would currently argue that the Beta Israel are "black", will they be considered "white" in fifty years from now? Who knows...

          As for nationality, no amount of equipment or funding will make it possible to tell nationality from cells. Nationality is not stored in genes or cells and people can relatively easily switch nationality without it changing their genetic make-up. My partner has two nationalities but is considering giving one of them up so she can replace it with another. As she is not planning to move (no change in isotopes) there will be no visible changes in her cells.

          1. Anonymous Coward
            Anonymous Coward

            Re: Some illnesses and treatments are race specific

            You must surely be in the running for pedant of the year award, Len!

            1. Len
              Happy

              Re: Some illnesses and treatments are race specific

              If you think I'm a pedant you should see my partner. Half of our rows end in her shouting "Where's the data!? Show me the data!"

              1. Tigra 07 Silver badge
                Thumb Up

                Re: Some illnesses and treatments are race specific

                "Where's the data!? Show me the data!"

                Richard Dawkins reference?

                1. Len
                  Meh

                  Re: Some illnesses and treatments are race specific

                  No, she's a medical doctor. She wouldn't even flick a light switch if it wasn't evidence-based to have an effect on room illumination.

          2. Albatross! Al-Ba-Tross!

            Re: Some illnesses and treatments are race specific

            My SO is a magna-cum-laude Anthropologist, and Len is completely correct.

            1. Anonymous Coward
              Anonymous Coward

              Re: Some illnesses and treatments are race specific

              My SO is a magna-cum-laude Anthropologist, and Len is completely correct.

              The virus of Cultural Marxism has permeated the fields of social "science" so thoroughly, this really doesn't surprise me at all. Group think is the order of the day and the toxic influence on society that this hateful ideology promotes is nothing short of scandalous.

              A pox on their houses!

          3. Tigra 07 Silver badge

            Re: Some illnesses and treatments are race specific

            "You can not detect "race""

            You can "determine" what it was. Better? Pedantry at it's finest.

            "It depends on who you ask"

            No? Name a country that allows you to be a citizen just by saying you are one. It's a status granted by where you are born, or where your parents were born (making the assumption you don't move obviously). Israel is the only exception, and certain sects of Jews probably share a lot of similar DNA and ancestry to justify that.

            "As for nationality, no amount of equipment or funding will make it possible to tell nationality from cells"

            You're splitting hairs. Look at "Genetic heritage" or "Genetic Ancestry". There's an entire field of science you're claiming isn't real and doesn't exist, without knowing anything about it. If you can tell which period of time a person is from, and which part of the world, you can most definitely identify their nationality. I'm not saying it's stored in their DNA, i'm saying you can identify it in specific instances.

          4. J.G.Harston Silver badge

            Re: Some illnesses and treatments are race specific

            "no amount of equipment or funding will make it possible to tell nationality from cells"

            However, if you can determine from those cells when that person died, you can contact various passport offices to find out what nationality they had at that point. So, *indirectly*, you /can/ determine what nationality a dead person had from examining their remains.

      2. phuzz Silver badge
        Boffin

        Re: Some illnesses and treatments are race specific

        I'd imagine that skin colour is probably a good indicator* of the risk of skin cancer.

        * but not the sole indicator, and I'll bet that income probably has a lot to do with it.

        1. rcxb Bronze badge

          Re: Some illnesses and treatments are race specific

          I can't see how, unless you're accurately gathering that data at birth...

          You need to be able to distinguish between the shades of brown of a fairer genetic groups (e.g. Irish, Scandinavians) who have gotten lots of sun exposure to those of darker genetic group (e.g. Africans, Indians) who have gotten very little sun exposure. There's sure to be overlap in the skin shades of those quite disparate groups with differing UV radiation exposure levels.

      3. Anonymous Coward
        Anonymous Coward

        Re: Some illnesses and treatments are race specific

        Races exist culturally, they do not exist biologically.

        Seriously? When is the last time you saw a White or Latino person with sickle cell anemia?

        So you think that the fact that I sunburn easily is a cultural thing and not biological and that hereditary conditions are urban myths?

        1. Len
          Holmes

          Re: Some illnesses and treatments are race specific

          Are you considering 'Latinos’ a "race”? That's a new thing for me. Do you consider "Hispanics" a "race" too? And what's the difference between a "Hispanic", “White” and "Latino” person? All have a European background, unless you'd classify indigenous Americans under any of those brackets. That would be a bit odd, though, as "Hispanic" suggests they have a Spanish European background, Latino suggests they have a Latin European background (are people of Italian descent Latinos? If anyone could be classified as Latino it should be people of Italian descent. Italians still speak a variety of Latin.) Meanwhile, the indigenous Americans came through Asia and probably have barely any European background at all.

          Many of those classifications are cultural constructs, some only exist in the US. Here in Europe we don't have Hispanics, we tend to call people from Spain ‘Spanish’, we tend to think of Rome when someone uses the word “Latin”.

          There is a cultural element to some illness. I mentioned Herzinsuffizienz (it translates as insufficiency of the heart) which is a common diagnosis in Germany but one you won’t find in many non-German textbooks. In some cultures talking to one self would be a tick box on a Schizophrenia assessment form, in other cultures it would not. The symptoms can be the same but whether they are considered an illness is cultural.

          But obviously, the majority of diseases are purely biological. Increased likelihood of developing sun burn, Sickle Cell Anaemia, or Thalassaemia are the result of specific genetic make-up that is hereditary to a large extent (for instance SCA can disappear through the generations via Trait to No Carrier). They are all biological, not cultural. And, as I said earlier, there is some genetic clustering, Thalassaemia prevalence is higher than average around the Mediterranean. Glaucoma prevalence is higher in the Caribbean and Scandinavia.

          It just doesn’t make any biological sense to subdivide the human race into various races depending on just a tiny fraction of their genetic variance. Where do you draw the boundaries ? The size of their jaw? Hair colour? Eyebrow relation to eye socket? Whether they have a flat or a rounded back of the skull? Whether they can digest lactose? Likelihood of developing Glaucoma? The shape of their femur? And why would you do it along those lines but not another? The most logical subdivision I know is that along Haplogroup lines and even that has little practical use except in the field of genetics. Most doctors do not need to know the haplogroup of their patient.

    3. Hugo Rune
      Boffin

      "some heart medications that work on Caucasians will kill Africans."

      An Example of race bias. Some Africans are caucasians.

      1. WolfFan Silver badge

        Re: "some heart medications that work on Caucasians will kill Africans."

        Including some of my relatives.

    4. paulll Bronze badge

      Re: Some illnesses and treatments are race specific

      " For example, some heart medications that work on Caucasians will kill Africans."

      Which ones are those, then? I've never encountered a cardiac med that has,"African patient," as a contraindication. Nevermind that I don't think we're even talking about an,"AI Doctor," with prescribing rights.

  7. philluminati

    We know!

    I wrote a blog post about this exact problem a few years ago

    https://blog.philliptaylor.net/defeating-racism-in-artificial-intelligence/

    as well as how Matroska can help eliminate bias from those algorithms.

  8. Blackjack

    Racist programers make racist AI

    https://m.youtube.com/watch?v=sllyEInE2MA

    Don't blame the machine, blame the programmer.

  9. Anonymous Coward
    Anonymous Coward

    It's great they are looking into this so poor black people as well as poor white people get to see what treatments they can't afford.

  10. Il'Geller

    The solution to the problem is very simple! For "training" it is enough to use dictionaries and encyclopedias, which have an absolute minimum of bias among all texts.

  11. Anonymous Coward
    Anonymous Coward

    Practially everybody is racist, and that's a good thing.

    If you can tell the races apart you are racist. e.g. "black" people tend to have darker skins than "white" people.

    e.g. If you know that someone fits into a certain "race" category, and you're trying to find them, you don't have to search through those that aren't that "race", thereby saving time.

    p.s. It seams that typical media when compares races(like this one), it implies that the goal should be that "blacks" have everything >= "whites", instead things being better for everybody.

  12. Anonymous Coward
    Anonymous Coward

    Not a race issue

    In reading the study, it would appear that the bias comes in against those with less money because those with less money get more subsidized health care, thus spend less of their own money.

    It isn't about race, it is about a faulty choice of proxy measurements - and using race to get headlines.

  13. james7byrne

    What a bunch of nonsense

    This algorithm has no impact on my life. I go see my doctor, he comes up with a treatment plan, and we move forward. No algorithm needed.

    The two Democrats are just trying to stir the identity politics pot. The fact is, each individual is responsible for their own health care. This is all between you and your doctor.

  14. Anonymous Coward
    Anonymous Coward

    everything is about PC, even algorithms. So that being said, 6 pints of bitters, please. The world is about to end.

  15. Anonymous Coward
    Anonymous Coward

    Bias on quoting for future cover

    If the system thinks black people are healthier because they spend less on care, is the system used for quoting for health insurance cover?

    So if it looks at two identical guys,both same age and weight and went to the same college/uni etc but one was black, would it offer that person a lower yearly rate for the same cover since it thinks they are less risk, bet not. I think they would charge the same and try to market more to black people if they REALY believed it.

    I wonder if thats how they noticed.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2020