back to article British cops told to scrap 'discriminatory' algorithms in policing

Human right group Liberty is urging UK cops to stop using predictive policing programs that put a "technological veneer of legitimacy" on existing biased practices. In a report published yesterday, Liberty said that, at the very least, the force should be more transparent about their use of algorithms in carrying out police …

  1. tiggity Silver badge

    Bias in, bias out

    Given innate UK police bias its zero surprise any police "AI" systems will use the same biases.

    Chance of them fixing it? .. about the same chance of stop and search targeting being modified to reflect actual "race" distribution instead of having a huge bias to targeting "non white" folk

    1. Solarflare

      Re: Bias in, bias out

      Playing devil's advocate here...but if algorithms show that a certain postcode has an elevated amount of crime and that same postcode is majority black/polish/white/muslim/jewish/chinese then is it actually discriminatory to police those areas more (and by extension stop more black/polish/white/muslim/jewish/chinese under suspicion)?

      1. MiguelC Silver badge

        Re: Bias in, bias out

        But it's not just policing (as in patrolling), is it? Stopping someone just because there is historical data of people making trouble in any given area is discriminatory, even if only to the innocents being stopped. Pre-crime is sci-fi and should stay in that realm.

        1. Solarflare

          Re: Bias in, bias out

          But is it? Is it any different from blocking IP ranges on a firewall or dropping email as spam using its destination?

          I get what you mean, I'm not trying to say that some people aren't unfairly targetted, I think that police randomly stopping black people because they assume they have a knife is wrong. But if there have been 200 stabbings in that neighbourhood then policing that neighbourhood and stopping people to check for knives seems proportional. If that neighbourhood is 90% black, does that change things? Personally, I say no, because I treat it as a local crime thing rather than a racial/cultural one.

          1. Anonymous Coward
            Anonymous Coward

            Re: Bias in, bias out

            Well if 'your' IP address gets blocked because someone else on the same ISP is spamming in you might be moderately grumpy. You might get the feeling that the authorities aren't on your side. You might think that as you are 'guilty' in the eyes of the law you may as well do something criminal.

            At least in the UK our police generally stop and search, rather than US style shoot and search.

          2. 's water music Silver badge

            Re: Bias in, bias out

            it is the old correlation != causation coupled with structural inequalities.

            If you have data that shows a correlation between skin colour, home postcode or whatever with criminality then if you have a limited budget it makes some sense to deploy your resources disproportionately at that demographic. You will get more 'hits' for your money which reinforces your sense that this is a good approach so even if the original correlation is based on historical discriminatory policing your approach may appear valid. If the affected demographic is relatively poor and under-represented within the planning process then it is even less likely that a discriminatory approach will be called out.

            A counter-example would be for a financial crime unit to adopt the approach of kettling the Square Mile and shaking down all the white guys in suits. They may well achieve a better hit rate than random selections but as well as being as unfair as harassing black people you could expect more successful push-back against such an approach due to the way power structures work in society.

            Intelligence led policing might be even more successful in percentage terms. Now ask yourself if non-causal correlate led policing is likely to reinforce cooperation in the community most likely to have useful insights to share,

      2. Warm Braw Silver badge

        Re: Bias in, bias out

        The fact that a certain postcode has an elevated amount of crime may not tell you very much about the perpetrators, merely, say, that the neighbourhood has a bunch of easily-nickable cars on driveways. If the crime is, say, burglary, the chances are the police are going to ignore it anyway.

        If all crime were treated equally, then its distribution would be usefully informative. When you start to make choices about which crimes to pursue, then the information may simply reinforce any unconscious bias in your original choices to the point if becomes Kafkaesque.

      3. Anonymous Coward
        Anonymous Coward

        Re: Bias in, bias out

        Well on that basis one would hope that police would concentrate their efforts on targeting Kensington, Chelsea and Westminister where the really serious criminals and drug king pins etc live. Stop a few Ferraris outside Harrods and serve unexplained wealth orders. But that might mean meeting expensive lawyers - so lets bust some poor kid for a spliff.

      4. Anonymous Coward
        Anonymous Coward

        Re: Bias in, bias out

        Makes it moderately hard to explain why Dorset and Devon and Cornwall are the most racially discriminatory police force in their stop and search stats? Maybe I missed the Truro Massive last time I ate a cream tea.

        1. Anonymous Coward
          Anonymous Coward

          Re: Bias in, bias out

          I went to uni in Cornwall, I was stopped around 4 times jogging to work from Penryn to Falmouth . One of them didn't even give me the spiel about "people matching your description... reported burglary bla bla bla" just police lights on pulled up in front of me and asked me what I was doing. Not a criminal I literally woke up at 4am every morning to clean in order to to pay my way thorough Uni.

          What is more odd is that I was one of like 20 black people in Penryn so unless the rest of them were all bandits in disguise I found it rather discriminatory... I hope it had more to do with my ninja-like jogging attire than my race.

      5. phuzz Silver badge

        Re: Bias in, bias out

        Or looking at it another way, a certain area has an elevated amount of reported crime. This could be because the residents trust the police more than in another area where there might be just as much crime, but the residents don't expect the police to do anything about it, so they don't bother reporting it.

        The result of basing a strategy on previous crime stats in this situation would lead to the police patrolling the first area, but not the second, so the residents in the second area never come to trust the police, and are still the victims of (un/under-reported) crime.

    2. Korev Silver badge

      Re: Bias in, bias out

      >Chance of them fixing it? .. about the same chance of stop and search targeting being modified to reflect actual "race" distribution instead of having a huge bias to targeting "non white" folk

      Would a couple of cops wandering along the street saying "We need to search three more Chinese females today to meet the distribution quota" be progress?

    3. Anonymous Coward
      Anonymous Coward

      Re: Bias in, bias out @tiggity

      Simple PC nonsense and the type of thinking that is getting people killed out on the street.

      The police have a limited amount of resources so if one particular demographic is overwhelmingly responsible for a given type of crime, then that is where the focus of their attention must lie.

      1. tiggity Silver badge

        Re: Bias in, bias out @tiggity

        Dross AC

        I used to live in a very rough area.

        I reported (at night) a burglary occurring on the street

        Police asked fr my location .. then said they do not come to that area after dark unless there is risk to life

        .. So, in that area free pass for "low level" (i.e. property crime with a low risk of person getting killed / injured) crime, rather than police actively targeting the crims they were just writing off an area as a high rate of property crime cess pit.

        So, did not report any further property crimes on that street.

        Area was predominately non white, but all the burglars and car jackers I saw were white in the time I lived there, so cannot assume inhabitant demographic is equivalent to criminal demographic.

        Unsurprisingly I moved to a different area

        1. Anonymous Coward
          Anonymous Coward

          Re: Bias in, bias out @tiggity

          So... they actually confessed over to you that they were aiding and abetting a crime? (by just letting it happen anyway)

  2. naive


    Has its influence on bias in algorithms, otherwise they would be pretty useless.

    1. Toltec

      Re: Statistics

      It looks like part of the argument is that the statistics themselves are biased. It is also pointed out that methods other than policing should be used to reduce crime, of course you would need to use statistics to know where to target that effort...

  3. John Stirling

    Predictive policing has the ability to be great if....

    Sadly it is true that certain sectors of society are more likely to be sucked into crime, and especially gang crime. Typically young men from lower income households with lower educational attainment, and lower levels of aspiration.

    The issue here is that instead of looking at 'young men who are poor, not doing well at school, and don't see a good future' (which is really difficult to profile) the algorithms take the easy way out and do a match with 'young black or muslim men'.

    There is nothing inherently less aspirational, educational, or attaining about any particular genetic inheritance - but there may be about certain cultural groups, and how those groups have integrated (or failed to integrate) into UK society. Algorithms have a distressing tendency to jump on the back of easily identitfiable statistical factors, which may 'work', but which will exacerbate the underlying issues by objectifying and victimising groups which have already experienced significant negative bias.

    Sadly unless society (or its proxy, Government) makes a concerted (for which you can read 'expensive') effort to actually address the underlying issues the problems will continue.

    A rather interesting study into educational attainment in I think South America showed 10% better education, incomes and attainment now in communities clustered around the sites of educational religious missions 200 years ago. No such positive affect existed around doctrinal religious missions. These affects are long term, and persist long past the original causes, unless positive action is taken to correct problems.

    Predictive policing is coming, and it will reduce crime, but it is the challenge for society to ensure it doesn't do so by making it even harder for those who are suffering under current systems to improve their position in society.

    1. Joe Montana

      Re: Predictive policing has the ability to be great if....

      As you point out, certain cultural groups are less compatible with UK society than others and the end result of this tends to be crime, but if you say this in public you get accused of racism against those groups.

      The fact is there are many different cultural groups in the UK and other western countries, some of them are highly successful and some are not, and it is those unsuccessful ones who are looking to pin the blame on others rather than accept responsibility for their own failings.

      1. Anonymous Coward
        Anonymous Coward

        Re: Predictive policing has the ability to be great if....

        You've hit the nail on the head Joe and rest assured the sensible, silent majority agree with you 100%.

  4. TVU Silver badge

    Re: Predictive mapping and individual risk assessments

    If these two techniques actually work then by all means continue to use them but if they don't work then consider other options like building up intelligence networks in communities.

    All that said, what would really help the police become more effective is if the Tories unsacked the 21,000 police officers and PCSOs that they slashed as part of their smaller state austerity agenda.

  5. Anonymous Coward
    Anonymous Coward

    Well when ~95% of all knife crime in London is black on black gang violence - that's where the focus ends up going, whether its putting preventative measures in place (and trying to identify where said crime is likely to occur to make it most effective) or purely reactive (going to where emergency calls come from) the result is the same.

    1. Anonymous Coward
      Anonymous Coward

      95% of knife crime isn't 'black on black'. Don't fall for the lazy stereotype. Studies show knife crime is correlated with poverty - not with ethnicity. I accept that a in London black people are often the poorest, but even within that context. Things are changing, probably getting worse, but if you look at youth crime figures in the last decade white perpetrators were responsible for at least 50% of knife crime, and black youth less than 25%. 95% is Richard Littlejohn lazy.

      1. Joe Montana

        If not 95%, it's still a significant percentage especially considering the demographics of the population as a whole.

        If knife crimes being committed by blacks are decreasing then perhaps this is a result of the police focusing their resources on them?

        And yes crime correlates with poverty, but the UK is not a third world country. Just because your parents were poor, uneducated and turned to crime doesn't mean you have to. The government provides free schooling and various opportunities for everyone but it's still your choice wether to take advantage and your own fault if you don't. I know many people who came from poor backgrounds and were able to get a decent education and subsequently get reasonable jobs.

    2. Graham Cobb

      Even if the stat was true, it would not justify stopping the 90% (equally made-up stat) of black men who are perfectly innocent of any crime.

      1. Anonymous Coward
        Anonymous Coward

        @ Graham Cobb

        Meanwhile, in the real world...

        The black population of London in 2010 was just over 12%, yet black males were responsible for 54% of street crimes and 59% of gun crimes.

        Profiling is the only sensible option in the face of such overwhelming statistics.

        1. Graham Cobb

          Re: @ Graham Cobb

          I have argued in another reply why the stats are, unfortunately, completely broken because of the impact of selection bias on the process.

          However, even if they were right, profiling is never the right option.

          * Individuals != profiles. In our society, each person should be treated equally, and are equally innocent until proven guilty. Even if 99% of people fitting a profile were criminals, that would not make it right to discriminate based on that profile.

          * Change can only happen if people can break away from stereotypes and profiles. Even if profiling catches more criminals but it does nothing to encourage people to adopt another course.

          * There are plenty of criminals who do not fit the profile. Why should they get away scot-free?

          Instead of profiling, the answer is more police on the beat, visible, engaged and, most importantly, accessible to and supported by the community. If the community sees the police as effective, and fair, police will be able to actually have an impact on crime.

          Policing is hard. Very hard. Deliberately so: society needs a police force who are fair and effective but respectful and with limited powers. Giving the police crap tools (and resources and training) just makes their work even harder and society sees negative benefit.

          1. Baldrickk Silver badge

            Re: @ Graham Cobb

            Agreed, but where do you put those on-the-beat police?

            Do you spread them around evenly, or put the majority in areas where these crimes happen more, even if those areas may tend to be populated by people of a specific ethnicity?

            Reading an article last night, it suggested that these gangs responsible for most of the... incidents are actually multi-cultural in that there are many races in them, though they appear to be non-white for the most part.

        2. M.V. Lipvig

          Re: @ Graham Cobb

          Wait, gun crimes? In LONDON? How can that be? I thought the only guns welcome in England were shotguns, and then you had to prove legitimate need?

          1. Baldrickk Silver badge

            Re: @ Graham Cobb

            There are unwelcome guns too...

  6. dnicholas Bronze badge

    Is it still discrimination...

    ...when it's demonstrably true?

    Just saying

    1. Graham Cobb

      Re: Is it still discrimination...


      1. dnicholas Bronze badge

        Re: Is it still discrimination...

        That's why we have (predominantly black) kids killing each other on the streets.

        But wah wah racial profiling

  7. Abdul-Alhazred

    PC Schweik blames the computer.

    So instead of "everybody knowing" (for example) the South Side is the baddest part of town (as in the USA), PC Schweik blames the computer.

  8. cantankerous swineherd Silver badge

    lots of police on patrol hereabouts.

    mind how you go.

  9. FozzyBear Silver badge

    let's get a few things straight here

    "Predictive Policing" is just new jargon for crime analysis. You analyse what has happened in the past, To figure out what types of crimes may occur in certain areas on certain days and times. You then organise the police in that area to be in those areas, days and times that the crimes are most likely to occur. There are no sophisticated algorithms. Very basic statistics. You find out 40% of the Break, Enter and Steal offences occur on a Saturday night between 8pm and 3am, in a certain area, you make sure the police on patrol are in that area during that time frame.

    You couple this with "Community Policing". Remember the "ole days when you would see the same police officer(s) occasionally, or regularly, walking around your local area. They would know who lived in the area. If you were good they would greet you with a smile and maybe a quick chat on how you were doing. If you occasionally got into trouble. you would still get the greeting but it was followed with a warning. If you were a shitbag that didn't care about the law or the trouble you caused in the local area. They would stop you,,find out who you were with, where you were, where you are going and what you were doing. This was shared among the other officers in the local area and was built up over time. Apparently known as "Individual Risk Assessments" now days.

    Combine these two and it is the most effective tactic for reducing crime in a given area. No amount of predictive modelling will ever trump it.

    After 10 years in the police , on the street, as a detective and later as a crime analyst. I can personally attest, that the people who shout the loudest about the thuggery of police. Even the mere presence of them in their area or street are always the ones who cry the loudest. Demand the most immediate and brutal punishments. Blame the Police for the lack of foresight, skill or lack of action when a crime is committed against them. The hypocrisy is completely lost on them.

    Here endeth my rant for the day

  10. Ken Moorhouse Silver badge

    The 2011 London Riots

    The 2011 London Riots would arguably have been a good opportunity to study human behaviour without taking participants' ethnicity into consideration (Wikipedia quotes those arrested as 29% white and 39% black in London). What was exhibited there was innate reaction to various environmental stimulii, regardless of ethnicity. The number of people in a specified area using certain body language inciting others to act similarly, setting off a chain reaction, even to those drawn in from the periphery.

    Another research possibility is studying the effects of ambient temperature on crime statistics. So if the temperature is x in place abc, AI might be able to predict a corresponding percentage increase or decrease in crime in that locality.

    To my mind these are areas of "macro criminal behaviour" where I feel AI could be usefully harnessed, rather than micro or person to person interaction where prejudice is more likely to be baked into observations.

    1. 's water music Silver badge

      Re: The 2011 London Riots

      By happenstance, I came across this the other day. I am not qualified to comment on the quality of the research

  11. random mathematician

    Bias is self-perpetuating in lazy statistics

    Don't be fooled by "what if this group really is more criminal" - apart from the main factor that others have mentioned, which is that socioeconomic factors are more predictive than race, you can easily envision how bias can set up a feedback loop.

    Start with two groups, A and B, which actually have identical proportions of members who carry knives (or any other crime detectable by a search). Due to cultural bias, more people from group A are searched than from group B. Also due to bias, when charged, a greater proportion of members of group A are convicted than those from group B. Now take a look at the headline statistic of people convicted of knife offences - wow, people from group A are massively over-represented! We'd better make sure we focus our policing on them! Now not only do you have the support of those with (conscious or otherwise) bias against group A, you also have people with neither bias or the ability to properly check the validity of statistics.

    1. Joe Montana

      Re: Bias is self-perpetuating in lazy statistics

      It's self perpetuating but it also has to start from somewhere, the police didn't suddenly decide out of nowhere that they were going to target a particular group or area - such decisions were made due to a prevalence of crimes by those groups or in those areas.

      And such a bias can also fade over time, if the police are stopping and searching people from a specific group that historically committed crimes easily detectable during a search, but find that the proportion of searches which result in an arrest significantly decreases then they will refocus their efforts on more useful activities having achieved the goal of reducing crime in one area.

      1. Graham Cobb

        Re: Bias is self-perpetuating in lazy statistics

        No. If you read @random's post again you can see that as the incorrect results persist, and the bias gets greater, the numbers get even worse. Even if the situation changes and Group A are carrying fewer knives than Group B their numbers will still be worse because of the self-perpetuating selection bias.

        What is required to get meaningful data is that the samples (i.e. those stopped and searched) must be truly random (and also that bias is eliminated in later stages, like deciding whether to caution or prosecute, and at prosecution and trial stages). Otherwise, the data on which the profiling, analysis and AI are based is just complete garbage.

        1. Baldrickk Silver badge

          Re: Bias is self-perpetuating in lazy statistics

          But statistics come from far more than just stop and search, things like reports of crimes committed, which will report the actual numbers.

  12. John Smith 19 Gold badge

    Let me note

    Correlation <> causation

    Models with (hidden) positive feedback will drive an outcome to an extreme endpoint.

    But is "predictive policing" the cause or the effect?

    Cause my gut reckons quite a lot of this is due to something like this

  13. martinusher Silver badge

    The biggest danger is that it may be effective

    I used to live in one of those postcode areas where "If it wasn't nailed down it was immediately nicked and if it was nailed down then it just took a few minutes to pry it up". Most people in that area were ordinary people, more or less, with only a relative minority being the actual criminals. After a bit you even got to know who was the likely puerp, the cops knew it as well but there's a lot of distance between knowing who the villain was and catching them with enough evidence to nail them.

    The algorithms just systematize the informal data collection and area knowledge of the beat copper of old. They should be more objective than an individual and so a lot more effective. The algorithm may seem to be biased, especially when its first used, because it has only existing data to work on, data that is probably biased. It will rapidly adapt, though, and you'll quite likely to find that its stopped being interested in dark skinned people in wool hats and started zeroing in on ostensibly respectable people who almost seem to fit in but don't quite.

    I should remark that as an ex-inner city dweller I'm not particularly 'pro' police. They're only human so they can spend far too much time going after the low hanging fruit (i.e. bothering me and my friends). I never found them particularly biased against creed or color but they do get a bit cynical about offenders since that's their stock in trade. This is what tends to cause bias -- they see the same faces, the same sorts of people day in, day out, so they tend to make generalizations about people. The computer may go a long way to putting them right.

    1. Ken Moorhouse Silver badge

      Re: knowing who the villain was and catching them with enough evidence to nail them.

      "...if it was nailed [] then it just took a [defence solicitor] to pry it up"

      Sorry, couldn't resist that.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2019