Errors in Australia's Centrelink debt recovery system were inevitable Since it announced a crackdown on outstanding debt in June last year, Centrelink has sent debt recovery letters to thousands of Australian welfare recipients. Early reports indicated that around 20 per cent were issued in error, although the true number may be substantially higher. This led to the appointment last week of a …
So how many payments did they manage to extract from non-debtors. If the amount is small enough do they just pay to make it go away. How is the distinguished from false invoice fraud. If this was a non-state actor they would be prosecuted for a single instance of non-valid demand for payment, but if you're the state... No imagine the defence for an Ozzie invoice fraudster!
The Human Services Minister, Alan Tudge, says that sending a letter to a non-debtor is not an error.
In that case, the minister won't mind if every non-debtor recipient of a C/Link letter sends him an invoice for waste of time and mental cruelty, will he?
Where are ambulance-chasing class-action, no-win no-fee lawyers when you need them...
Once upon a time people used accounting systems, not AI, big data or anything else. Accounting systems can summarise each account with a balance. That balance tells you whether you're in debt, credit or neither. Why complicate a simple process by throwing AI at it?
"Whether 99% accuracy is a good result or not depends very much on the split between false positives, false negatives AND the underlying base rate "
Where the decision leads to legal action the acceptable error rate for false positives is zero. If you were accused of some offence of which you were innocent would you consider it OK to be found guilty and incarcerated providing the false positive rate was considered acceptable?
False negatives are trickier. A false negative can also lead to injustice* beyond the occasional guilty party escaping on grounds of reasonable doubt.
Such high standards makes criminal investigation a stressful occupation for anyone taking it sufficiently seriously.
*A complainant of assault being wrongly disbelieved could come into this category.
"common-or-garden SQL database?"
They were solving that even before relational databases were a glint in Ted Codd's eye. They were solving it in the days of tape-to-tape. They were solving it in the days of mechanical accounting machines. They were solving it the days of quill pens & paper. It all goes to show that if you want to fail really badly, use a computer.
It's actually much simpler than that.
It's to due with using source of data in a way it wasn't used before.
Eligibility for many Centrelink payments are based on a fortnightly (2 week) cycle. It doesn't matter what you earned in the previous fortnight or will earn in the next, the payment for this fortnight is based on this fortnights specific circumstances (income etc).
However, since for taxation purposes it's the total FY year that matters, the Australian Tax Office (ATO) only cares about your Financial Year TOTAL income, therefore the ATO only keeps, and provides when requested by other agencies like Centrelink, the FY 12-month total amount of taxable income.
So, say, 5 years ago this was your FY situation:
fortnight 1: you earn only $500, you might be eligible for, say, $100 in rent assistance. So you get $100.
fortnight 2: you earn $1000, and are not eligible for any rent assistance.
fortnight 3 and 4: same, $1k ea, no rent assistance.
fortnight 5: you again earn much less, say only $300, so you are now eligible for say a $150 rent assistance, so you get paid $150 (total $250 so far).
fortnight 6 and 7: you earn $2000 a fortnight.
fortnight 8-11: you get fired, no job, no income, so you get $250/fortnight in assistance, for another $1000, totaling $1250 for the financial year so far.
fortnight 12-26: you find a job, a decent job paying 3k/fortnight in the remaining 15 fortnights of the year, for $45k, plus the previously earned amount is $52800.
So ATO's records show u made $52800 in the FY. ATO gives that data to Centrelink, who then divide that total by 26, for a tad over $2k per fortnight.
Then the Centrelink datamatching algorithms go, hey, we paid this person $1250 in rent assistance, but they've earned $2k per fortnight, therefore they would never have been eligible for rent assistance, send letter demanding return of the $1250. So unless you can prove that you were entitled to those payments, that in those particular fortnights 5 years ago that your income was with the entitlement range, you are screwed.
So it is entirely the fault of whoever decided to use unfiltered ATO data in the way it is being used.
@eldakka
Thanks for that. It means that if this system issues debt letters solely on this basis it's not fit for purpose. As I explained in a comment above the acceptable rate for false positives is zero and what you describe cannot avoid false positives.
In a human-operated system a case would be built by one team, e.g. police, and passed to a second, DPP, CPS or whatever for review before it gets to court. The second team and the court itself should act primarily as checks on the first team's work, not as an excuse for the first team to get away with sloppy work.
In the system you describe the output should be no more than a list of cases to be checked against the original fortnightly data. It should not be possible to issue debt letters which do not give the dates and figures for the incorrect payments.
It's actually worse than that. Each employer has the ability to enter a start and finish date of employment and submit that to the tax office. So lets say your example was reversed where the person was working the first 6 months and then fired but the employer put the entire financial year as the employment period. Then the debt matching assumes that the ATO data is 100% correct and all the other Centrelink data is incorrect (i.e. letter of termination, etc) and you owe everything you have been paid since termination since you were obviously still working and Centrelink employees were too stupid to work that out.
The second part is people studying, since you only have one set of dates to pass to the ATO someone who works at a supermarket at night on and off for a financial year will be flagged at getting a constant amount due to the fact that the ATO data is 100% correct and we shouldn't pay you while you said you were 'studying' so pay it back (and working around Christmas, Easter, etc when there is a lot of cash involved can certainly make it interesting).
I think this the wrong name for the system - it should be the 'Centrelink Debt Creation System'.
From the Governments point of view it appears to be the perfect system as it shifts the onus to the recipient to prove they are innocent. The effort to sort this all out later must have increased many times so would it not just be shifting the cost and adding to the paperwork that Centrelink has to follow up or do they just have a follow up system that says 'Computer says no'.
Excellent article. However, the data matching challenge for Centrelink didn't require AI or complicated heuristics. In the end, it was a matter of understanding the two data sets. Someone did. But, someone (probably someone else) decided the fact there would be false positives didn't matter. They decided that any case where annual income reported to the ATO converted to fortnightly was higher than reported to Centrelink should be treated as a valid positive until proved otherwise. Someone, crudely, didn't care. More, they also decided that the same employer name spelled differently in the two data sets should always be treated as two different employers thus doubling the purported wages paid (i.e. one wage reported to ATO and another wage reported to Centrelink). Again this wasn't a mathematical challenge. This was a crude effort to inflate the numbers. Perhaps this started as a rash promise to the Minister by a self-important public servant and escalated from there (I was in the APS for 34 years and have seen such self important people stuff up advice to the Minister).
Also, the bureaucratic ancestor of Centrelink, the Department of Social Security, was doing solid data matching on (mainframe) computers in 1980. They were even matching data across different binary encoding standards. They were clever. They picked the highest risks and correctly found the overpayments and underpayments. They knew what they were doing and they did it well. These days, such compentant people don't get promoted and are ignored when it comes to designing important systems.
If this was a private company, there would be consequences from the regulator.
One way or another the governement will take half of what we earn. It does not matter if you call it a petrol tax, or a licence to provide a phone service, or an employers national insurance, tax on cigarettes or VAT. The trick is to keep it cheap to collect, and easy to enforce so it is seen to be fair. Both overpayment and underpayment is a big problem for low incomes. If you are overpayed, you are likely to get a correction at some point later and you may not have the reserves to cover it.
If you can not calculate accurately the benefits now, you need to rework the benefits and the billing systems until you can actually do the maths. You can require employers to use payrole packages certified to report the info the government needs each time payrole is run, eg weekly or monthy.
Even if payment itself was cash in hand.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
