Uncle Sam wants to tackle bias in algorithms by ordering tech corps to explain how their machines really work

Theoretically this should be bipartisan

Conservatives keep complaining that e.g. Google's search algorithms are biased against them. Or do they just want to whine about it and make up conspiracy theories, and not risk looking to see whether that's actually the case or not?

Though senators are probably assuming there's some "source code" that could be looked at to see if there's something like "if user is black then don't show rental to them" or "if article is anti-abortion then bury it on page 6 of search results". With a neural network all you can do is send it sample inputs and see what it does with them on an individual basis, you won't know WHY those actions are taken, so such regulation is not going to produce the intended outcome.

Right. If they knew how to arrive at the same decision it would be possible to implement a decision tree (if... else... fi).

What they are asking about are (among other things) biases in the training data and how these could be corrected for. Let's see what will happen, and if it changes anything at all.

If the algorithm isn't following any kind of logic, then why should we imagine its decisions will provide any benefit?

If it is following some kind of logic, then it's not unreasonable to require that the owner of the algorithm should be able, on demand, to explain its process. The decision tree may be unfeasibly complicated to put into a generalised if... then structure, but it should still be possible to map the path to any particular conclusion.

If it doesn't allow for that level of accountability, then it's not fit for use.

One problem not addressed is all decisions are made in a partial information vacuum, a fog if you will. When one makes a decision, one does not all the relevant facts about the issue and may never know all the relevant facts afterwards. So one makes a decision based on experience, information at hand, and some navel gazing. How murky the fog is various but it is there. Many wrong decisions in hindsight were not poor or stupid decisions based on what was known at the time.

Re: The best way to stop this nonsense...

Already exists - called dice.

Re: Garbage in, garbage out

The same Google that used its massive database to predict flu outbreaks. Only to find that its predictions were..less than stellar.

@Jack of Shadows

Thanks for the tip on that paper!

No one, it seems, is doing the insane things I was doing as far back as 1987 with non-parametric, non-linear, let alone autonomous. Don't ask!

I don't need to ask, I played with that a bit in the 90s too... Brrrr...

Testing vs secret sauce

"The problem I do see in play is that the FTC will be completely incapable of ever evaluation the truthfulness of the submitted document as it will always lack the expertise unless it has the budget."

The FTC don't actually need to see under the hood. What they need to be able to do is test the system to see what results it is given. If they show bias, then that's all they need to show the company isn't up to scratch.

TRaining data

"But I can't help suspecting that talk of prejudices and biases is unhelpful. "

As a data scientist, I disagree. It's fairly easy to make sure your learning algo isn't inherently biased, or biased in the way you want*.

Ensuring that the data you train it on isn't biased is much harder. Since real world data often is influenced by societal biases, even relying on accurate data alone won't mitigate it.

I have argued that using only data from the majority group as the training set might result in less bias, but YMMV.

Another issue is that certain items of information are protected, in that they can't be used legally to influence the decision. It's fairly trivial to deduce protected information, so does deduction count as knowing? So instead of being biased on skin colour (because it's not shown to the algo) it's biased on region. Instead of saying "no pakis" it says "no-one from Bradford"

So unfortunately bias is the biggest issue with automated interactions, as once you've trained it, you can't pull those biases out. If it's a human with biases, you can at least retrain them.

* human diagnosis, you care more about deadly diseases and those you can test for than straight percentage likelihood of which illness is causing the presented symptons. So you bias it towards reporting even 1 in a million possibilities.

We've long had this problem in disparate impact analyses where a particular sample, say a corporation or hiring or really just about everywhere people are under a microscope, doesn't match the population at large and, therefore, groups are forced into some sort of consent decree to force that sample environment towards matching the population. None of the agencies, courts, legislators bothers with figuring out why a small group doesn't match population, the automatic assertion is "because discrimination!"

Sorry, statistics (and computer science) is my first degree. If I turned up a sample that matched the population I'd be wondering how that happened. Variance, the average of the sum of the square of the deviations from the mean, are looked at and expected to be non-zero. Whatev's.

So, I have little hope for any application of skepticism here about why something doesn't produce "desired" results even before we get into the correlation/causation examination. Sometimes "too hard!" is a valid point.

"It's really easy to explain. You start off with your standard multilevel regression and post-stratification model, you see, but then..."