Wading through a world of uncertainty

Programming fail

loggedin.aspx - "You won't see this because this page basically redirects to the relevant page based on role type"

O RLY?

This does not fill me with confidence regarding the qualities of the "game".

not a survey site

It does route you to one of those tedious survey sites, but you can ignore that, go back and take the quizzes.

The couple I tried where simple. You score points for a combination of the right answer and your certainty that you are right.

One quiz told me I had a 90% certainty level because I was 100% certain about 9 questions and not sure at all about the 10th. That's a simplistic measure of certainty....if I'd been 90% sure of each question, I'd have the same certainty level but my approach to answering the questions would be subtly different.

Still, it's an ongoing research project, so they can fix that sort of stuff with enough feedback,

genuine research vehicle ?

Well if it is a data scavenging scam, they suckered me in. But after taking enough tests to build a statistical sample, I would say that it is actually a genuine research vehicle, not just somebody's get-rich-quick scheme. It did get me thinking about risk management in decision making which is its stated purpose.

Finished

Apparently this project was set to be completed in June 2010, so what you see is the polished completed product.

http://gow.epsrc.ac.uk/ViewGrant.aspx?GrantRef=EP/E018092/1

The mind boggles.

buggy? unpolished?

sounds about right for software coming out of QUB CS School.

anon 'cos I studied there.

The point is to calibrate your uncertainty

Once you have done more than 4 quizzes, you get a calibration curve, showing where you are over- or under- confident. The point is to see if it is possible to develop your skills in estimating your subjective Bayesian probability. The scoring formula used is Brier's proper scoring rule - but modified so that you cannot get negative scores.

The point was not to make a commercial game, but to develop a research tool to help us find out if people can improve their skills in estimating odds. It was tested on over 500 members of iPoints. It turns out that if you are in a hurry, you don't improve at anything - in fact you get worse. If you take more than 20 or 30 seconds per question (i.e. you stop and think), you get better over time.

As for the coding, it was done by a games developer attached to Brunel University - nothing to do with Queen's University Belfast (where a Ph.D. student has been researching the learning effects in the Management School, not CS). In any case, a commercial game costs $10,000 to build and test. There is a limit to what one person can do when contracted part-time over a year. The big difference from commercial games is that the uncertainty is explicit, not hidden in the code that drives a monster. Imagine someone developing an uncertainty engine, analogous to physics engines, that could then be used in commercial games.