Why Enterprise Analytics - and why now?

Most businesses already have a strategy

You'll have a tough time ttrying to sell a non-intuitive conclusion to a CEO - no matter how many scads of data you have "processed". If you can't present a consistent (and simple: very, very simple) argument about why a particular strategy will bring huge returns, there's little possibility that they'll risk their career and bonus on something that doesn't appear to make sense.

Most companies have a very straightforward and well understood business strategy: find out what the competition's doing and do the same. It's not so much about making huge profits, as avoiding huge losses. While no CEO ever got fired for doubling the dividend, not many get fired for making a loss - provided all the other, similar, companies are doing the same.

smart data dumb tools

I like it .. so HPC is like getting a more powerful power tool. As you correctly identify the quality of your shelves is determined by the lowest common denominator in the chain of tools in the process.

Similarly with enterprise analytics the ability to process large amounts of data is not likely to be the limiting factor. The reason that fluid dynamics is such a good application for HPC is that there is a well understood model that is computationally intensive.

Trying to find the same in a business context leads me to madness or down the route of dealing systems based more on game theory than predictive modelling.

Corporate IT has kindly enabled me to pull enormous volumes of data going back 10 years to support the development of my regression models, and boy are we pleased with ourselves. Technically I can build an effective predictive model from a few thousand records and it will be just a useless as one built using the whole data warehouse. 10 years of historic data tells me nothing in a world/market/game that is constantly changing. The past is not a good indication of the future, the markets constantly change and the game landscape (other players actions) constantly change.

The challenging is to get accurate, fast feedback or measures…. and to make quick decisions based on this in order to react faster than the competition… and the chain of tools that must be employed is the corporate landscape is looooong.

I can build simple adaptive models that react to the changes that I suspect would be very effective but I don’t have reliable real time data measured at point of delivery, (ok and I have problems with good evaluation functions). Furthermore I don’t subscribe to the view that you need to waste time extracting the data.. why hawk data out of the environment it is collected in when we can embed adaptive models in the front line systems. But this is not likely to happen as we in IT continue to ‘solve problems’, ‘meet requirements’ based on the premise of data warehouses and the need to process large amounts of data.

“Smart Tools…Dumb Data” …. aaargh they have go me at it now.

Sorry that should of course have read give me “Smart Data…Dumb Tools”

Those aren’t easy questions to answer, particularly by an amateur economist.

What cheek.

The average guy reading this will have an IQ over 130 and ranging up to 170, the average woman reading this, over 115 ranging up to maybe 150, and the average economist?

To find out who knows what they're talking about, you simply have to identify all the economists who've made a mint buying puts because they predicted the collapse, and have since retired to Provence. Oh. that's right, there aren't any. They're all still putting their names to letters saying how good Gordon Brown's policy of tax and spend is, just like they did last time.

I once ended up in Bermuda in 91, at a wedding with a guy who used to work for the IMF in New York, discussing my ideas on the domino effect that was occurring out of the far east, having just spent the previous day on a fantastic course (92 - drove 340 yards on the first,) with a high ranking fed official, who told me in confidence that the Fed weren't going to do anything substantial and were going to let it run its course, (although this was all pure co-incidence, my wife booked the holiday, she was bridesmaid.)

We talked about economics, and he said, economic theory was so simple that by the mid 70s you could model it with flowing water and pipes. That was, he noted, until Save the Whales came along. After that, he concluded, economics became merely a side effect of the shift of human behaviour, public opinion could simply block a hole, like Nestle's problems, or Barclays in ZA.

The way to forecast is therefore to get someone directing your business who is astonishingly clever, but is also a cynic because that's what customers are.