Big data is big business right now, and it will not only be embiggening as this decade rolls on, but it will become normal, like enterprise resource planning, supply chain and customer management, and other perfectly normal parts of the corporate computing landscape. But that will only happen, it seems, if we get people involved …
"...becoming data scientists..."
Oh god what a waste.
Re: "...becoming data scientists..."
Train as a data scientist
Then watch as your job is off shored a few months later.
"This looks like a situation that could be fixed if companies spent less buying back shares and wasting money in countless other ways and actually invested in training the big data workforce they think they will need"
This assumes you can train people to do this. Although occasionally that may be true, it takes a certain mindset to work on this kind of thing. Even with Windows Server you often find that "trained" staff don't have a clue how to use it or how it works, they just manage until someone smarter comes along and fixes their mistakes.
This is a similar mistake to that other common one, that Architect is a more senior Consultant, and Consultant is a more senior Engineer. They are in fact wildly differing skillsets which suit different people.
Its interesting that you made that comment.
Some major corporations have taken that same attitude and have decided to push this work off to their offshore lower cost labor pool.
Over time, they may get it. Things will be easier as the tools evolve.
But the truth is that Big Data increases the required skills by an order of a magnitude.
You go cheap early, you get burned and realize that you've wasted your money and time.
"Some major corporations have taken that same attitude and have decided to push this work off to their offshore lower cost labor pool"
AC didn't say anything about offshoring. The important factor here is skills, and big data is not something you can really teach to a person with no prior interest or experience. It's the same with storage, I see lots of people making lots of mistakes with SAN because they were sent on a training course to learn how to use it. Even now, customers are asking for storage based on capacity and I suspect that big data will be the same where around 1% of "experts" know their stuff and the rest are bluffing.
We will need a lots of space
.. ....for all this Big Data
Getting and storing 'lots' of 'data' is never an issue.
Deriving something meaningful always is.
But, big data people, welcome to the curse of dimensionality.
Too true: "Adding hay does not make finding needles easier" is one of my favourite sayings when discussing pattern recognition. The key to successfully finding things is in reducing data by knowing exactly how to obtain all relevant information rather than just flinging more data at the problem.
Harry and Michael, give yourselves honorary degrees from the University of the Fucking Obvious.
Agreed, Steve, it is! it is fucking obvious
Somehow the fucking obvious escapes many people
Take politicians. How many projects are out there based on the premise that as long as we collect ALL the data we will be in a better position to find terrorists/paedophiles/other undesirables? Lets take fingerprints from everybody entering the country! Hooray!! We have struck a blow in the war on terror!!!
No, you are just showing the general public that you are doing something, by pissing them off big time every time they enter your country.
When will these guys learn that data is not the same as intelligence?
Mine is the one with the shiny new degree from the UFO (sorry, couldn't miss the chance to hijack that acronym for the University of the Fucking Obvious)
And your post reflects why big data is to all and intents mostly just noise. Because most of it is meaningless and add nothing.
Did you have anything constructive to say before doing your paper round?