Reply to post:

It's like crack for system architects, too.

A certain regional water company has relatively recently started shoving all of its telemetry and SCADA data into Azure data lakes. All of it. Terabytes per year. Now, Azure data lakes are very cheap - it doesn't cost much for the data to just sit there, so fair enough. However, when the time comes to do something with that data, that's when they've got you by the shorties.

There will be so much data that it will need to be partitioned in some way before you have at it with something like Hadoop (or whatever the MS equivalent is), which will mean:

* An IT project in its own right within the organisation;

* Hiring of big data analysts/specialists;

* Hadoop/Map Reduce specialists;

* Additonal cloud resources the churn the data, partition it, validate it.

And then, when you realise how much it's going to cost you also realise that they've got all your data. I mean seriously, what are you going to do? Ask them to post it to you on a USB stick?

I thought the idea of cloud was you hired the infrastructure for the time you needed it, churned, got your results, and span it down. Just like it was in the 60s and 70s. But modern IT departments seem to be falling into the trap of 'the cloud' - presumably because it is easy to use, and can be very cost effective. But you need to strictly control what you're using it for, and so far I'm not seeing that. Therby it becomes a false economy; they're merely shifting the costs into a different column.