It has its advantages: speed, reliability, etc.
But, if you have functional problems in your system and you have to repair some "non expected use cases".. it is a real pain.
The NHS has ripped the Oracle backbone from a national patient database system and inserted NoSQL running on an open-source stack. Spine2 has gone live following successful redevelopment including redeployment on new, x86 hardware. The project to replace Spine1 had been running for three years with Spine2 now undergoing a 45- …
Agreed, though from the admittedly fuzzy description, it sounds like it's a simple document store rather than a database. That would explain having Tornado and RabbitMQ in the mix.
If things do start to go wrong, it should be easy enough to move to Postgres hstore or jsonb and get some of the reliability back.
>> I'd not have put reliability on the NoSQL list.
Depends on the complexity and temporality of the data. SQL is all too often a golden hammer (https://en.wikipedia.org/wiki/Golden_hammer). Massively distributed message-oriented systems, such as Spine2 appears to be, can make ACID impossible (for a given level of performance, anyway).
All design is compromise. Software design is doubly so. (Apologies to Douglas Adams)
I don't hate Microsoft, I don't hate Oracle, I'm just miffed that this happened under a Conservative government, and that it was Nullabor who were in bed with Microsoft and the US consultancies.
Kudos, though, to Richard Bacon, the Conservative MP who spent so much time on the PAC going on about Government IT wastage and incompetence (as well as voting against the Iraq war). This is what he said in 2011:
I'm surprised, your surprised, about the Tories. I'm no fan, but Tory policy is and always has been to cut Government spending, so it is pretty much in line with that.
Labour have always been the bigger government party.
If your a middle earner, you're still screwed whoever you vote for, so from that side it makes little difference.
The problem with the Tories, is that they normally save money by robbing Peter to pay Paul.
Such as cutting the government's cost by increase in the cost for local councils instead.
Such as cutting government's borrowing by using PFI, which is still borrowing but through private channels meaning it's not counted as "government debt".
- Really? So what other way is there to save money. Look - unless you are New Labour who just get to imagine money that doesn't exist in order to buy votes, then 'Robbing Peter to Pay Paul' - or - 'Managing Funds' is how stuff gets paid for. There is only so much money in circulation, you starve one budget (Peter) to feed another (Paul) as and when required. If you have any financial responsibility in your day-to-day life then you 'rob Peter to pay Paul' every day - ie: 'Can't go for a curry with the lads tonight because the car insurance is due' or 'Will pay to get the guttering cleared to avoid paying damage due to damp'.
Honestly - how else do you think budgets get managed other than Robbing Peter to Pay Paul.
Failed, runiously expensive, overly ambitious, poorly defined, mis-managed government (especially NHS/defence) initiatives with endlessly deep pockets of public funding is what keeps the IT industry afloat - both parties agree to walk away, nothing has to be delivered, no harm done, but lots of lovely dinners and educational *events* have been shared - everybody (who isn't a Tax payer wins)...
....for going Open Source but they really should have looked at Postgress. They probably could have saved more by simply migrating to Intel/AMD and keeping Oracle instead of re-writing everything, unless it's a fairly trivial system in which case it was over specified to start with.
I can't image that they spent 3 years on this and didn't consider all the options, they must be a pretty savvy bunch if you read the other articles about this. I assume that the Riak system they built was the best option in this case.
I also doubt any system where peoples lives are at risk would be considered trivial.
Why does anyone think that moving from one DB technology to another is difficult? Any (reasonably) modern application should be designed with an abstraction layer to translate between different vendor DBs. When the SW licencing costs from the like of Oracle for these scale of DBs amounts to many £M, then the relative cost of migrating from one DB to another is likely to be tiny. Plus the proven ability to change vendor puts the user in a very strong position to negotiate favourable licencing terms. Most commercial apps use pretty generic SQL, migrating from say Oracle to DB2 is less difficult than you think - be brave - give your DB supplier a hard time, there are lots of options out there and the likes of Oracle know very well that the genie will soon be out of the bottle.
There are probably good reasons they dropped Oracle (aside from the high cost):
1. relational databases are good if you have lazy administrators - but all those views and joins come at a high cost.
2. their primary need was for a key-value system, which can be much more efficiently done without SQL. This will increase their price-performance point by a substantial margin.
3. if they had any sort of distributed requirements, they are much better off rolling their own (simply by adding a field to hold the date/time a record was updated and writing their own replication engine) - in tests I heard of using Oracle replication, it wasn't possible to replicate to more than 7 hosts before network overhead (of ACID transactions I suspect?) killed the system. Dropping ACID and using a custom replication engine (which wasn't very expensive) fixed the problem
I don't know of your spreading FUD for the sake of it or you are simply misguided.
1. RDBMSs are good because administration isn't onerous.
2.Not sure what your point is. Keys are a useful way to access data and make sense of it. They perform very well.
3. I can't comment on Oracle replication as I haven't used it but I have seen other RDBMSs replicate to over 200 remote sites. ACID really isn't that costly and I've often seen even average site manage thousands of ACID transaction a second which is unlikely to be needed in the case in the article. In memory databases up that again, to the point it is no longer a limiting factor even in the most exteme of applications.
Multi-master SQL databases to 200+ sites with full ACID, really? Master-slave, I get, master-master for all sites, I really am having a hard time believing.
Furthermore, RDBMS administration isn't onerous, but it's not easy either (and it's certainly a specific skill-set). A lot of NoSQL databases are much easier to administer, and can be administered by standard SA's, not DB SA's. You can get a multinode, fully HA Riak cluster up in literally 3 minutes and start transacting against it. Dynamically adding and removing nodes in that cluster also can happen on the fly. I'd like to see any classical RDBMS do that.
"for going Open Source but they really should have looked at Postgress"
They might as well have used MSSQL server if that was the case - it has a lot fewer security holes and a lot more functionality for a start. The clear thing here is that the database doesn't really do much other than act as a big indexed file store - so crappy and limited feature products like No SQL can do the job...
>They might as well have used MSSQL server if that was the case - it has a lot fewer security holes and a lot more functionality for a start.
Did you even read the article ?
1. They want to save money
2. They said RDBMS is overkill
BTW, nobody in their right mind would consider MSSQL for anything
If they wanna save even more, when are they ditching Windows ?
also lucky to hear a british accent amongst a lot of the doctors and nurses in many urban hospitals.
This can be issue when are attempting to understand how life threatening an illness is from medical staff. I really didn't need any additional stress when my wife had our two kids, and they "forgot" to give her any food for dinner on one occasion despite her asking a number of times!
I don't care where people come from, is the level of service I worry about!
"It logs the non-clinical information on 80 million people in Britain"
Er, what? This country only has a population of 64.1 million (http://ons.gov.uk/ons/taxonomy/index.html?nscl=Population)
Surely you can't account for another ~16 million records from births/deaths turnover, duplicate records, health tourists, etc ??
> ... so Riak must be more expensive than SQL/Server then...
On the reasoning that the "cheapness" quotient of Oracle's stuff must be negative? Does seem like the only remotely coherent interpretation :-|
Methinks English is not the author's native tongue.
"It processes more than 500 complex messages a second."
There are some 36,000 GPs in the country, seeing an average of one patient every 12 minutes; making a total of 50 patients per second. That's an order of magnitude less than the number of messages per second that the system is processing. What are all these messages?
I'm a little curious about the claim of the messages being 'complex', but to your main point, and I'm just guessing here, a gp could lookup patient details more than once per consultation. In addition, writing a prescription might count as one more, and that may trigger off other internal messages to check for drug allergies or suchlike (and multiple prescriptions might multiply that?), and if a patient were referred to a hospital I can well imagine that too kicking off a little cascade of internal messages.
I repeat, just guessing but it seems possible.
I know nothing about this system but in my doctor's surgery (not in the UK) the receptionist pulls up the patient record before the appointment to grab details, gets on with other queries and only after the appointment fetches the details again and then applies necessary updates.
I see a minimum of three messages there, and that's before the extra lookups other posters have suggested.
It's not just GPs that generate the data, hospitals do, too. And each visit to a health professional that uses the system can generate multiple messages, e.g. referral, prescription, operation note, observation of reaction to medication, and many, many other things. Without knowing what the system covers it is impossible to tell, but in light of that 500 messages/second sounds rather low.
Some stuff will be entirely internal to the surgery system.
A GP could have a consultation, write a prescription, book the patient back for three more visits and it could all be internal up save the prescription being sent to HSCIC for the central records eg the GP Presentation Level Prescribing Data (available for download in monthly chunks if you fancy flicking through it)
But initiate a scan, or some tests and the messages are sure to start flying....
"There are some 36,000 GPs in the country, seeing an average of one patient every 12 minutes; making a total of 50 patients per second. That's an order of magnitude less than the number of messages per second that the system is processing. What are all these messages?"
You're assuming that GP's are the only users of this system.
How many pharmacies are there (that have a link into the system)?
How many hospitals?
"It processes more than 500 complex messages a second."
Is that all? In the late 80s/early 90s I worked for an airline CRS processing 20K entries/second of a *roughly* equivalent complexity.
Mind you, we had a nice big S/370 (actually two - one for comms and one for agent entries) to play with and as much DASD space as you could fit in a football (UK edition) field-sized room. And a nice dark room to trap newbie tape monkeys in.
Kids today eh? They think they invented it all...
Precisely, not only do these children not even know what a transactiojn monitor is, they have no clue why you would even need one. They are still in the sandpit building sand castles, watching them collapse and then blaming the collapse on the type of sand and move to the next sand pit.
It is as if 40-50 years of accumulated systems software knowledge was jettisoned when VB was invented!
The youf of today, our future - eeeeeeeeek!
That may be true but I'm sure that NoSQL will have its own class of vulnerabilities when subjected to poor coding.
The point is that poor coding is the fault, not the technology in the case highlighted. Oracle is my background so I know it is easy to prevent, I assume that it is similarly trivial in other SQL implementations to avoid SQL Injection.
Your original point is not an Oracle problem, but a coding problem, of which I'm sure NoSQL has its own versions of such problems...
"So you think they should have stuck to numerous incompatible local systems?"
Do you know why ARPA net (whose protocols the internet uses) was built?
To allow researchers to at remote sites to access incompatible machines without needing multiple terminal (apparently some really did work that way) on their desks.
And by incompatible I mean machines that could not even agree how many bits they have in a byte, or what alphabet to use.
IOW it's all about setting up data standards, mandating that different parts of the NHS use them and letting the individual parts (and the are very individual parts in the NHS) work out how to implement them.
So no I don't think p**sing away £15Bn to get systems to share data better was needed or good value for money.
"Prior to this they didn't even have a global NHS email system".
Just a single address book to hack and you can spam the whole NHS.
"Different kind of querying, direct key lookup, secondary indexes, range scans etc etc"
Errr, which commercial relational DBMS doesn't have these things?
Nothing you mention is even remotely different or state of the art. You should perhaps familiarise yourself with how commercial relational database management systems work, and why they work that way. After you get a basic understanding of what is going on, you can then tackle query optimization, which is the actual "secret sauce" that makes things work.
There is a class of applications for which object store style databases are appropriate, and this case might indeed have been one. It detracts not a jot from the power of the relational model (as distinct from implementations) for solving the vast majority of data base problems.
The problem with Oracle pricing is often not even the number of UK Dollars. It's the sheer hassle. Oracle contracts are so complex that they make the Goatish One's bids for your soul look like a picnic. Especially if you are beyond the trivial size. Oracle comes knocking if you add more CPUs to your system, if you assign more CPU capacity to your virtual machines, unless you also want to pay licenses for the VMs that you *don't* run oracle on for some reason. If your database grows beyond a certain size. If you have more users than you used to have.
With open source solutions, you can merrily shift data from one server to the other, grow it as big as you want, for as many people as you want and nobody gives a damn.
Oracle pricing is based on a complex rolladex system on someone's desk. It simply states the amount in 'protection' you pay. It doesn't really matter what products you use, as it's a number that can only go up, not down.
"What's that you say you don't need product x on 40 servers, that's fine. By the way your 'discount' on product Y has just been halved due to falling below our 'bulk buy' threshold."
Bill before <= Bill after
I share your skepticism, but NHS no doubt has been running the numbers & picked a solid project to blaze the way.
ACID isn't the sole domain of SQL engines, & note they avoided Berkeley DB (which is Oracle's kv db that scales) & went with a set of Erlang sol'ns.
Oracle's racket in IT is well known & viewed with disdain amongst those who have the freedom to know better. But then the software gravy train has been marshalled by all sorts of Barnums & Ponzis, incl. CA, SCO & M$. It's all coming undone due to multicore, virtualization, big data & handhelds ( which themselves will soon sport 100x the computing power of a Intel 486). It's a great time to start a grass roots project what with the maturity of libs & languages, api & protocols, with enuf power to keep things aloft.
Just when you thought that Erlang had given away to Go, the Ericsonian Prologists scale the walls of the Citadel. 'Let it fail' is their motto ( pass the message). This is the Zen of functional actors, & neither M$ nor Ellison's yacht fags know what to do about it.
Biting the hand that feeds IT © 1998–2019