It was Microsoft wot done it

SQL of ten years ago

I think it's perfectly true to say that SQL Server 6.5 was a terrible product. In fairness to Microsoft much of the codebase came from Sybase SQL Server and the product wasn't rewritten until version 7. In the years since SQL Server has become quite a good database, in our SAP implementation there has never been a database crash in it's existence (we've had it about 7 years). I agree though that Microsoft was being heavy-handed but really they were protecting their interests. I have been involved with Oracle projects using their software some of which of absolutely appalling quality but we all had to sign gag orders before getting the code. In fact quite a few times Oracle's own supposed experts couldn't even install and run their own software.

What did RON use?

Remember the Office of National Statistics RON system that achieved a whole new level of suck:

http://www.theregister.co.uk/2007/05/03/ons_computer_delay/

That was adapted from Mantech's VRVweb I guess from the story:

http://investor.mantech.com/phoenix.zhtml?c=130660&p=irol-newsArticle&ID=434107&highlight=

"ManTech's next-generation VRVweb(R) system is a Web-based, integrated vital records application with modules for Birth, Death, Fetal Death, Marriage and Divorce."

And a quick search for [sql server site:statistics.gov.uk] suggests they prefer IIS+SQL Server for their web stuff. (It returns a bunch of error messages from SQL Server's ODBC driver).

So anyone care to name the technology used in RON?

Re. MS Spot Changing

> can you point us to a modern version of the Bloor Report?

Yes, for example you can download the report on SQL Server 2005 free from Bloor's website.

http://www.bloor-research.com/research/product_evaluation/

741/microsoft_sql_server_2005.html

A couple of quotes from it:

"SQL Server 2005 is highly scalable...."

and

"We have not previously considered SQL Server to be a serious contender for data warehousing, at least not for large sites. However, Microsoft has made significant inroads into the market over the last few years."

suggest that Bloor now sees the product as much improved. In addition, the existence of such a report tells us that Microsoft does give permission for testing and I'm sure that companies like Bloor now insist that they get said permission is writing.

> why believe that the company or their product is any way changed?

My beliefs about the change in the product are based in part on reports like this one but also on my own experience with SQL Server, Oracle and DB2.

10 years ago there was, in my opinion, a huge gulf between the products – now there is not. They still have differing sets of strengths and weaknesses; but at least they are all in the same ball park (cricket pitch for our UK readers).

My belief about the change in the company is based on my interaction with it over the years. Am I swayed by the PR? Of course not; just like everyone else I have ever met, I KNOW that I am personally unaffected by marketing and/or advertising…….

We need a free press...

"suggest that Bloor now sees the product as much improved. In addition, the existence of such a report tells us that Microsoft does give permission for testing and I'm sure that companies like Bloor now insist that they get said permission is writing."

Well, no they gave permission for this report. It doesn't follow that Bloor's would be allowed to publish negative ones. Or give blanket pre-permission sight unseen.

That search [sql server site:statistics.gov.uk] returns a bunch of errors which doesn't look like a fully fixed product. A lot of them are query bugs, but a lot are timeouts and connect failures.

statistics.gov.uk is a 25000 rank site, which isn't a particular busy site, about 8000 uniques a day, 16 a minute for an 8 hour working day. A unique visitor every 3.5 seconds, say 3 page views, thats 1 page a second.

Even assuming Google crawl adds a bit to that, that's lot of failures for a light load.

Error: -2147217871 [Microsoft][ODBC SQL Server Driver]Timeout expired in Explorer.ASP: ErrorState = Failed to run directory explorer query ...

Error: -2147217900 [Microsoft][ODBC SQL Server Driver][SQL Server]There is insufficient system memory to run this query.

etc.

Database Reports

Now a days, it is more likely that political correctness prevents honest reporting. For example, few are willing to publish MySQL's abysmally awful TPC-C benchmarks. To even suggest that they should participate in the benchmark is generally called "FUD".

Scientific?

So the best, most scientific testing methodology is to find a random website that uses product X, get it to produce an error message and assume (having no knowledge of the setup) that the database must be causing the problem (and not the fact that they are, for instance, using an obsolete driver?)

I'd suspect that if Bloor had been up for a fight, Microsoft would have had difficulty. For instance (I assume this was in America) news organisations have some constitutional protection. For another, the validity of shrink wrap contracts is all a bit dubious, particularly when they remove rights you could reasonably assume to get in return for handing your dollars over the counter.

Title

"So the best, most scientific testing methodology is to find a random website that uses product X, get it to produce an error message and assume"

Read comment 3 to understand why.

"For another, the validity of shrink wrap contracts is all a bit dubious"

The real problem here is, if people aren't allowed to make reports showing SQL Server poor performance then how do you know it's not a dog and MS is covering up?

If they've done nothing wrong, what do they have to hide?

Nothing to hide...

So, (whoever the anonymous post was) you think that if someone/some company has done nothing wrong they have nothing to hide?

1) You have posted anonymously, that suggests, by your logic that you think you have something to hide.

2) In case you haven't noticed, there are loads of people out to get Microsoft. (No I'm not suggesting that they haven't done anything wrong and don't have anything to hide)

3) I could go off on a general civil liberties rant here, but I will merely mention that every person that I know who "has nothing to hide", also has curtains.

Stupid example

"So the best, most scientific testing methodology is to find a random website that uses product X, get it to produce an error message and assume"

"Read comment 3 to understand why."

er no, any badly designed and configured database is going to result in errors, especially time out errors. That's not the product at fault, its the admin/designers/developers fault.

Next you'll be blaming every broken down car on the manufacturer regardless of how conscientious or not the owner is regarding routine servicing and checking.

Can't handle 1 query a second reliably

"er no, any badly designed and configured database is going to result in errors, especially time out errors."

An average 1 query a second, results in timeout errors?

Again, I'd want to see free and independent benchmarks before accepting that no other database can deliver and average load of 1 query a second. Am I asking too much?

"Next you'll be blaming every broken down car on the manufacturer regardless of how conscientious or not the owner is regarding routine servicing and checking."

J.D.Power annual survey exists for car, why not databases? Why should they have the right to block bad benchmarks?

Fraser:

"So, (whoever the anonymous post was) you think that if someone/some company has done nothing wrong they have nothing to hide?"

Clearly I think they're hiding their benchmarks because they show something is wrong. Nested selects performance for example, I get very poor performance from SQL Server for queries that use nested selects. I'd like to be able to read benchmarks from sources I trust to tell me how each database performs with these.

But as it stands, bad benchmarks would prevent the report being distributed.

This is not acceptable, why should Boors be prevented from telling the truth about a bad product?

Mr Crank

"An average 1 query a second, results in timeout errors?"

This was an estimation by another commenter (not a respected database benchmark tester) based on a performance of a single website, not inspection of the database itself and certainly not taking into account how badly the database was configured.

And what's more even 1 query a year can still result in a time out error if a badly written query meets a badly designed and un-optimised database.

You seem to have completely missed the point that you cannot slate a product by one person highlighting one bad implementation.

yes the products should be benchmarked, but at the same time, database installations and configurations are not always straight forward and it would be fair to invite all vendors to send representatives along to assist with the testing.

And as for JD Power... a completely different method of assessing product function/performance through user surveys. Not really benchmarking now is it

Freedom

Now what I'd really like to know is how is it possible that the EULA can suppress constitutional rights to freedom of speech. Is it? Probably yes, but isn't it sort of absurd?

If you think something published was a libel, slander, whatever, than sue. Same if you think the benchmark was "bad". Isn't that how it's supposed to work?