As a DBA I would say that the majority of developers I see working on databases have no concept of performance considerations.

There are often tuning opportunities well beyond the order of magnitude - you can often find things that take 10 - 30s and pare them down to a few milliseconds. Daily loading tasks that ease over the 'day in a day' threshold can be brought down to a few minutes.

This is all usually before you even get as far as remodelling the database design, just making sure the queries aren't doing frankly stupid things.

By investing in serious DBAs and serious dedicated SQL Developers to work alongside the client side developers you can make quite astonishing improvements to concurrency and throughput.

As I said - not a new concept at all, but then lessons do seem to have to be learned over, and over and over again in the IT industry.

The last J2EE project I worked on, we did some serious performance tuning: more than 90% of the tuning was optimizing SQL queries. So when on Oracle, learn how to use and understand stats packs and execution plans.

As usual, it's a case of using the KISS principle. If your database schema looks too complicated, it probably is and you will end up doing a stupid amount of joins that will make your app grind to a halt. This tends to happen when you try to map a complex object model to a relational database.

One thing on Oracle that should make alarm bells ring is if someone tells you they used materialised views to increase performance. This should only be used with caution and you should understand the trade-off: if a normal view performs badly when you read from it, a materialised view will perform badly when you write to the tables the view is built upon. You can't have it both ways.

Happy tuning!

SQL Server optimisation is my hobby.

On a recent job, I turned a system from one DWH query every 45 seconds, to 280 in 45 seconds. I wrote a TOPANDBOTTOM(1000) clause ordered by multiple columns, in six lines of transact SQL. Didn't even need the CLR, though you can do some outrageous things with custom aggregates.

The thing is most people don't believe it's possible until they see it.

I regularly achieve greater than 50 times performance improvement. Once I reached a factor 400, but this was on an nhibernate system, and all that object relational technology was designed to be slow as pigs just to sell more computers.

I've failed interviews because there are guys out there who genuinely believe it's more sensible to scale out middle tier servers and cache everything, than to scale out an array of read only replicated SQL servers, and load balance them.

Noone believes it when they see a stored procedure go down from 10 seconds, to 0.2 mS.

I find the people who can do this kind of thing have a set based mindset, often at the expense of something else.

I'm so glad Linq has arrived, because it means we can do the same thing in the business logic too.

SOA or not ! Good comments except this is an age old problem, how to access information. I agree with RichardB except my experience of DBAs ( unfortunately ) is not very good, sorry. SQL ( structured query language NOT necessary using a relational database ) mostly refers to relational access ( it is a little more ), now you define your database relations instead programmers in programs and if you do it right it works, in theory. But right means that you really know how your 1000+ tables and 10000+ relations are used AND on which channel, disk array, cluster node, RAID array, read or write or both they they are used and what kind of crazy queries the end users create. One query wrong way and the whole system stalls, maybe sorting something over and over again or maybe accessing disk in random order instead of sequentially for a billion line report and seeks killing the system or maybe the account numbers (keys) were in city order (%90 access going to one poor disk in 100+ array ) or whatever.. All old problems but now the responsibility of DBA who should (IMHO) actually be designing the business logical access, do it right and let the infrastructure people take care of physical layout. As RichardB said, an old problem and DBA should be a (good) mediator between application/end user groups and infrastructure. And please, don't let end users create their own reports, the idea was a disaster already in 70's and it is not any better today, you never know what they do but what is sure, they will stop any online system, beware especially corporate statisticians running wild!