For both performance and capacity reasons, companies running large transaction processing systems, whether they are tickled directly by Web users or just end users working behind the company firewall, sometimes have to partition their production relational databases. This practice, called sharding, is a pain in the neck. …
High performance MySQL
Why MySQL first? If you want to build a high-performance system, you shouldn't be using MySQL anyways. It's slow and not even near feature complete.
Probably because there are lots of mysql shops struggling with performance, so they need to shard first. Mind you they may not be the best people to try an extract money from for DB tools.
In some cases MySQL is actually very fast because it is not feature complete.
Well - MySQL is a very popular database, and there are allot of people with scaling issues around it - so it was a very good fit for us.
I think this is the idea of sequioa before
.. before they became tungsten or something. will probably try it out in the near future.. anyway good luck boys!
Thanks. Would love to see you as a customer http://www.theregister.co.uk/Design/graphics/icons/comment/happy_32.png
So can you stack these?
If it looks like a single dB, can you take one of your shards and replace it with another of these boxes and a handful of shard behind it?.
Rinse, repeat, recurse...
Yes you can... http://www.theregister.co.uk/Design/graphics/icons/comment/facepalm_32.png
Only for giant databases
50GB is not a large database. On a reasonably specified server, the whole database could be in memory. (In the by now antique Oracle 8 product if the Shared Global Area (SGA) was bigger than the database size on disk then the only disk I/Os while running were database writes - a tiny proportion of the normal total. The speedup was massive.)
With current server performance there is probably no benefit from sharding for most databases below 1TB (and probably a number of 10TB databases would be better kept unsharded).
Throw cheap hardware resources (CPU,RAM,SSD) at database performance before using expensive human resources
You beat me to it
I was thinking the same thing: 50GB? Large? Our smallest customer has a 1TB database. Some are getting on for 300TB. We use a proprietary database though as it's a highly specialized application and a normal generalized database wouldn't cut the mustard.
Still... damn! Why didn't I think of doing this?! :(
I agree - 50GB is not a big database. It depends on the database, machine size and what you do with the database.
However - I disagree that only 1TB databases will enjoy sharding. If you check out our benchmark at http://www.scalebase.com/resources/performance/ you'll see that even a 100GB database got major performance improvements with sharding.