M9K vs E25K
Having had personal experience of both M9Ks and E25Ks, the latter is heavily hamstrung by relatively slow interconnects and what was a very old fashioned I/O system based on PCI (the fastest I/O buses on an E25K were 66Mhz 64bit which means that it's not really up to running a dual port 4Gbps FC card (especially when you know that FC is full duplex). Nor, of course, is it up to 10Gbps Ethernet standard. There ae ways round it - more cards in I/O system boards, network aggregation and SAN I/O balancing, but it looks increasing old hat and expensive compared to a modern x86 server, let alone direct competitors.
In comparison, the M9K has got PCIe and is a vastly more capable I/O box and the interconnects are a lot better (and, the RAS features are better too).
SMT is something of a mixed blessing. The various very similar versions of this (Hypertheading on x86, the equivalent on Itanium and Power) are fine where you have lots of largely independent threads working on large data sets with enough processor cache misses to make it worthwhile. However, in the case of a large, unified database like Oracle, there are large numbers of points where individual threads need exclusive control of a resource. The path time through those sections has to be kept as short as possible on SMPs or the DB will waste much of he CPU away on spin locks. If you enable SMT on a DB like this, then the threads on the same core will contend when they are both busy and if you believe the figures, you get, at best, 20% more throughput with SMT which means that two threads despatched on the same core will each only run at about 60% of the speed (on average). Path lengths go up, contention increases, cpu is wasted. In general, single DB benchmarks work better with fewer, faster cores. That's not to say that more cores can't be used - if your application uses a lots of procedural code running within the DB then this isn't necessarily true, but for a benchmark like this, it probably is.
So, like all these things, it's the application that matters. As it is, the M9K is a very decent bit of tin and much, much better than anything Sun had to offer in that space.