not a mention on instruction set
Which is a shame, because POWER architecture used to be very inefficient in the area of synchronization and memory model. Did they fix that?
Big iron sales are still generating $6bn to $7bn a year for IBM - which is enough to justify designing its own Power processors and building its own wafer baker. At the Hot Chips conference at Stanford University on Monday, some of the chief architects behind the Power8 electronics were on hand to show off the feeds and speeds …
Which is a shame, because POWER architecture used to be very inefficient in the area of synchronization and memory model. Did they fix that?
After a bit of research there seem to be a big step forward: IBM added hardware support for transactional memory! At least this is what I gather after cursory read of https://www.power.org/documentation/power-isa-version-2-07/ , chapter 5. Not bad.
"...After a bit of research there seem to be a big step forward: IBM added hardware support for transactional memory! ..."
Tthe last in line. SPARC T5 was the first commodity cpu with transactional memory. The old Sun ROCK cpu also had transactional memory, but it didnt make it to delivery. But all that old ROCK research has found its way into current SPARC cpus. Intel Haswell has transactional memory too. But it is nice that IBM tries to catch up on other cpus.
The external Centaur chip opens up other possibilities as well. Such as the Hybrid Memory Cube.
At the High end like Power8 its a smart move. The basic concept has been proposed before (e.g. RAMBUS) but the additional costs of additional components has always been a problem at the low end.
With much of the mass market shifting down to low power portable devices the DIMM memory market is going to suffer so I think there is room for innovation at the top. IBM are obviously not committing to anything other than DDR3 but they have kept their options open.
where's the STFU ?
You mention '40ns, 9.6GB/s' for the core-to-memory-controller link, and then say that a system with eight controller chips can have 230GB/s CPU-to-memory bandwidth; which of those figures is right?
Nice to see that at least someone is still upping the non-x86 chip roadmap. I still hope someday we will finally break away from the Intel stranglehold, hopefully.
>someday we will finally break away from the Intel stranglehold, hopefully.
There is always SPARC. Ok ok just kidding but Intel's stranglehold is already disappearing. Their nemesis is ARM. Even the datacenter may not be safe for Intel soon.
Is it just me, or does that first picture of the chip look exactly like the picture of the Intel chip that looked exactly like the picture of the Apple chip? Is there just a Getty library of chip images?
IBM is still pushing an aspect of technology that SHOULD be competitive with Intel's primary processor technology. Reducing the need to access regular-memory SHOULD benefit single-cycle-execution technology especially. It is good that IBM has matched Ivy-Bridge 22nm process, but IBM probably won't exceed Intel in the future with process-size.
That means that IBM needs to open the door to higher single-thread operation (GHz or IE efficiency). This might be a cool thing to track over the next 1-2 years.
"That means that IBM needs to open the door to higher single-thread operation (GHz or IE efficiency). This might be a cool thing to track over the next 1-2 years."
The biggest barrier to better single thread performance for everybody is memory latency. The memory architecture for Power8 is unbelievably complex and has tremendous bandwidth, and all that complexity is a good effort to overcome the fact that DRAM latency is way too slow in comparison to the core speed. But IBM and everyone else needs faster high capacity memory technology.
You can judge how hard it is to do. Despite the vast improvements in silicon manufacturing technology IBM and everyone else still has only about 64k of L1 cache running at core speed. We can put billions of transistors on a chip, but we can get only a few tens of thousands of those to operate as memory running at 4GHz. Unbelievable after all this time. IBM did pretty well with the Cell processor (256k core speed RAM next to each SPE), but we seem to have gone backwards since then.
>IBM did pretty well with the Cell processor (256k core speed RAM next to each SPE), but we seem to have gone backwards since then.
How did they do well? They lost money on the Cell and discontinued it. It was such a dog for general computing (multitasking, tasks not conducive to streaming) that it is what finally drove Apple to Intel (you can actually hear the PS3 groan for the forever it takes to bring up the xbar while in a game). They needed all that L1 memory because of the massive latency to move data around in their convoluted design. What was funny was Sony actually thought they wouldn't need a GPU at first on the PS3 with the Cell. Once multicore general commodity CPUs hit the market in force Cell with its need for special programming and optimization was doomed.
Also note that not only does L1 cache eat up chip real estate as you increase it but as is common with cache increasing it often offers diminishing returns.
"....Unbelievable after all this time. IBM did pretty well with the Cell processor (256k core speed RAM next to each SPE), but we seem to have gone backwards since then...."
Well, you need thirteen (13) CELL cpus @ 3.2GHz, to match one single SPARC T2+ @ 1.6GHz in string pattern matching benchmarks. The CELL benchmark was done with heavy optimization in asm, loop unrolling etc. The SPARC developer did a school book implementation in C, and still was 13x faster. You need in total, 13 x 3.2 GHz = 42 GHz in aggregate power to match one 1.6GHz SPARC. The SPARC T2 uses it's GHz over 20x more efficient than a CELL cpu. Do you consider this fact, as CELL being good? Maybe that was the reason that CELL is discontinued and there are no CELL2 cpu. If CELL really was successfull, it would have lived. But no.
Well it is good that the POWER8 is up to 2.5x faster than the POWER7, in theory. The SPARC T5 is up to 2.4x faster than the POWER7 in real life benchmarks today:
When will the POWER8 be released? Next year? Same year as Oracle release the SPARC T6? The SPARC T6 servers will be twice as fast as the SPARC T5 servers. Oracle has just announced the M6 cpu on Hot Chips. And Fujitsu has also announced a new SPARC64 cpu there too.
Also, Intel's latest Xeon E7 cpus are faster than POWER7 in some benchmarks. So the POWER8 can replace up to 2.5 Intel cpus. (IBM claims the the POWER8 can replace up to seven Intel cpus, IBM marketing is a bit over ambitious)
It would be really funny if IBM focus on performance talking about POWER8, "it is so 2000ish to talk about performance. No one cares about performance anymore", said IBM. When Oracle released the SPARC T5 and crushed all other cpus, IBM said this:
“[Performance] was a frozen-in-time discussion,” Parris said in an interview Wednesday. “It was like 2002–not at all in tune with the market today. Companies today, Parris argued, have different priorities than the raw speed of chips. They are much more concerned about issues like “availability”–resistance to break-downs–and security and cost-effective utilization of servers than the kinds of performance numbers Ellison throws out."
So, when IBM could not compete anymore, IBM said no one is interested in performance. But when POWER8 finally catches up on SPARC T5, will IBM start to talk about performance again? That would be hilarious. :)
I am doubting the POWER6 bandwidth numbers presented here. Back in the days, the POWER6 had 240GB/sec band width or so, according to IBM. I dug a bit, and found out that IBM had added all bandwidth in the cpu, L1 bandwidht + L2 bandwidth, etc. But you can not do like that. If there is a bottleneck of 1GB/sec in the cpu, the bandwidth will never be greater than 1GB/sec. This is a famous theorem in discrete math, called the "MAX FLOW = MIN CUT". It says, the greatest achievable bandwidth is the bottleneck. So, IBM marketing was a bit over ambitious. :)
It reminds me of when IBM claimed one Mainframe with 24 cpus can replace 1.500 of the x86 servers. It turned out that all x86 server idle at 1-2% and the Mainframe was 100% loaded! Also, one of the Mainframe cpus are way slower than a decent x86, so in no way it can replace a single high end x86 server doing work. IBM marketing was a bit over ambitious in this case too. :)
Funny how POWER is getting more similar to SPARC for every iteration. I remember when IBM mocked the new and radical SPARC Niagara cpu. Back in those days, all cpus had 1-2 cores. SPARC Niagara _started_ at 8 cores and went upwards. IBM said CPUs are better if they have one or two strong cores, than many lower clocked cores. So, POWER6 ran at 5GHz with 2 cores. And there was talk about future POWER cpus running even higher, 6-7GHz or more. Just like Intel Pentium4 prescott.
But today the POWER8 and all the other cpus (Intel, etc), all have many lower clocked cores and we dont see POWER cpus consisting of one single strong core anymore. That race is over. No more pushing one core to sky high Hz. So, it seems that IBM finally understood the way forward is many lower clocked cores. The same path Sun treaded 10 years earlier. If you push a cpu to very high Hz, it will use too much power, so the only way to keep power usage at bay, is to lower Hz. But in that case, you must compensate with more cores. So POWER8 has 12 lower clocked cores, each running 8 threads. Quite similar to SPARC T5 (16 cores each running 8 threads). There are no 1 core POWER8 running 2 threads at 8-10GHz, that was a dead end.
So says one of the last few people buying or selling SPARC chips. Business has learned what happens when it gets itself locked into Larry's pricing. The only hope for SPARC would be if Oracle sells it off like it should have, otherwise its like Itanium, a dead man walking. Larry will only throw good money after bad to soothe his ego for so long. The guy really loves making money after all.
Well asdf, you clearly haven't done your homework. Locked-in to Larry's pricing? Have you compared systems pricing between SPARC and Power lately? Clearly not!
A SPARC T5-8 is 3-4x lower priced than an equivalent performing Power 770 or Power780 system. Even a high end configured SPARC M5-32 system is lower priced than the slower high-end Power 780 configuration! Maybe that’s why IBM's Power Revenue has been hemorrhaging these last 5 quarters?
And even software licensing is half the cost on SPARC vs Power whether running Oracle or IBM SW or even SAP for that matter! But as you say, Larry is a business man and a successful one at that being one of top ten richest people in the world. Would he invest in a dead end business? Doubtful.. Would he invest in a business where he can be successful, become #1 and make a lot of money? Absolutely! And whats making him (and Oracle) money (hint: Oracle has remained profitable and continues to show revenue gains since the Sun acquisition)? Selling engineered systems based on best of breed technologies like SPARC and Solaris and beating IBM and HP. Have you seen IBM's last 5 quarters of double-digit revenue declines on its Power Business? Even IBM's business as a whole has been down for last 5 quarters! How about HP's last 7-quarters of double-digit Itanium declines? Sure, when Oracle first took over Sun 3+ years ago, SPARC was in a decline. But not today. Oracle has invested $3+BN in R&D these last 3 years on SPARC with a solid roadmap (which is public by the way) to the future and it shows-Oracle is delivering. Fujitsu is also heavily investing in SPARC with its own SPARCX technologies. Why? Because there is huge opportunities for growth here and with Intel abandoning Itanium, and even showing signs of defocusing on high end Xeon due to ARM beating the crap out of them, who's left to address the commercial enterprise market? Oracle.
"And even software licensing is half the cost on SPARC vs Power"
Software is licensed per core and even the old Power7 has more than twice the performance per core of the new T5. So even if Oracle tries to level the field with a core factor(very similar to handicap in golf which gives the lesser players a chance to "win") IBM comes cheaper since software is by far the dominating cost of a system.
With faster cores being more useful for single threaded work(batch) and superior virtualization which utilizes the expensive software even more, IBM wins by a clear margin.
Power also runs AIX which is a real enterprise OS. You can't even upgrade a Solaris 10 system to Solaris 11. A complete reinstall is required. Welcome to the land of Windows.
Old Power7 has more than twice the performance per core of the new T5? Time to redo your math! On which benchmark? Which comparison? And why are there no Power7+ benchmarks to compare to on database benchmarks like TPC-C or TPC-H? And don't get caught in the trap of comparing a fully configured SPARC T5-8 versus a very low configured Power7 system and assume that per core/performance will scale linearly to higher end configs! If it did, Im sure IBM would have published the result! Unless of course you're buying an 8-core system.
On TPC-C OLTP benchmark, SPARC T5-8 achieved 8,552,523 tpmC w/128-cores. The closest throughput Power7 system achieved 10,366,254 tpmC w/192-cores (using 3x more CPUs!). SPARC T5 has a 24% per core advantage! SPARC T5 is also 2.4x faster per chip and 2.5x better $/tpmC than fastest (and closest comparable)Power7 result.
On TPC-H @ 3TB, SPARC T5-4 achieved 409,721 QphH with 64 cores while the fastest Power7 with 32-cores achieved 164,747 QphH. SPARC T5 is 24% faster/core and also delivers 2.1x more performance using half the # of CPUs than the Power780. Both TPC-C and TPC-H have price/performance metric which shows SPARC T5 being far superior to *any* Power6/Power7 results. And this price/perf metric includes SW & HW costs! And with no Power7+ results available, we can only assume its slower...
And as for AIX vs Solaris, Solaris is years ahead of AIX and now that IBM is so focused on delivering Linux on Power, AIX is certainly falling behind faster. Why after all would IBM release Power systems that don't support AIX?
And finally, do you really believe that Power8 will have superior performance/core than Power7 or Power7+? with 50% more cores @ same 4GHz, its unlikely that perf/core will improve, especially on larger systems. I guess we shall see sometime late next year!
"...Software is licensed per core and even the old Power7 has more than twice the performance per core of the new T5..."
Yes, I do not rule out the possibility that POWER7 might be faster per core, on _some_ benchmarks, yes. However, on database workloads the SPARC T5 is faster both cpu wise and core wise. In fact, the SPARC T5 is the worlds fastest cpu today. For instance, in SPECint2006 and SPECfloat raw cpu power the SPARC T5 is faster, just check the benchmarks. The SPARC T5 is also 2.4x faster than POWER7 per cpu in real life TPC-C benchmarks, not only faster in theory.
The T5 has 16 cores, and the POWER7 has 8 cores. So if the T5 had the same oomph per core, T5 cpu would be 2x faster. But the T5 is 2.4x faster, which means the T5 cores are way faster than the POWER7. So, there might be some benchmarks where POWER7 is faster core wise (I dont know which, though). And there are benchmarks where T5 is faster core wise, including databases.
The SPARC T5 beats the POWER7, core for core in database workloads. So you dont need to license as many cores as you need on POWER7. Plus you have twice the number of cores in T5, so if you need even more performance you can get it, leaving POWER7 far behind. So the SPARC T5 hardware is way cheaper than POWER7, and the SPARC T5 is also cheaper in license costs.
What is best? A quad cpu with 4 strong cores, or a cpu with 64 cores, each core slightly slower than the quad core cpu? The 64 cpu will be faster, because it has more cores.
The reason Oracle comes out fairly well on cost in the TPC-C is that they price their software support on Oracles largest systems at $2,300 a year. I could of course ask Oracle-employee Phil how many enterprise customers run this fantasy "support" model but I don't bother since both he, I and everyone else knows it is zero. The list price for Oracle database with a few options is close to $100,000 plus a hefty yearly percentage for support. And that is per *core*.
So any customer who knows how to operate a pocket calculator ie. not a typical Oracle customer, will find that IRL software totally dwarfs the cost for hardware by a factor of 15 to 20. So the logical conclusion is to aim for most bang for the buck on your DB licenses and that is:
Fast cores -> not sparc.
High utilization of cores - > absolutely not archaic Oracle Ldoms virt
And isn't it kind of embarrassing for Oracle that a 5 year old IBM server without the benefit of flash and SSD is on par in TPC-C with the newest and biggest Oracle server with twice the cores and 8 times the number of threads and 80 Tb of flash?. An Oracle system which is so new that it isn't even available for customers yet so no one else has the chance to do a compare.
"Why after all would IBM release Power systems that don't support AIX?"
Wow, after Oracle letting most of the Sun tech die a silent death it is good to hear that the FUD department is still expanding.
What is correct here is that for some of the Power models you have one Linux-only version which is cheaper(competes with x86) and one version which runs a choice of AIX/i /linux(competition fast fading away).
"Solaris is years ahead of AIX".
OK, you are never out in the field so I will tell you how this works. If you are at Solaris 10 and want to go to Solaris 11 you cannnot upgrade. So one has to find or buy a new server, install your new OS and application there and then you have to replicate what is years of configuration from the old production server. In the end you migrate the data and do a switch. This takes weeks, is risky and has extreme cost. You can call Solaris "the most advanced operating system in the universe"(true, Sun has stated this!) all you want but it is still amateur night.
With AIX it is a 10 minute reboot on an upgraded rootvg and a few minutes of relinking Oracle and you have upgraded your major version of AIX. That is enterprise for you.
And before you start about ZFS begin by telling us why Oracle does not use that tech in any of the aforementioned benchmarks.
There are lot of inconsistiencies and plain wrong "facts" in your post. You should do some reading to catch up. Or are you up to date, and deliberately write false things?
"...Fast cores -> not sparc...." We have shown official benchmarks, and the SPARC T5 has 25% faster cores than POWER7. And, the SPARC T5 has twice the number of cores. So, yes, the SPARC T5 has faster cores, and is also the faster cpu. Up to 2.4x faster than POWER7 on TPC-C benchmarks.
"...Wow, after Oracle letting most of the Sun tech die a silent death..." What are you talking about? Oracle is capitalizing heavily on both the SPARC CMT cpus (T5) and the SPARC M6 cpus. And Oracle bets heavily on Solaris, too. And Java. etc.
"....OK, you are never out in the field so I will tell you how this works. If you are at Solaris 10 and want to go to Solaris 11 you cannnot upgrade. So one has to find or buy a new server, install your new OS and application there and then you have to replicate what is years of configuration from the old production server. In the end you migrate the data and do a switch. This takes weeks, is risky and has extreme cost. You can call Solaris "the most advanced operating system in the universe"(true, Sun has stated this!) all you want but it is still amateur night...."
You are totally off here. It is obvious you are never out on the fields. But let me tell you how it works. If you want to migrate a Solaris 10 server to a Solaris 11 server, you utilize containers (you know, the tech that IBM AIX copied and named it WPAR, just as IBM AIX copied DTrace and named it ProbeVue) and zip the entire Solaris 10 server, and dump it into an container on a Solaris 11 server. And then you are done. You can also do that with Solaris 9 and Solaris 8 servers, and dump them onto an Solaris 11 server, via containers.
You have some reading to do. Or, you have just not understood what Solaris Containers are good for.
"If you want to migrate a Solaris 10 server to a Solaris 11 server, you utilize containers "
What? Are you kidding? Making a flash archive out of your Solaris 8 server and run that in a branded container on Solaris 11 is nowhere near an OS upgrade. You still run Solaris 8 but now with some trickery I wouldn't touch with a 10 foot pole in a production environment.
Running Solaris 10 in a container on Solaris 11 is just some half assed attempt to escape the huge task of doing a real upgrade. A task which is comparable to migrate to another HW/OS-platform. Which many do by the way.
"You have some reading to do. Or, you have just not understood what Solaris Containers are good for."
"...If single-thread performance is the most important thing for a piece of work, a core or set of cores will step down the threading automagically and run it with fewer processor threads...."
This sounds exactly like SPARC critical threads introduced in the old SPARC T4, where an important thread will take over a single core. So the core will only run that single thread. That is the reason the T4 has strong threads, and also extreme throughput if needed. You choose during run time.
And also, POWER8 cpus are finally getting transactional memory, first introduced to world in the SPARC T5 cpu. But Sun's old ROCK cpu also had transactional memory years ago - but it did not make it to delivery. So the ROCK research shows up in those SPARC Txx cpus instead. And Intel Haswell also has transactional memory now. So, SPARC T5 has transactional memory, later Intel Haswell also got transactional memory, and now IBM is trying to catch up with transactional memory too. Better late, than never, though.
As I wrote:
"....Funny how POWER is getting more similar to SPARC for every iteration...."
It would be cheaper for IBM if they just licensed SPARC cpus, so IBM did not have to catch up all the time.
Clearly the Oracle marketing team had an extra shot of espresso today. Statements about SPARC outperforming Power7/7+, copying SPARC features, SPARC is significantly less expensive than Power, oh yal and the old reliable "Oracle software licensing cost is 2X more on Power, etc, etc is laughable. These statements are misleading at best.
You compare your Sept '13 T5 with 128 cores to a 2008 entry for a Power6 (N-2 technology) with 64 cores and only yield a 40% better number.
Exa* is not best of breed, rather it is cheap white boxes cobbled together with expensive Oracle RAC clustering, limited SAN support, internal flash, etc. If it was best of breed it would use Cisco, HP or IBM x86 technology with VMware and flash technology from Fusion, Violin or IBM Flashsystems (ie TMS)....oh, by the way then it would also use IBM DB2 and pureScale instead of Oracle Database and RAC.
The net net is that benchmarks are great and fun to watch. All vendors should do them....nice to see Snoracle finally do some as they have been largely absent for years. However, it really comes down to the efficiency of the server to deliver that performance to real life workloads. I just read how the SPARC M6 uses critical threads to essentially hog all of the processor resources. That isn't so good for running multiple VM's on a server is it? With Power servers I can deliver a QoS to each VM as well as dynamically change how much cpu, memory and I/O to increase performance, throughput or both based on the businesses need.
Oh yeah, lastly to the old reliable "Power is twice as expensive as x86 or SPARC" comment. There is more to the story than just the Oracle licensing factor with Power being 1.0 and most other platforms being at 0.5. Because of the efficiency (and performance) of a Power server I can usually run a workload with 1/10th the number of resources required by x86 or SPARC. So, even with Oracle manipulating the licensing factors that still means a x86 or SPARC server would require 5X more licenses.
"....Clearly the Oracle marketing team had an extra shot of espresso today. Statements about SPARC outperforming Power7/7+, copying SPARC features, SPARC is significantly less expensive than Power, oh yal and the old reliable "Oracle software licensing cost is 2X more on Power, etc, etc is laughable. These statements are misleading at best...."
Oh, this sounds like good old IBM FUD. Lots of unsubstantiated negative claims about a competitor, with no proofs - this is actually the very definition of FUD. Spreading negative rumours which have no bearing in reality. "Yes, I heard this horrible rumour, but I can not prove it is true" - because it is not true.
So, if you do claim that SPARC does not outperform POWER7, or SPARC is not cheaper than POWER - can you show us some hard facts that prove your stand point? Show us some performance benchmarks. And show us some pricing comparisons. Come on, I dare you! :)
If you can not show any hard proofs (which you can not because we are telling the truth, just check the benhcmarks we posted) then it is the IBM camp that is a bit over ambitious again. :)
I remember one of the diligent IBM supporters here, who has not been active as of late. I said that the x86 is faster than POWER6 in LINPACK benchmarks, in response he said "no, it isnt. The POWER6 is faster in LINPACK because the POWER6 has faster cores" or something weird. I cant recall the logic he used because it was wrong, I cant think like that. I pointed out that you need two POWER6 cpus to match one Intel Xeon in LINPACK, but he said "well, the POWER6 has faster cores, so it is faster in LINPACK". Sure, the Xeon had six cores, and scored twice as high and if you count core wise the POWER6 was faster, true. But that does not make the POWER6 the faster cpu, does it? I dont understand the logic IBMers use.
I also said "in this benchmark the SPARC T2 has higher throughput than the POWER6 system" to which he replied "it doesnt matter, the POWER6 has lower latency which is the important thing!". Later I showed another benchmark where the SPARC T2 had lower latency, to which he replied "it does not matter, because the POWER6 has higher throughput, which is the important thing!". I really dont understand the logic IBMers use.
And now here comes this "PowerMan@thinksis" claiming lot of weird stuff, without providing any benchmarks nor pricing examples. Out out the blue he says we are all wrong, without pointing out the errors. "Trust me on this, but you are wrong, I can not tell you where you are wrong, but I know you are wrong. I am a doctor, trust me". I really dont understand the logic IBMers use. Or, lack of logic that IBMers use. Or just marketing aggressiveness the IBMers display.
BTW, have you heard that AIX is going to be killed off? It will supposedly happen sometime in the future when cheap Intel x86 cpus are catching up on expensive POWER cpus. Because IBM only does high margin business, and if the x86 are cheaper than POWER with same performance, then IBM will shut down POWER and AIX too. AIX runs only on POWER, and without POWER servers AIX can do nothing. So the POWER future looks grim. Yes, it is true, I really heard it. I am talking out of the blue? I am spreading false rumours that I just made up, or can I prove this? Well, read it here yourself and see I am not spreading false made up rumours, like the IBM camp does. No, instead I back up my claims with hard undeniable facts, straight from IBM.
"...Asked whether IBM's eventual goal is to replace AIX with Linux, Mills responded, "It's fairly obvious we're fine with that idea...It's the logical successor." A replacement "won't happen overnight," Mills said, but years of experience designing operating systems at IBM and other companies means developers know just where Linux needs to go. "The road map is clear. It's an eight-lane highway."