Posts by Jesper Frimann

Prices on powervm

"Has IBM shipped it for free, or has IBM been charging people for it since 2001?"

Brrr.. that was a good question. I cannot say for sure. But I have looked at some of the old configurator output I got from IBM Sales for solutions I've done in the past.

And on all the POWER4 solutions p690'es there is no virtualization charge, and for the power5 solutions I have on the HD there isn't either.

Now for the power6 there are charges after POVERVM became a products, but on blades you get a standard edition of POVERVM for free.

As far as I know and can see from the SUN and IBM homepages, then the features that are a part of the hypervisor on the T5XXX series are also free on a POWER servers. You can do logical partitioning virtual networks. So the free part on power servers are at least equal to the free part on T5XX servers. Now wpars, which is akin to containers are also included for free in the OS.

But when you want shared pool lpar, Virtual IO servers, partition migration wpar dynamic movement of wpars etc. then on most of the servers you will have to put money on the table.

It looks to me, from my understanding of things, that IBM have made sure that what you get for free on T5XXX'es you also get for free on power boxes. And then the fancy stuff you have to pay for, not really surprising. And I have no problem paying for Shared Pool Logical Partitioning, it pays for itself many times over.

"SUN predicted that virtualization was going to hurt their hardware sales, but they also thought it would drive their hardware sales, in the low end. I personally think virtualization hurt Sun's sales more than they expected."

I must admit I have trouble seeing where SUN will make the big bucks on their current strategy.

"If RCK shows up in 2009, great, if not, the 4xT2+ will be enough for my applications, running LDom's and Containers."

'good enough' is the key word here. You have to remember that the other hardware vendors also move forward. If Rock comes out in late 2009, early 2010 it will be facing POVER7. Which looks a bit like a monster.

// jesper

Well

deadmonst:

Well we are using it, everywhere we can. I mean we are replacing p690's with power 560/ 550'es where just savings in maintenance is paying for the damn migration project. And yes the POWERVM is a new marketing name but the basic stuff have been on servers since 2001, and the main functionality of the current powervm have been around since 2004.

Ron Skoog:

Well, today VMS doesn't even support vpars on Integrity servers afair, which is kind of a bugger. And watch out a Mainframe guy might hear you and start telling about how they virtualized back in the 60'es before even I was born.

David Halko:

The style hypervisor that is shipped with Niagara based boxes is a logical partitioning technology hence you divide the machine into smaller bits, allocating CPU threads to different domains. That type of hypervisor have been shipping on POWER servers since 2001. Now the hypervisor that started shipping with the POWER5 boxes in.. 2004 I think it was, is what is using what Big Fat Blue is calling shared pool logical partitioning (SPLPAR), here you just allocate shares of CPU ressources, and when a partition isn't using them others can. The same functionality that you would implement using containers on a solaris system. In my book virtualization on power is years in front of the, just as clear number 2, which is SUN and HP really have to get things going with regards to virtualization.

I don't really think people realize that by embracing virtualization like people that build POWER and also Solaris solutions have done, is actually hurting IBM and SUN hardware sales numbers.

Gartner and IDC have called this effect the virtualization effect, and it's also hurting the x86 sales bigtime.

And it really isn't that strange, cause you are pumping up the utilization of the physical hardware when you are using virtualization. So if you had a workload that ran on a 24 core system using partitions and with a 20% average utilization. If you now can get a system with cores of twice the speed and you can use virtualization to raise the average utilization to 60% by letting the partitions that don't use 'their' CPU ressources, let other partitions use it.

Then you are actually reducing your needs for compute power from 24/(2x3)=4cores, which means that you can jump to a p550, if we use POWER as an example.. Now if you had just made a 1-1 migration (lpar->lpar) to a new box with cores that were twice as fast you would have needed 12 cores. Then you would most likely have bought a power 570. And the price of a 4 core power550 is something completely different to a 12 core power 570.

// Jesper

RE:They are not full function cores

Well virtualization is not good for business. Going from partitioning your server, like on a M class machine to a shared pool of processor resources, like on POWER systems would mean that the average utilization of boxes sold would jump quite significantly. Hence you would sell fewer, and possible smaller servers to replace old ones.

Sure using containers helps, but that glove doesn't fit all IMHO.

// Jesper

@HANS on SAP

You are right Hans. SAP is crap seen from an IT perspective. But if things start to run MUCH slower when you upgrade, to a box that SUN have advised you to use. And the box you are upgrading from is 5 years old.

And it is not only SAP, to quote the comments that a former fellow student, which I have worked with when I was a consultant, had on Niagara based boxes. "It is crap that they sold me such a box and it is clearly not made for this kind of workloads" He tried to get a RAC cluster to perform on a handfull of T5120. He is one of the most Hardcore Solaris and Sparc fans I know, and also a very skilled DBA.

Now the truth behind these stories is that not all workloads are well suited for the Niagara based boxes.

And migrating a workload that have been written/tuned/setup for running with few threads, on a few fast processors is not always that easy. Not without doing a rewrite, reconfiguration and new tuning.

Not to mention locking problems, memory use for all those threads etc.

Niagara is *great++* for what it was designed for, but IMHO SUN is trying to sell it in where it doesn't fit.

// Jesper

Well...

isn't the problem that it's gonna be a little to little and much to late.

I mean if SUN get rock based systems out in 2H, about the same time as Tukwila, and then AFAIK POWER7 will be just around the corner, so IBM will most likely already have benchmark results out.

And that will mean a hard time for a system that was originally slated to compete with power5+ and Montecito based systems.

And still they will need single threaded performance, this is the main issue that I hear/encounter as being a problem, with the current Niagara based servers.

Just a while ago I heard the lead SAP architect where I work, complain that it was the second time that he had been involved in migrating a customer from an oldish SUN server to a new niagara and that the throughput had sucked. The new system actually ran slower than the old one.

So yes it is good for SUN that if they get Rock out of the door in 2H, buuuutt.. still it's late to the game.

// jesper

ROCK is coming...

at a time when the other UNIX vendors will be shipping Tukwila and POWER7. But it is hard to know. I mean the messages given from SUN was that it would arrive in late 2006, early 2007, then it became 2008, then mid 2008, then it was 2009 and now it's late 2009

I wonder how a chip originally scheduled for 2007 will fare against the competition. I mean there is big difference between competing against POWER5+ and last iteration of POWER6 or first generation of POWER7.

// Jesper

Well, it is not only a economical problem

Well IMHO their problems are more fundamental than many think, I personally find that SUN is not looking after their customer base. Which is very loyal, some to the point of fanatics. It seems to me that they have raised prices on the boxes that they normally sell to their loyal customer base. And this will surely generate more revenue, for each box they sell, but it will also scare some away.

IMHO SUN's loyal base is the lowend SPARC servers that run Solaris. There are A LOT of those out there, and you used to be able to get a decent performing box, to a price that was lower than it would cost you to get one from HP or IBM.

Sure the HP/IBM box would be faster. But the difference in price in the favour of SUN and the fact that the IBM/HP server wasn't that much faster, meant that there really wasn't any reasons to migrate to something else.

That have IMHO changed.

Now if you today have a v215 or V125 or even an oldish v4X0 you want to replace with another SPARC server, then You basically have 2 different choices Buy a Niagara based server/Blade or buy a M3000.

The same kind of money that gave you a v215 will now give you a a 4core 1.2GHz T5120

And the money that gave you a v125 will now get you a 4 core 1.0Ghz T1000.

This is not an ideal fit for many SUN customers, as you with the Niagara based servers might end up having worse single threaded performance than on your old server. And for many workloads that will mean worse throughput.

That leaves the M3000 as the other option.

Now the M3000 is 15K USD and it's not that fast. SUN just released a SAP2Tier benchmark of 4130 SAPS. That is a good deal less than old submissions from HP and IBM.

The rx2660 using Montecito and the p505Q using Power5+ both does around 5500 SAPS.

Now the rx2660 would be a good deal cheaper, if you choose to use lower clocked Montvales rather than the highest clocked ones, and would most likely cost around 10K and a p505Q is 5.5KUSD.

This means that low end servers from HP and IBM now aren't just faster but also cheaper than their SUN counterparts.

This is bad for SUN, and we haven't even started to talk about Linux taking marked share yet.

And what about their server product lines ?

There is a install base out there with SUN Enterprise and SUN Fire servers who have been waiting for a replacement, in the form of Rock based servers. And if you've ever talked to SUN sales people then many told their customers to jump over APL servers and go directly to Rock based servers.

And well now it looks like SUN will try to make these people move to MX000 systems:

http://www.sun.com/promotions/campaigns/index.jsp?pid=ssepromo&intcmp=2218

And I would imagine that this would mean that HP and IBM will unleash the sales dogs of war, on the customers that have oldish SUN equipment, and either take business away or force SUN to discount more than it would have liked to do.

So what will happen to SUN ? Hopefully the will survive in some form as the IT industry will be a poorer business without them. But they need to change things. I agree with the with Cris Mellor (the writer of the article) that I don't think that neither HP or IBM would pick up SUN. But a alliance between Fujitsu and IBM might, where Fujitsu took over the Hardware and IBM some of the software division, but I don't know how much money IBM would actually pay for getting their hands on Java.

// Jesper

Re:Re:RE:Re:hRE:Efficiency

Hmm.. Bill you don't get it.

Let me try to explain, it is not SPLPARS versus containers, AIX has it's own version of containers

called WPARS which is basically a clever extension of the workload manager. Furthermore WPARS have the ability to be able to move between machines, much like the vmotion of vmware.

So there is nothing wrong with the 'container/jail/WPAR/IVM' concept, of running semi isolated containers inside an OS image, but one size doesn't fit all.

For example I would never run a SAP test environment inside a container/wpar inside the same OS image that also held the production environment. It would be madness.. not that I haven't seen it done.

Now the overhead, where I btw in my 4 years of working with SPLPARS have never seen anything near 10% (4-5% I've seen), and it is really not an issue. You have to compare it to LPARS/domains/v/nPars where you have the same problem as you have with smaller rack stack and pack'em machines. If you aren't using the CPU's capacity inside a partition, then nobody else can use it and it goes to waste. On a machine running SPLPAR you can reuse the CPU resources if they aren't used by the virtual machine they are allocated to.

On a machine that runs SPLPAR you can easily reach 60-70% average utilization of the physical machine, compared to perhaps 15-25% on a machine that runs partitioned. And the 4-5% you are paying in overhead you get back ten fold.

Let me give you an example from the real world, I'm currently in the process of analysing what we need to buy to replace 8xp690'es with around 192 power4+ cores. And some of those machines will still do 30K+ TPMC per core. So they are still with todays eyes fast. But I figure I can squeeze those 8 refrigerators into 2x16/3x12/4x8 ways power 570'es, and still have room left for other consolidations.

And if you really really don't want to loose those few percent then run your virtual machine in LPAR mode. Then you have the same limitations as domains and vpars.

But I've never experienced a 2+ socket power server that is power5 or newer that have run out of

cpu resources. They will always run out of memory first, or IO first, cause people size those resources to low.

I would say, depending on your application, if you for example are running SAP with an Oracle Database, and test, education, prod and development on the same machine, then having around 32-48 GB of RAM per CPU (or core) is not a bad idea on eg a power 570.

Now is the POWERVM hypervisor a single point of failure ? Yes sure it is, but it is a lot lot lot more stable than a OS, it's not a program or oslike thingie that executes under the virtual machines, I would more compare it to a read only shared library. Sure an error in the library could bring down a virtual machine, but to bring down the whole machine is very unlikely.

Also I would have no problem having virtual machines running on the same physical machine that were on all four different zones of a firewall, besides having to neuter the HMC network :)

// Jesper

RE:Re:hRE:Efficiency

Well... overhead can be calculated in many ways.

Well if you take SUN domains, HP's v/nPars and IBM's LPARS is that what they basically do is to divide the server into smaller pieces.

This is good if you do not want to pay for 8 CPU's , that your server has, running Oracle when you only need 4 CPU's, hence you can make a partition that only runs on 4 and on the rest of the CPU's you can then install another partition on, the remaining 4 CPU's

Now the bad thing is that all the CPU time that isn't used is still wasted, so if you have an average utilization of 20% of both your 4 CPU's partitions. Then you actually have an overhead of ... 80%.

In comes SPLPAR, shared pool LPAR, which you can run on the power5(+) and power6 hypervisor. Here you truly virtualize your CPU's and CPU resources is seen as a pool from where you can get what you are entitled to. which have been set up by fairly simple rules. This means that I can keep on making virtual machines on the server and tap into that 80% idle CPU time from the example above, simply by the fact that if no virtual machine is using the CPU resources then another virtual machine can. Now this is virtualization, not like LPARS,v/nPar's or domains.

// Jesper

Dear Bill

Dear Bill.

The functionality you talk about there have been available in AIX since AIX 4.3 and is called Alternate disk install. Now let me point out to you that AIX 4.3 is more than 10 years old.

The Original feature let you clone the OS of a running system to a new set of disks and you could then patch do a full os upgrade whatever on the OS image on these disks, and then boot from them and then boot back to the original disks if you wished so. You could also use this feature to clone systems, and that could be useful for systems where you hadn't any network on them yet. Have had to do that quite a few times.

Now the current version of alternate disk install is quite a bit more advanced, and doesn't need disks to clone on.

I must admit I'm getting a bit tired of hearing smart fancy Solaris marketing names like Jumpstart, liveupdate for things that have been on AIX for years. Sure Network Installation Manager (NIM) isn't as fancy a name as 'jumpstart' and liveupdate sounds much much better than alternate disk install. But that is AIX, no catchy names and fluff. It's just a really really good UNIX, and under it you have a hypervisor that's perhaps only triumphed by zSeries, and under that again you have the damn best hardware you can get.

It's like the A10 warthog: ugly, effective and almost impossible to bring down.

It's not that Solaris is a bad OS, I'd just think that AIX is a little better. But if you factor the hypervisor and the POWER hardware into the equation. Then it's really not a race.

// Jesper

First of all

the newest AIX version is AIX 6.1 not 5.3.

ZFS looks nice but many of the features I see pointed out have been in AIX for years Snapshots compressed filesystems or have been added in AIX 6.1 cryped filesystems etc.

Not that ZFS doesn't seem more advanced than JFS2 cause it does.

As for patch management then AIX have always been good at that. I mean I accidental patched some key libs that a running Oracle database used many years ago, now that is bad. But after just doing a rollback of the patches I could actually close down what should be closed down and repatch the system.

I also remember being able to backup a filesystem where there weren't any disks below it, straight from filecache. Saved a whole development departments from some red ears.

But still it's the whole package that counts, and much of the work done on 5.3 and 6.1 seems to have gone into virtualization, or rather to integrate AIX with the virtualization done by POWERVM.

I mean they must have rewritten a lot of code, just the accounting part must have been a nightmare. Imagine you have 100 virtual machines on a machine using 200 virtual processors that run ontop of 32 actual cores, and all those virtual machines all run at more than 100% utilization if you use standard sar commands. Brrr...

// jesper

Solaris and AIX

Well I think that both Solaris and AIX are the 'good guys' when it comes to following POSIX, You might argue that AIX typically uses the POSIX values by default and you sometimes have to specify it with Solaris, not like HPUX *SIGH*.

I do think that Solaris and AIX are a bit different. Solaris being the favourite of the app. developers, and AIX being the favourite of the Sysadmins. I had to debug some IO performance problems on a Solaris 9 box some years ago. And I was amazed at how primitive the tools were compared the the ones I was used to from AIX 5.3.

And when you talk to AIX and Solaris fanatics you usually notice that the AIX fanatics are almost all Sysadmins, but the Solaris fanatics have a very high percentage of DB admins and developers among them.

The reason that I think that 'AIX servers' are the best, is the combination of hardware, virtualization layer (POWERVM) and the OS.

Which means that IMHO there really aren't any clear weaknesses when you compare an AIX solution to one based upon Solaris or HPUX.

With regards to Oracle licensing, If you move from SPARC to POWER6, then you'll end up with your licensing savings paying for your new servers, if you do it right. It takes a little design work to harvest the savings but the POWER hardware, POWERVM and AIX will give you the tools to do it.

Sure you have to fight off Oracle sales people (or some other software company) who will try to trick you, cause they don't like loosing licensing fees.

I once had an Oracle sales guy who wanted to have licensing fee for x3 the number of cores in the system, because we used uncapped micropartitions. But luckily the Oracle licensing are pretty clear.

And remember there is NO EXTRA VALUE in paying for 12 licenses compared to paying for 5 if they are of the same type. But having better hardware does give you value.

And just to close the Oracle bit. Do not.. do not.. not not not run Oracle on the T5X40 servers, they make amazing (but far to expensive) webservers, but license cost will ruin you.

// Jesper

A few errors in this article.

First I wouldn't compare the chipset of the x3950 to the power 570 and power 560. The power servers do not have a external chipset in the same sense as the x3950 do.

Try to have a look at this redbook that deals with the big brother of the power 570 the power 595:

http://www.redbooks.ibm.com/abstracts/redp4440.html?Open

Furthermore the new 4.2GHz power 570 with a maximum of 32 Cores is a 4 CEC (building block) system, hence each CEC have 2 CPU boards which each has 2 'sockets', giving it a total of 8 cores per building block.

The only problem I see with these new systems (power 560 and power 570) is that they have can house less memory than the power 550 and the 16 core power 570 per core.

I do think that the power 560 have been overdue for some time. But it is a nice upgrade from the p560Q which could house 16 1.8GHz power5+ cores and only 128GB of RAM.

// Jesper