Google is considering new ARM-based designs for its home-brewed servers, according to Bloomberg. The financial newswire's report is sketchy, offering the observation that Google “ is considering designing its own server processors using technology from ARM Holdings Plc” together with a quote from Google spokesdroid Liz Markman …
That'll be ex-Intel CTO Pat Gelsinger defending Intel then.
Interesting that Google announced that they're looking at a particular technology; didn't think they did that type of thing unless they were already set on using it. No idea what this means, but curious nevertheless.
Re: Pat Gelsinger
Actually, I think the article is pretty close to spot on in it's conjecture. Look at history: when Google announces it is looking at a particular technology it is because it is seeking to generate buzz, interest and - ultimately - a strong community following. These technologies are rarely truly "new", but they are unique twists on extant stuff that need a far broader base of people to get involved in order to make things really happen.
Here I am thinking of Android, Glass, Wave, Google +, many of the "in depth features" of Google Maps (such as Moon, building interiors, etc), Hangouts, Ventures, and Google TV.
When Google "just does it" then the technology in question is usually truly pioneering (GFS, Analytics, Google custom switches, Google AI load balancing and so forth.) These are core to their business operations and their competitive advantage and they don't like to share info until they are well on the way towards the next version.
To me, this says that Google has already made the decision to pursue ARM server processors and are in the first stages of a prolonged run up to the creation of an ecosystem. This is the hype phase. Next will come a lot of hint dropping and "idea mining." (Where they wait for the wider tech ecosystem to come up with great use cases and means of working together.)
After that will come a limited trial and prototyping phase (invite only, natch). This will be followed by "invite your friends" which will effectively be a free-for-all to anyone interested, but with enough legal CYA-this-is-still-a-beta to be get-out-of-jail-free, also "we can steal your ideas and not compensate you".
There will then be a bit of a lull as the community collapses inwards a bit, sheds the less interested and becomes more about a hard core of evangelists. Meanwhile, Google will be beavering away on internal stuff (probably on version 2 of implementation internally by this point) and readying a public release product. Google will quickly snap up the top 30% of the hard core of evangelists to staff the new consumer-facing department and launch the product.
They'll start slow, but subsidize heavily. The idea is to gain market share. The 70% of evangelists who weren't bought up will be cranking out applications like mad in the vain hope of getting hired at Google, driving a new explosion of interest and the foundations of a real ecosystem. The hardware will slowly creep up in price to just above 1.5x cost. It will stabilize here.
Google, meanwhile, will have had it's minions building an entire ecosystem delivery apparatus around the software layer and slowly start to "own" the ARM server market. Other competitors will have entered by this point, but they won't make a real volume dent.
By this point, we're 7-10 years out and Google will be on the 4th or 5th iteration of its hardware internally and will already have a sub-department filled with PhDs devoted to driving innovation in all areas of ARM server design that will give it a massive leg up on competitors. Those designs will find their way into the "consumer" product a few years later creating an ecosystem cycle much like the GFS/Hadoop one.
Overall, Google will have accomplished several goals:
1) reduced the cost of it's own infrastructure.
2) kickstart a "physicalisation" movement (likely complete with the ability to "vMotion" workloads on physical hardware the same way we can virtual ones. HA, Fault Tolerance, etc will all rear their heads.)
3) remove the "ownership" of the server market from Microsoft, Unix/Non-Google Linux and VMware.
4) made server applications something that can be - and increasingly will be - delivered through it's "Play" store (necessary for Google to retain control while allowing other manufacturers to bear the R&D burden of actually making the kit for punters.)
5) created a movement towards the TRUE integration of on premises server apps with cloud-based processing and application provisioning (only really possible if you create an ecosystem with zero legacy).
6) create an "enterprise" component to it's IT offering that suddenly makes ChromeOS and Android relevant in a way that they can't be today. (True app-store delivered integration of everything, backed by central Google Cloud processing, Google-based authentication, etc.)
Ultimately, it would lead to a completely new way of thinking about resource usage, data storage, identity, security and so forth because this new ecosystem simply wouldn't have the x86 legacy. It would start small, face fierce resistance and struggle for every inch of ground...but Google's gotten good at managing the human side of these sorts of things. They know how to create evangelists, create an ecosystem and offload the risk (and most of the heavy lifting) onto the community, swooping in only once the model has proven itself to cream the profits.
Worst case scenario, none of the above materializes, but Intel absolutely pisses itself in terror and starts busting a nut to prevent the ARMpocalypse. Google then gets cheaper servers and the ability to bully Intel for the next two decades.
Either way, Google wins.
Re: Pat Gelsinger
What it means is summed up nicely in the last paragraph of the article and why the news came down through Bloomberg. Google is about to ask Intel for something big.
It won't be a straight cost issue either. It'll be some feature Google wants but Intel isn't providing. Changing your vendor of a core business product is a big, big deal in any industry and when the change is between two major powerhouses in an industry the news doesn't just appear. It'll have been industry knowledge for a long time and ARM specialists would have been being headhunted for long while.
This is nothing more than old school leveraged negotiation, it's been going on since Lloyd's invented the practice. The idea is that analysts will start talking about all the implications with the goal being to make major Intel shareholders lean on management to keep Google as a customer. Keeping a customer may or may not be worth it, only Intel can decide that. Sometimes your biggest customers simply aren't worth dealing with... We'll know in a month or so what Google is wanting.
As a rule, anytime a financial news outlet is breaking news about a significant shift in anything that isn't a financial product or service then what you're dealing with is marketing, plain and simple.
Re: Pat Gelsinger
Well, Apple owning them on phones in America must smart a little, and now Apple are about to wrap themselves in the flag with a US based factory and maybe a fab in the future, if what we hear is true.
Maybe Google can't afford to let that go unanswered and want to fab their own silicon in the US too.
What they would then probably want from Intel would be a licence to make their own chips to customise them precisely as they wish, and also to be sure they are NSA jiggery-pokery free.
Re: Pat Gelsinger
"Maybe Google can't afford to let that go unanswered and want to fab their own silicon in the US too."
Mercantilist preening as driver for entering a lower level of the application stack doesn't sound like a formula for big success.
Re: Pat Gelsinger
America is 350M(ish) people. There are 7 Billion of us out there and Google has the bulk of the market share everywhere else.
With the rapidly growing wealth gap in the US (that has seen virtually every single dollar of the "recovery" from the latest recession go towards the super-rich with almost none of it making its way down the stack) the US is of rapidly decreasing relevance. To put it simply: an increasingly impoverished and indebted populace with a rapidly shrinking middle class has ever less disposable income to spend.
Apple can have the USA. Nobody cares. They can clean up there for the next decade and every year they will continue to show less growth than the last as the US population is simply no longer capable of sustaining Wall Street's appetites for corporate growth. A decade from now they will be a middle-tier country no different from so many others.
Google prefers to play the world at large. To continue to bolster growth they need to chase booms and abandon busts. When an economy looks to be slowing down or closing it's borders to outsiders Google moves on and invests elsewhere. They are perfectly prepared to jumpstart the economy of an entire nation in order to skim a % off the top as profits.
Real US purchasing power stagnated years ago. It has been in decline for the past several. Worse, the hoi polloi are paying off their debts and this is further restricting consumer activity. At the same time issues like the NSA being douchenozzles, laughably terrible US government (literally, laughing stock of the world) and continued horrible foreign policy mean that pepole of other nations dislike the US more today than they have in quite some time.
Wrapping yourself in an American flag will win you points at home, but kill you overseas. If your domestic market is collapsing, that doesn't seem like the brightest move.
Oh well, Google, Apple, Amazon, Microsoft...screw 'em all, I say. I've no loyalty to any of 'em.
Re: Pat Gelsinger
do you think it's time for vmware to port vSphere from x86 to ARM ? or Microsoft Hyper-V to arm ?
Or doesn't it make sense at this point ?
Re: Pat Gelsinger
Vmware had better get started, pronto! Xen already have an ARM hypervisor - http://www.datacenterdynamics.com/focus/archive/2013/07/open-source-xen-hypervisor-now-supports-arm-servers
Google appear to have been testing the waters with an Emulated ARM platform - http://code.google.com/p/armware/
Re: Pat Gelsinger
VMware has an ARM hypervisor. Remember the "VMware for phones" thing a while back? Pleas etell me you don't think they abandoned it.
Also: Windows RT is Windows 8 recompiled for ARM. The Kernel and major services are largely intact. That means Hyper-V for ARM exists.
The issue isn't that the existence of ARM hyperviors, it's the existence of applications for ARM that would drive ARM server OS adoption (and hence hypervisors.) Further compounding the issue is the ARM SoCs aren't cross-compatible. So you can't fully pass back a Samsung SoC into a VM and then vMotion that across to a Qualcomm-based server.
Right now, only a select few ARM providers are really heading into the server market. Here nVidia and AMD are the really big players to watch for. Microsoft and VMware are both working closely with those vendors (and others), but the market needs to exist before they start bringing resources to bear on it. If Microsoft bets on AMD (for example) and the market chooses nVidia, then Microsoft is left holding the bag.
Similarly, is Microsoft devotes a whole tonne of resources into compiling the hypervisor for all possible SoCs (along with getting drivers made for Windows RT for all possible SoCs...egads!) then they could be pissing a great deal of time and effort into a market that just won't get born.
They have the tech right now, but everyone is in a holding pattern until someone takes a huge risk and dumps a big pile of resources into ARM server. This is what startups are for...and there are several in stealth mode. The 2014/2015 ARM server startup bloodbath will in fact determine the outcome of the x86/ARM relationship for the next two deacdes.
...unless Google (or any other Big Name) decide to become kingmakers. The investment pattern I described above would easily change the market. It would also trigger all out war between all major parties. That is something everyone wants to avoid right now, they are still licking their wounds from the mobile wars.
Google, however, managed to offload their risk onto Samsung for the bulk of those battles and is in pretty good shape. They could tank the ARM wars pretty well and have a crazy amount of IP (from Moto) that is relevant.
Still, the "gentleman's agreement" is to let the VCs take the risks, let the market decide (I.E. see who gets to revenue first and most consistently) then buy up the winners and short the losers. The Bad Blood Microsoft is stirring up with both VMware and Google, however, make me think that might not happen this time.
Google and VMware have reason to make common cause against Microsoft in an ARM arena and just enough of hte right egos have been bruised by Microsoft's asnine behavior that I remain convinced they may well decide to play kingmaker out of pure spite.
The real blocker here is that VMware has strong ties to Intel and Google doesn't have any real strong ties to any ARM makers except Samsung, whom they would rather not crown (given the animosity over Samsung's attempts to control Android.)
All the above leaves me wondering if Google might not do something amazingly hostile like buy AMD. It would be foolish, but not too foolish...but none of my sources point in this direction (yet). That said, Google (like Apple) has an enormous pile of money that they can't repatriate. Buying an ARM vendor (or at least a foundry) that is located outside the US is not remotely out of the question, if it came wiht the right people.
At this point, there are enough reasons to do this thing and enough reasons not to do this thing that it is really going to come down to ego. For all that a lot of Register readers believe that decisions like this are made on a purely rational, purely by-the-numbers basis my experience says otherwise. The more I learn how the tech industry works, the more I realize that "who snubbed who, when and why" is a huge factor in how this all plays out.
Our entire industry has been thrown into revolution more than once because A pissed of B and B decided they were going to show A what's what.
As you pointed out by noticing the hypervisor issue - and the article author by noting the ecosystem - this potential ARM server play is not simply a matter of numbers, nor is it simply a matter of poking Intel in the eye. The considerations are many and vast...and it seriously could all come down to "I'll show X who's boss."
Makes ya think, eh?
Re: Pat Gelsinger
Well, if Google wanted to poke Intel AND Apple in the eye comprehensively, ARM Holdings market cap' is only 22.8Bn.
Good luck getting that one past the regulators and lobbying though.
Many thanks for the enlightenment too, Trevor.
I think you're overthinking this a bit.
Google builds commodity servers. They used to buy actual servers, but in successive generations they've customized more and more, but they are still easily recognizable as PC servers, they just remove the bits they don't need and buy the bits they do at larger and larger commodity volumes to drive down the price.
CPUs are just another commodity to Google. They need cycles, they don't care whether those are x86 or ARM cycles, so long as energy input for a certain amount of computation output is minimized. In fact, the actual cost of the CPU itself is probably down the list a bit since it is dwarfed by the power bill for running 24x7x365 for two or three years.
They don't really care about building an ARM server ecosystem, because it won't be useful to them. They just need an ARM server CPU, they can ride on the ARM mobile ecosystem that Apple and Google have already built up. A compiler is a compiler, Linux was ported to ARM long ago, etc.
So what ARM CPU to buy? Well, AMD is working on one, there are a few others, or they can design their own. I'm actually skeptical of them designing their own because if they go so far as to hire their own CPU design team it makes more sense to use something other than ARM, because ARM licensees cannot implement their own extensions to the ARM ISA. I suspect there may be certain things Google servers do often enough that creating highly customized SIMD instructions to accelerate it could be worth it, but they do not have that option if they go with ARM.
Re: Pat Gelsinger
Isn't one of the points about going to ARM is that its cheap to make lots of lower-power CPUs. Hypervisors are not required because the "waste" from a less then flat-out CPU is minimal.
Of course, the OS should be doing what the hypervisor does, but that's another story...
DougS: the cost of porting everything they have to ARM is huge. If they go ARM it also places them in a position of either
A) being beholden to a given manufacturer without much of an ecosystem (Intel might be a single manufacturer, but there is a hell of a lot of stuff that bolts on to x86 systems!) or
B) designing everything themselves, including potentially having to write drivers and what-have-you for any widgets they want to add on that aren't part of the design as it exists now.
Cycles are not "just cycles." Attached to the CPU are networking components, disk components, sensors and so forth. It takes an ecosystem to ensure you have real choice (and thus the ability to grind your suppliers.)
The fact that you have a Linux kernel on ARM doesn't mean you have Google's Linux kernel on ARM. Nor their apps. Nor support for any apps they use that aren't in house. Also: lower power CPUs means less likelihood of virtualisation which means having to figure out how to move workloads like virtualisation but from a metal install.
A move to ARM carries with it changes that could indeed be smoothed over by having an ecosystem built up around your CPU of choice. I think it's more than "just a new source of cycles."
Re: Pat Gelsinger
In my experience:
- ARM has an advantage over Intel in terms of low power consumption, while
- Intel has an advantage over ARM in terms of performance.
What's different is that ARM licensees are easier to deal with than Intel. IMO Intel needs to shed some of their bureaucratic nonsense and their draconian contracts that make people afraid to do business with them.
Why do you think the cost of porting to ARM is huge? Shouldn't most of their code be in C or C++? Do you really believe it is hard to port C code to another architecture in 2013? Especially when both are running on the same OS? Or do you think that ARM Linux isn't well tested? Google only has about a billion such devices in the field today.
20 years ago I ported a suite of X11-based medical imaging code running on SunOS to Solaris, OSF/1 (on one of the first Alphas sold, well before ANYONE had ever thought about writing 64 bit clean code) and HP-UX. That was far far more difficult than what Google will have to do porting from x64 Linux to ARM64 Linux. SunOS let you write such terrible code, other systems (especially HP-UX) were a lot more picky and a lot of bad code had to be fixed along the way. Plus what I had to work with was written by people who were researchers who happened to write code, not professional programmers like Google has writing their applications. Google will have it way easier than I did.
You vastly overestimate the difficulty of their task. There's a good chance everything will compile first try. Yes, at that point there will be tons and tons and tons of testing required, because it doesn't work just because it compiles, runs and appears to work, but the actual number of problems found and changes to the code required will be few and far between, and you can easily identify the places you need to look carefully at (i.e. stuff like bitfields and complicated structures, where weird alignment rules and compiler bugs may lurk)
I don't think I'm overestimating anything. I think it's more than "just making the code work." Workloads as they exist - including entire segments of the application design - were tuned for x86-based systems. Bottlenecks will appear and they'll have to be identified, worked around and redesigns done.
In addition, hardware support for ARM isn't great. Each SoC is it's own little world, and a lot of what's out there (widget-wise) doesn't have ARM Linux drivers. Want to use Inifiniband to lash together your servers? You're probably going to have to get a custom driver. Want to use a Micron PCI-E flash device? Same deal.
I don't think it's impossible, or even uneconomic. I do, however, think that in the long run Google would do far better to create an ecosystem around the SoC it chooses so that others can take some of that R&D off their shoulders.
I'm not doom mongering by any means, merely being pragmatic. The bigger the ecosystem, the larger the component choice (without having to go to the mats with every vendor) and the wider the code base. Additionally, you bring up an entire generation of devs ho are designed to think around the limitations and design quirks of your chosen SoC variant instead of x86.
Really, there's no rational reason for Google to "simply switch CPU vendors" without going for the ecosystem play. For that matter, there is nothing syaing that their existing workloads even make sense given the differences in CPU to I/O balance between ARM and x86, so on and so forth.
Testing, planning, more testing, redesign, lots more testing...it's part and parcel of shifting architectures, especially when you measure your computer in acres.
bit and pieces
On the 5/08/2013 the register published :
"Ex-Cray supercomputer interconnect guru Scott leaves Nvidia for Google"
FUD until you see a contract with Nvidia
It's all about price: specifically Google asking for a cut on Intel prices to stash in one of their offshore tax-free pots.. a cut they expect the rest of us to pay for. If they were serious they talk about paying Nvidia to build an ARM version of the Xeon Phi.
Re: FUD until you see a contract with Nvidia
No. Big supplier changes like this are almost never a price issue, it doesn't matter what industry you're in. This is how product customization, distribution, preferred roadmap access and other strategic deals are negotiated.
When you get into big business, in any industry, cost is near the bottom of the list in negotiations with major vendors. There are far more valuable things to be discussed than price. Price is something that can be moved fairly easily. Getting your vendor to make a custom product for you for example, and not share it with anyone else, now that is a a big deal that is far, far more valuable than a discount.
We won't know what the ask is for a few weeks, but it won't be pricing.
Re: FUD until you see a contract with Nvidia
> We won't know what the ask is for a few weeks, but it won't be pricing.
Perhaps they just want an RDRAND that hasn't been backdoored.
x86 enjoys an enormous lead in terms of drivers
Hardware is becoming obsolete all the time and new hardware replacing it.
It's as easy to write new drivers for one platform as another, as Linux on ARM proves.
It is extra funny since Intel stopped producing drivers for their QX3 microscope several version of Windows ago.
Their own prompt yields '0 Results' - I ended up getting a new device.
Re: x86 enjoys an enormous lead in terms of drivers
That means practically nothing when it comes to servers. If your designing your own hardware for millions of nodes you're not plugging anything new into it ever.
"May" means nothing
Like any company doing a lot of research and looking towards the future, of course Google is looking into what it would take to shift from Intel to ARM. No doubt there is even an evaluation of switching to MIPs or even SPARC. Why not? There are potentially huge wins to be made by switching architecture.
A switch to ARM would allow Google to go beyond just multi-core. With a good interconnect system they could essentially build supercomputers in chips,or building blocks for those. That's the difference with ARM... pay the license fee and you can design the chips YOU need. With Intel you have to take the chips they decide to make.
Considering there are at least ten ARM systems running Linux for every x86 system running Linux, it is not at all suprising that Linux is getting rather mature on ARM.
Re: "May" means nothing
Remember that if you have got the money, Intel will adapt its stuff for you, which is, and I cite, "Intel's answer to easily configurable, custom-built ARM-compatible chips".
The story is that Google might develop its own server architecture, based on ARM. That doesn't mean buying anything off the shelf, it means licensing technology from ARM and doing a whole load of custom development on top of that.
Somewhere in that custom development they might very well want to adopt IP from other architectures such as MIPS, Sparc, PowerPC, or even x86. There's no either/or between the architectures, but the prospect of a base that's neither x86 nor ARM seems remote, and of those ARM is the one whose business model welcomes licensees doing their own custom development.
I have said for..
years ARM is the way to go, Intel will become old hat. The WINTEL dictatorship will come to a grinding halt soon. Even Intels newest chips arent a patch on ARM's offerings. Whatever Intel try to do ARM laways beats its it by 2 or 3 times. x86 is so 10 years ago
Re: I have said for..
Agreed. That's why I bought ARM shares back when they were less than £1. So far, so good.
Re: I have said for..
So you think licensing technology and building your own processors is going to offset ready to use, fully developed products with support and stability behind them? I'm sure as hell glad you aren't managing my money.
Products based on ARM technologies have a role, absolutely, but they're two fundamentally different things. You're saying assembly line cars are going to be displaced because kit cars are available. The technical aspects of one product over another, for any products in any industry, are only one part of the equation. Same with cost, it's only one part of the equation that has five or six other equally important variables.
Re: I have said for..
I am so glad you arent running a company that needs to look into the future. I'm happy you are happy sticking to the present and past
Re: I have said for..
Little man, I made my first two payoffs based on my assessments of the future and my assessments of the future are still highly valued by my company and our VC group. I was successfully predicting the future before you could type and will continue to do so successfully for many, many years.
You can't manage a company or a strategy to the future if you aren't in control of the present and had ownership of the past as well. You're talking about things you don't understand, making emotional judgements about things that don't care about your love-in. You're combining the absolute worst traits in business and setting yourself up to fail. Fail hard. You should stop.
ARM means Linux, and Linux means everyone wins.
Google is already a Linux shop so they can port their code to any architecture, of course. But a megasurge in ARM servers would really give Linux a big boost. There are still way too many organizations running Microsoft's overpriced server software.
A year or three? You are thinking of some other company, maybe. Google moves faster than that (when it wants to).
Bad news Mr VMWare
Google doesn't need you
Just like they don't need Dell or HP to make their HW.
Hmm I thought The Chocoloate factory servers runs on Linux natively and not VMware?
not needed but desperate to get mentioned here. Glad I could help,
Wasn't it a few decades ago that people like IBM said that PC based servers wouldn't cut it. Then PC servers took over the world.
Now people are saying ARM can't cut it as server CPUs.
Yes, I know there are still mainframes out there. The number of them is tiny compared to x86 servers. I don't think ARM will wipe out Intel in the data centre, either. But it'll certainly give Intel a run for their money.
Variety & competition are good, so bring it on.
Who will design it?
Even if Google wants its own ARM server chips, it does not need to design them themselves. They can get AMD, Marvell, Samsung, or a whole host of other experienced chip companies to do so for them. Or they can, like Apple did, buy a chip design company to do it in-house. But I don't really think so. Google does not have Apple's paranoid need to have full control over all aspects of the hardware, and for the same reason I don't see Google getting into designing processors for phones or tablets.
Re: Who will design it?
When I did chip design 20 odd years ago we could tell what every part of the transistors were doing every nanosecond if we wanted to. Google know what they want their processor(s) to do, ARM will provide them with the kit to design their processor, whichever fab they choose to use will give them the most accurate data necessary to simulate the chip to death before it comes near to finalising the design.
I think Google will probably design it themselves and absorb any skills they might be lacking and I would really like to see what they come up with - but I dont think they'll let me,
Intel's 4 largest customers
Curiosity: who are Intel's 4 largest customers? Bloomberg don't say. US Gov and Mil, IBM and HP? Not necessarily... Any breakdown by agency for US Gov? Any sources?
Google To Buy AMD
or purchase a major stake ?
Google wants Intel to compete with ARM for its foundry work
Google has tradeoff studies in hand that show what optimized chips mean for its servers. Google suspects that a chip with one general purpose CPU and about sixteen vector processing GPUs and about a billion bytes of flash memory/GPU is near optimum for heavy graphics an AI computations. Details of how ARM or Intel chip foundries function will influence the as/built chips, but make no mistake, Google will get the chip performance desires one way or another.
Expect 2.5 times more giant warehouses with each doing 6 times more computing within five years. Five years means 8x more stuff on a chip; now that Intel cares about power, maybe half the power per chip. Intel need not become uncompetitive for server business. Its up to Intel.
I thought Google were going to use the Power chip for all there new stuff
- Bugger the jetpack, where's my 21st-century Psion?
- Something for the Weekend, Sir? Why can’t I walk past Maplin without buying stuff I don’t need?
- Review 'Mommy got me an UltraVibe Pleasure 2000 for Xmas!' South Park: Stick of Truth
- The land of Milk and Sammy: Free music app touted by Samsung
- Privacy warriors lob sueball at Facebook buyout of WhatsApp