back to article Facebook friends bash servers, storage, and racks into bits

When you are the company the size of Facebook, you get to dictate the terms of your server acquisitions to your vendors, and a year and a half ago, the social network did a remarkable and unselfish thing and opened up the hardware specs for its first data center and the servers that were built to run in them as the Open Compute …

COMMENTS

This topic is closed for new posts.
  1. Nate Amsden

    intel vs amd ?

    How about Intel vs Intel? or AMD vs AMD? They change sockets pretty regularly. I haven't paid too close attention to CPU sockets since the Socket 7 days when things really were cross compatible. I suppose these CPU engineers aren't thinking forward enough to put enough pins or whatever to take into account future connection points. It would be nice if we could have a common socket to be sure.

    But even if we do get it, how many will actually use it ? I think surveys have shown that at least enterprise customers almost never upgrade the CPUs on their servers. I remember a spat I had with HP many years ago on the DL380G5, I had purchased it with dual socket dual core for Oracle Enterprise Edition - which was licensed per-core. The dual core procs had the highest clock speeds. The DL380G5 was advertised as a quad core capable system (we had other 380G5s that were quad core but just not running Oracle). Then we went to Oracle Standard Edition which is licensed per socket. So I wanted to move to Quad core.

    I expected to just be able to drop a quad core CPU into the system but it didn't work. I even had HP support do it themselves I didn't want to fry the system. They couldn't get it working, after about an hour of escalations they finally figured out the specific motherboard I had was not quad core compatible (even though the socket was the same). Had to switch out the MB - so we did (the savings of Oracle easily made up for these costs). Slight drop in performance (2xdual core to 1xquad core that was clocked at a lower frequency), but massive cut in license fees.

    I checked a couple years later and HP had updated their spec sheets saying some models are not quad core capable.

    "NOTE: Future upgradeable to Xeon® Series 5400/5300 Quad-Core processors (Please contact your

    regional sales support to determine quad core upgradeability. Some of the earlier shipping models may

    not support this upgrade)"

    what a pain! but in the end it was worth it.

    Myself I don't see these facebook things taking off in anything other than service provider types - perhaps rackspace and the like. Rest of the world will be 19" for some time to come, having an extra 3.5" disk just isn't adequate justification for breaking compatibility - especially when storage companies have come out with innovative designs that put disks deep within the chassis and still maintain hot swap ability(3PAR has been doing it for about a decade other more recent designs are much more dense though), it's good enough.

  2. Sil

    Dreamy?

    That hardware interoperability and flexibility will improve over time seems quite inevitable and that specific hardware will be developed to answer Datacenter specific needs - as opposed to the way smaller needs of most corporations, seems logical.

    A full decoupling of processor, memory and storage seems impossible though, first because this will make it extremely difficult to debug and diagnose hardware problems with exponential number of interconnect possibilities, second because systems evolve over time and not only to please and protect chip manufacturers, making it very difficult to do plug and play in a fully open manner over time without artificially limiting the system and last but not least because of ever smaller transistor processes I would assume that chips can and will integrate more functions (cpu, gpu, memory controllers, interconnects)

    To want a system where x processors, y rams and z storage, not to even think of interconnects or networking are plug and play together and over many générations seems to me similar to the dreams of a fully layered and/or tiered and/or expandable/reconfigurable software system. It's not for today nor for tomorrow nor the day after. It gives a good sense of direction and domain definition, it's heavily researched but to my knowledge were are very far from the panacea and progressing at a frigid pace.

    Also human cost of changing and reconfiguring hardware is high and error prone, it will be interesting to see whether it is truly more economical than loadbalancing virtualized payloads over additional ready-to-use standard racks from hardware manufacturers.

    One can only thank Facebook and the Open Compute Project to keep us informed and to freely share the results of their experiments, no doubt their hard learned lessons will be of interest to many and the pressure toward more openness only they can apply to manufacturers wil benefit many of us.

    1. Rams
      Angel

      Re: Dreamy?

      IMHO it'll definitely work. Somehow, it's just history repeating itself. We moved from mainframes computer, to commodity hardware (then companies started to move back to some professional level hardware). And why did they do that? Simply because it was way to expensive to change the mainframe, or loosing to much time+money when it would break down and you need some super experts to find the problem.

      Here, assuming it works, you can just remove some layer of complexity: we need to upgrade the hardware, let's buy some new blades and its associated NAS storage. You just replace that by: we need to upgrade the hardware, let's find the bottleneck and change only that piece. We'll save time on migration and everything else.

      Things which seem impossible will just be common in a few years. Guys inventing HDD of 1 metre radius probably didn't think at that time that eventually we would use 512GB SSD for nothing.

  3. Peter Gathercole Silver badge

    Hey, he's invented the Mainframe...

    ...again.

    Back in the day, you had a cabinet or two for the processor(s), at least one for the memory (especially if it were core), another for each disk string controller, and then more for the disks themselves, and then additional cabinets for front-end processors, tape drives and any other ancillary devices.

    It was perfectly possible to add and remove memory, disk controllers and strings of disk without having to replace the computer as a whole. Or you could replace the processors, and leave the rest of the system untouched.

    I remember on weekend in 1985 when I went home on a Friday night, after using NUMACs crusty old IBM 370/168 which was collapsing under the strain, and came back on Monday morning to the same system with an Amdahl 5860 that to the user was identical, just a lot faster.

    Professor Harry Whitfield (director of the computing laboratory at Newcastle University at the time) wrote the following in his annual report for the year:

    "The installation of the Amdahl 5860 in late September 1985 and its introduction into service in early October must be regarded as the major event of the year. The whole process went so smoothly (and unannounced) that users 'merely' noticed that the system had suddenly become much more responsive and five times faster."

    I admit the analogy is not perfect, but there are serious similarities.

  4. koolholio
    FAIL

    ARM stands for...

    "A right mess"

    Features:

    Slower

    Higher error rate

    Fancy and Convenient, yet so many 'bugs'

    F*C*book is what I'm thinking --- error bits included!

  5. Magnus_Pym

    Buying power

    I think the point is that if someone like Facebook uses it it will get made rather than just proposed.

  6. btrower

    Future Future Proof

    I applaud facebook's efforts.

    As a 'computer guy' starting on systems with 128 bytes (sic) many, many years ago, I have seen the many standards come and go.

    It should be easily possible for components to interconnect over their entire lifespan. The article is correct about increased up-front costs, but I would prefer to spend $300 on a power supply that continues to always supply clean, compatible power through modular connections for years rather than constantly turning over cheaper power supplies.

    Two gigantic impediments stand in the way of 'future future' proofing stuff:

    1) Bandwith considerations including capacity, latency and addressability is constantly a limiting factor in these things. A network object, whatever it may be, should not be constrained by design. Things on the network should negotiate linkages to get the 'best' one supported at both ends and along the channel. Both speed and latency should anticipate negotiating speeds that increase without limit.

    2) Objects that limit everything, such as addresses should be open ended so there is *never* a constraint on addressability, no matter how improbably large an address space should become. For those who need a concrete example, I should, even though we can't foresee using a 8192 bit wide address, be able to specify an address that is greater than 2^8192 bits wide. My ability to develop software should not be constrained by *your* imagination.

    ----

    The two issues above are constantly breaking things and we never seem to learn. Even though we can only imagine petabit speed and sub-nanosecond latency, there is no sensible reason to cast that *limitation* in stone.

    Imagine if old processors could fit into any modern socket and run as originally intended.

    Imagine if slots on a motherboard could all accept a smaller card and the card still work as intended.

    Imagine if a glacially slow device that has no need to be any faster can still communicate to other devices even though they are orders of magnitude faster.

    Imagine if there was no possibility of things like the current idiotic shortage of IP addresses. IPV6 is something of a stillborn mutant IMO. Why not take this juncture, since IPV4 is about to become unworkable, to switch over to a new addressing protocol that will *never* run out of addresses?

    Let the 'low level' guys deal with stuff like how to translate standards into hardware. At the point that things interconnect, they should only be constrained by their actual mutual bandwidth and latency. Protocols should support things *beyond* what the designers imagine are practical limits. If you think about it, that is the essence of planning for the future.

    We are already in a time when replacing broken things is often cheaper than repairing them. We are rapidly approaching a time when it is more costly to operate a device (electricity, floorspace, whatever) than to replace it. To ensure that we can smoothly transition in the future, we should make everything that *can* be future proofed, future proof.

This topic is closed for new posts.

Other stories you might like