back to article Rackspace to hit GLOBAL CLOUD REBOOT button to flush out Xen security nasty

Rackspace has warned its customers that it plans to reboot all of its servers across the globe to nix a security bug that was first spotted in the Xen virtualisation platform earlier this week. The managed cloud outfit told its customers about the "maintenance work" in an email, seen by The Register, that was sent out early on …

  1. Steve Knox
    Paris Hilton

    Do they not have the capacity or capability...

    To spin up a patched host cluster, migrate existing guests to the patched cluster, elastically growing the cluster as necessary?

    Wasn't that the promise of the cloud? No downtime because if there was an issue your guest could be dynamically moved to a fixed environment, which could grow as the buggy environment shrunk?

    Where I work, our network admin has done that many times with our little VMWare cluster, migrating live clients to new hosts, patching the orginal hosts, and migrating back, allowing maintenance to have exactly 0 impact on operations.

    I know, I know, Rackspace and Amazon are massively more complex environments. But if the increased complexity doesn't give you even equivalent stability, WTF is the point!?

    1. A Non e-mouse Silver badge

      Re: Do they not have the capacity or capability...

      I believe the cloud providers give you resilience by having servers spread across regions & data centres - which are supposed to be isolated from one another in every way possible.

  2. kr8tr

    It is not (of course) that simple

    Much more goes into play on issues like this. It SOUNDS simple to replicate on fresh hardware, etc, but all that replication takes time, takes bandwidth, requires a 2X infrastructure and that doesn't include the fact you need to validate the copied data before you delete the original, etc. Plus, managing IP address changes, the pure amount of storage required and the complexities of doing this on a massive scale with tens of thousands of servers and you start to get the picture why this is not done often and why it just can't be done now without customer pain.

    It is just not do-able with our current technology. Not in a cost effective way. Nothing customers really want to pay for anyway...

    Rob

    Rob La Gesse

    VP, Social Strategy

    Rackspace

    210-845-4440

    @kr8tr

    1. Doctor Syntax Silver badge

      Re: It is not (of course) that simple

      "It is just not do-able with our current technology. Not in a cost effective way. Nothing customers really want to pay for anyway.."

      But as Steve said, wasn't that the promise of cloud?

      1. Bill the Sys Admin

        Re: It is not (of course) that simple

        agreed this is the major selling point of the cloud! I agree maybe its not cost effective...but imo its being marketed very poorly if this is the case.

    2. Steve Knox

      Re: It is not (of course) that simple

      I understand there is additional complexity, but

      "requires a 2X infrastructure"

      is completely wrong.

      Either you're making this up or you do not understand how to manage a virtual infrastructure. You could do this with a single additional host with enough resources to support your largest single-unit guest environment. That would be quite slow of course, but you could do this in a reasonable amount of time with 1.25 to 1.5x the current utilized infrastructure.

      And if you don't already have at least 1.25x utilized infrastructure available to begin with, I definitely don't want to be playing in your cloud.

      And if your infrastructure isn't capable of live-migration of guests from host to host, you need to invest in technology less than five years old.

    3. Anonymous Coward
      Anonymous Coward

      Re: It is not (of course) that simple

      Kudos for monitoring and replying in a timely way without too much management BS... ;)

  3. jake Silver badge

    Eggs.

    Basket.

  4. Anonymous Coward
    Anonymous Coward

    Migration

    Some of the larger cloud providers (I believe Amazon and based on this Rackspace as well) do not migrate VMs, so even if they had sufficient hardware capacity to do a rolling upgrade, they can't.

    The reason for this is not just cost saving (though I'm sure that plays a part), but migrating a VM is not entirely transparent - some workloads won't tolerate the brief downtime or various device resets that occur, and so will fail anyway, and with PV Linux VMs older kernels would often crash (and a cloud provider generally has no way of forcing their users to update their kernels). Even with HVM guests, if you're not using some form of PV drivers, the device resets the emulated devices may see can be sufficient to cause an upset.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like