Re: A question on hashing
And collisions are a non-starter on SHA256/512 and aren't most newer implementations using something other than SHA1? The article you link to is a circa 2007 SHA1 walk-thru.
37 posts • joined 7 Sep 2008
And collisions are a non-starter on SHA256/512 and aren't most newer implementations using something other than SHA1? The article you link to is a circa 2007 SHA1 walk-thru.
I've been having internal discussions similar to what is in this post... prior to the big news.
Storage Ed in this other thread says this:
"You dismiss EMC dropping their pants. Their long datacenter dingdong will steamroll Pure and all these Flash startups. They can drop their trousers all day cus Xtremio like Pure is cheap commoditty hardware. The materials cost for a 50TB atray from either vendor is around $80k.. Tops. And EMC can operate with a dingleberry of margin while Pure starves."
What I mention internally is that same hammer is aimed at EMC. I'm sure EMC just loves the fact that XtremIO wins the VMAX takeout not "the dreaded competition." A pyrrhic victory. Not sure what it means other than being dependent on hardware margins (and software and maintenance thereof) is not a good place to be.. now and going forward.
Read the article and elsewhere, Infinidat is doing log structured writes also.
And as they describe it , virtual RAID groups. Each disk participating in numerous RAID groups, more here:
And no I wasn't implying that mainframe support is rocket science. It's just that the two have different target audiences. Without mainframe support, is it Enterprise (yes Enterprises run a whole bunch of kit, maybe a fairer description would be it isn't targetted for the high-end Enterprise?) But yeah, Nimble is interesting. Do you think they'll ever turn that standby controller into an active controller so it too can participate in serving IO?
Nimble went a traditional RAID under-pinning. Even went to triple parity in 2.1. The RAID arrays in Infinibox are 14+2 64k chunks dispersed as described. Caching appears similar but there are a number of similar caching schemes with SSD at this point. Nimble has standby controllers. Seriously? That's kind of lame. Infinibox is geared towards the Enterprise with mainframe support coming soon. Yeah, many Fortune 500s still support mainframes and some have quite a few of them. Regarding fragmentation, I don't think they fear it but embrace it. Writes hit "idle" disks (idle being relative but with 480 disks, some must be more idle than others) and reads are mostly cache hits.
Wait a second...
Some of us are paying attention out here, by the way.
What about this, doesn't HD growth still outpace SSD in the future (see chart in link below):
Is it no accident WikiBon study doesn't touch on total PB on the floor and ratios SSD<->HD?
If one of these guys is using tape, we could hope for a price war. At price parity and consumer as an end-user here, I'm not leaving Glacier. Photos and work docs when I leave is what I have in the big ice cube. Someone writes a nice interface like Fast Glacier and Google cuts their prices in half, I might consider switching. About taking it out, small users like me can trickle it out so I wouldn't pay to do that.
> IBM was the first to introduce sub LUN automated tiering,
Soran and crew created it and introduced it at Compellent in 2005. I'm not seeing anything
that talks about Easy Tier existing in 2005 nor 2006. Hard to tell when Easy Tier was introed via
dated google searches but it looks later than 2006, is that right?
As far as I can tell, the ceiling on the service life is mechanical. Everything I read, everything I've been able to google. If you have some hidden knowledge that shows otherwise, please share I'm much interested and would have me re-work my presentation B^).. heh. If there were a 7 year warrantied hard drive, the big players would use it in the 1000+ drive frames and perhaps extend their bundled warranties out further and make their long term costs less, ROI better , etc. There are so many reasons a much longer warrantied drive makes sense, but it isn't here yet - if ever. I believe it is a hard physics/mechanical problem. Again Charles 9, if you can show us otherwise, much appreciated.
> But note your own words: "other factors being equal".
The only thing I was trying to do there is "apple to apples", in other words a 7200 RPM 2 TB versus the same. 15K 2.5 900 GB SAS versus same, etc. So some twit didn't try to run off in a direction "of what about this and that" which invariably happens.
> IOW, they didn't have a good reason to build a seven-year drive.
Well.. this is like the 100 miles per gallon carburetor, it doesn't exist but fun to speculate none-the-less. I'd speculate that if vendor A were to deliver a 7 year warrantied hard drive, they would have captured a large segment of the marketplace (other factors being equal). The reason to go beyond 5 years is there and always has been.
Well actually, the service life (or warranty.. how long it is under warranty - take your pick) is 5 years. The reason manufacturers don't go beyond that is they can't. Spinning parts and the failure rate greatly increases. Plenty in the GooglePlex that speaks to this. These guys think that 50% of their drives will still be running in year 6:
"Going forward, we'll want our 3/4/5/6TB drives to last longer. My current server box is running 2TB drives, and most of the drives have 4-5 years of spin time already, with no urgency to replace them any time soon."
Maybe because you don't understand the risk? Everything you have RAID6? Because at some point UBE/URE may bite you in the ass and a RAID5 rebuild will go belly-up.
I don't think extending service life of hard drives will happen. It's 5 years for a reason. If someone could do 7 years, they would have and they would have cornered a nice chunk of the market. The problem of course is spinning parts, they only last so long.
You like to trot that article out don't you? It's from 2003. I think I read an article in 1980 that said IBM was killing off mainframes.
"POWER8 is the last POWER generation."
"IBM was showing off a part, has systems of all sizes up and running in its labs using the Power8 chips, and has been designing the Power9 processor for quite a while already, according to Starke."
Say... you aren't perchance a lib are you? Libs tend to make up facts, don't let facts get in the way of a good story, etc.
Haven't goggled it, but that wasn't a five year plan perchance? Five year plans don't work out so good. Heh. Problem with some of these larger companies, they lack elasticity. They can't change fast enough and/or they mis-interpret the change around them and head off in a direction and land some place the industry isn't going. High-end monolithic storage arrays on the decline... "hmmm... what does that mean? Where should we be heading?" etc. As Potts quotes Jobs: As the much-venerated Steve Jobs said: "If you don't cannibalize yourself, someone else will." I wouldn't suggest for a second this is easy, some will get it right others will shore up the walls for a time being and speak good wall street babble on earnings calls until it becomes quite clear they have been hollowed out and there is considerable erosion behind the walls. The "two for one" spends that Amazon, Google, FB, MS are employing are having a devastating effect on IBM,HP and other traditional players. Borrowed that last bit from Campbell, here: http://storagemojo.com/2014/10/17/shadow-it-pt-2/
"So you choose, 32-socket POWER7 or 16-socket POWER8 - you will not get better performance by choosing POWER8. So what is the point of POWER8"
In this case, cheaper licensing costs. One reason Power eroded Oracle on Sun hardware over the years is the Power perf was so much better even with the "factor", Sun made a lot less sense to run Oracle. I've been part of a migration , a huge migration from Sun to Power, migrating to a lot fewer cores and saving a bunch on Oracle licensing costs.
"A table of hardware costs Wikibon has prepared shows that, looking at 10-year cumulative hardware costs, a disk-only archive costs $5.5m while tape-only is far lower at $0.8m."
There's a Dilbert cartoon moment out there somewhere.
"Tape isn't sexy, let's kill tape"
"What's sexy got to do with it, it is a lot cheaper"
"I read in an airline magazine that facebook has a cool archive, we need to do something similar or we won't be cool"
"We don't have budget"
"I'll find the money, I think there are several projects that are a bit fat as it is"
"What will it gain us in the long run?"
"I'll call up me buds and we too will be in an airline magazine, free adverts, plus we will be cool!"
Actually... Comcast's greatest fear is when Verizon FIOS hits a neighborhood. I've watched our neighbors one by one switch to Verizon. I can tell collectively we are still with it by the wireless routers available. I got tired of the nickel and dime. I'd call them up to get the discount after my "special" expired, get sent to "retentions" and quite politely mention the great Verizon deal I am staring at, Comcast would do the right thing and give me the "special" pricing. After last go-round , I tired and switched to Verizon with lifetime DVR+triple-play, etc. they don't play the game , after special, my price went up all of 10-15 bucks, no biggie. The in-laws went with Comcast... I spent 3 months and a number of phone calls on their behalf just to get that $200 Visa debit card. You have to fight tooth and nail for those things.
Where will they come from? Overseas I would bet.
It won't be me. Been courted on several occasions to do storage. Not a chance. The nights and weekend work is enough to kill you. I've watched a few tired wretches do the wrong thing, one unfortunately blew up several hundred VMs ("hmmm... SRDF in this direction.. , wait, no, this direction ... oops"). The stress and demands are not worth it and the smart ones avoid it like the plague.
I wouldn't pretend everything belongs in the cloud. Certainly not with some of the workloads and access you describe. For others, they felt they had no choice. Netflix being the poster child.
http://bit.ly/1gVnxqX "The overriding reason for the Netflix move to a public cloud generally, and to AWS in particular, was that the company could not build data centers fast enough to meet the often-spiky demand of its users and that AWS was the only game in town in terms of the scale that Netflix needed." More details about that here and about - Google is your friend.
But yes, there will be corner cases and situations where it will not be a good fit. But the cloud folks are bending the cost curves down and the bean counters will count the numbers and the dash will begin. I'm not saying the dash is underway - but it will happen. "Then tell me, with a straight face, that the future is to have all workloads in the cloud." Maybe the reasons won't be good, but the numbers folks rule the day in the long run there will be a tipping point, I'll bet you on that and it will become quite apparent.
"The cost in lost profits from downtime" Yes. I often use this example of a cable cut and the havoc that resulted: http://bit.ly/1eCCFJT I expect there will be SMBs that totally moved to the cloud, cables are cut and folks have to work from home for a few days. Yes high profile burps in Amazon, but those are becoming less frequent as they tighten their processes.
I agree with many of these points. Because of latency and bandwidth, all pieces would reside
in the datacenter you are accessing remotely. Trying to do the hybrid thing is very expensive.
Regarding DR, I would hope there would be next gen solutions for that SRM on steroids or something
Elsewise, no Stuxnet to slow down a uranium enrichment program.
I recall back-in-the-day at a conference sitting next to folks at lunch and asking about what they all up to. Surprised at the number of consultants that worked at Nuke plants supporting the VMS infrastructure. Of course, I'm sure a lot of that has been ripped and replaced with Windows, with multiple layers of firewalls and VPNs - one would hope (if not air gaps, I have no knowledge nor care to regarding actual setup). If the majority of SCADA was still VMS based, we would have been so screwed ... no way to stuff a virus on it and slow down uranium enrichment programs - that's for sure. Bombs away (a lot sooner than planned - heh)!
Oh for those not good at reading between the lines or interpreting intent, there is quite a bit of snark in this post.
... and makes the nightly news as Brian Williams intones about the cesium plume approaching California. Narrative? Nuclear = Bad. Bald Eagle killing Wind power generation = Good. Oh the ecological gordian knots the greenies twist themselves into.
"This could go a long way to explain the c. 2°C temperature difference between urban and rural areas." That and two other factors. Cities are concrete and asphalt heat islands , secondly temperature recordng stations are often very poorly placed. On roofs or too close to heat sources like parking lots.
Long term trends? Sure, how's this work for you?
"And maybe a bit a scientific analysis might be welcome, as well: what part of the physics underlying the concept of radiative forcing do you find issue with?"
What are you talking about? Trot something out. Be specific.
The problem of course it is a very hard sell (warming) when you are freezing your ass off and you haven't seen this much snow in decades (large portion of US of A). Likewise, Europeans probably remember the winter of 2012: http://en.wikipedia.org/wiki/Early_2012_European_cold_wave
Couple that with a pause or plateau for 17 + years now in temperature rise:
http://www.forbes.com/sites/jamestaylor/2013/09/26/as-its-global-warming-narrative-unravels-the-ipcc-is-in-damage-control-mode/ toss in lowest ever Antarctic ice melt: http://www.theregister.co.uk/2014/01/03/antarctic_ice_shelf_melt_lowest_ever_recorded_just_not_much_affected_by_global_warming/
And the warmists are suddenly feeling like a politician with numerous scandals without a kiss-up press in their pocket. Oh. that's right, they have a compliant press, still a tough sell, ain't it? Barry was doing a full court warmist press conference from a golf course in Cali, handing out a billion plus for California to fight warming or some such. USA ain't buying it at all as warming polls very poorly here. Press on warmists, it'll be a tough slough.
Yeah.. and I suppose it gets worse. Imagine best laid plans. Google isn't coughing it up but I recall an airline in Minnesota that did the right thing, two separate carriers and then the construction company that cuts through the fibre bundle that is carrying both carriers. Easy prediction: We'll read about a company that went out of business because they were cloudified and the day and a half they were down, the customer abandonment rate was so severe they never recovered. And it will be a cut cable that put them out of business.
"On-premise data centres will also need bulk, online disk storage, with RAID rebuild time a continuing problem"
I keep seeing this... time isn't the issue. Failed rebuild is (obviously). The straw man is an additional
drive failure while rebuild is taking place. That's extremely rare. The problem in a RAID5 rebuild is
a bad block when rebuilding, tits-up at that point. To get around this, RAID6 is the answer for most.
But now we veer off into the more painful write penalty of RAID6, throw a lot of drives at it and the
pain lessens (simplification). MTDL for RAID6 is 110 years (google: intel raid6 paper). Which has
one scratching their head when you read about ZFS triple parity RAID. Still trying to figure out why
triple parity. Finally, SMART tech has most drives undergoing pro-active replacements making RAID5
less of a risk. But RAID5 still scares me. I've seen or heard of too many RAID5 failures on rebuild.
I perseverate - sorry.
I stumbled upon this and thought it rather interesting. Perhaps a case of the left hand not knowing what the right hand is doing. Or a bit of indirection?
Either way, one lift quote I found interesting:
Additionally, the economics of Glacier are not competitive with tape. Glacier's pricing is being promoted it as low as $0.12 per GB per year. The comparative costs for a Spectra T-Finity tape library are $0.0008 (cost per GB per month amortized over 5 years). Add in power, floor space and personnel costs for all 5 years and the total cost should still be well below $0.01 per GB per year for the period.
I'd think if it is close to a $.01 per GB per year, there is plenty of headroom for Amazon (assuming Glacier is tape) to come down in price.
This makes sense. Is the customer for this faster tier real small business and does cloud only backup? What would a larger business use it for, if a deep archive wouldn't Glacier make the most sense? For me, I'm more excited or interested in the decreased costs of Glacier for extended family photo archives and I store important documents just in case my laptop blows up. I would think Glacier is best for "just in case" solutions. Oh... the most important stuff for retrieval is in Gmail accounts. Free is best and I don't mind TLAs trundling through my stuff.
Re: erasure coding. Perhaps nothing is erased. TSM redux.
There were 3 quotes there, not a single quote. In a competitive industry, you often have to rely on somewhat sketchy sources - you would think. But anyhow, come over here for a follow-up comment:
Glacier is based, we understand, on data stored in tape libraries
Let's call it Snowfield for short and predict a cost of $0.0125/GB/month with Glacier potentially dropping to $0.0075/GB/month.
"This is due to the Glacier racks not having enough power provisioned to them to spin up more than about 1/5 of the drives in any given rack"
You sure about that? Reference please!
"Contrary to previous ideas that Amazon's Glacier cloud archival storage uses disk-based object storage, the latest word is that it is based on LTO6 tape. This comes from a nameless but senior person in the IT industry who "cannot talk about it". El Reg has also heard the same from another source in the general IT industry.
This idea of tape being the Glacier store fits in with the longish retrieval time for Amazon Glacier data and the cheap-as-chips cost structure Glacier has."
the latest word from a reliable source reveals that it uses LTO6. This was leaked out by a reliable senior source of Amazon, at the recently held IP Expo.
SpectraLogic is going from strength to strength. It has just been confirmed by a third person "familiar with the situation" that Amazon's Glacier archive service uses SpectraLogic tape libraries, thus enabling its low cost.
Which makes sense. Architecturally, I think they made the right choice with other LTO Gens coming online. I suspect Amazon will punish soon by reducing Glacier's already cheap costs. Using tape as a back-end , they have a lot more headroom to reduce as tape is quite a bit cheaper versus hard drives.
But the problem in general - long term archiving - demands the lowest cost. Retrieval time? Meh. We'll see how EVault does if they stay ~50% more costly than Amazon.
Tape is dead.. arrggg my head is exploding. The mainframe is dead too.
Anyhow, how about a real-world counter-point?
Now maybe some kool-aid drinking tape-is-dead fanboi floated the idea at the Chocolate Factory to
replace tape? And then someone else said "Why? And how are we going to justify that? More
importantly, how are we going to justify the greatly increased cost?"
But back to the puzzled tech author of the link above... tape? Seriously? And he prattles on about
how long it takes to restore, how many tapes (LTO2? ummm... how about at least LTO4 - but I digress).
He misses the main point... how much cheaper it is, he writes as if it is more expensive. How silly...
as if The Chocolate Factory hasn't figured out the cost.
Always comes down to money.. funny how that works. The tiers are considerably cheaper as you go down on $/GB, if they weren't ... there would be no need for them. I've been in massive shops with VTL and there is always tape in there somewhere. Financially, you'd go broke with multi-PB and keeping all those backups on de-duped HDD. I don't know how many times I've argued, if a tier is a factor of 5 or more in cost cheaper, that tier will be around for quite some time. Especially with database archiving solutions. The stale data resides on a cheaper tier. Makes no sense to keep that on a more expensive tier.
This is a joke, right? Do you even know how MTBF is calculated?
Do some reading chucklehead.
"LaBar and his colleagues report that testosterone levels among women and Democrats in the trial were unaffected"
Shouldn't that be "immeasurable" or perhaps "tiny like their kit"?
Gee whiz... what's with you guys? You get to the Rog and fall in line to talk up Sun?
Is one of the sign-ons to be a sun fanboy?
"The collection of all other vendors grew their total revenue at 38.5 percent, more than twice the market rate. We might possibly assume from that surprisingly high growth rate that vendors such as 3Par, Compellent, Data Domain, and Pillar are making good progress at the expense of all other vendors except Sun and NetApp."
Why not just go and see what fellow writers say? Chuck for instance?
The big market share gainer according to Gartner?
That would be "other" category at 2.3%, meaning more and more of the pie is going to the newer entrants -- collectively, that is. To me, this means that the market still has plenty of room for innovation from smaller vendors.
So there you have it. Those fast growers are now 2.3% of market share. Big whoop.
They certainly aren't eating into the others.
Sun? Irrelevant. Please lay off the Sun koolaid. Their stock tells you all you need to know about how far they've fallen.