Funny...
"The deployment model is as a hyperconverged architecture where both storage and application services are run on the same infrastructure.
Hmmm. Sounds a lot like Hadoop.
IO, IO, it's off to Weka.io we go: Let's take a look at a file system upstart
Weka.io claims its file storage device can scale out to billions of documents. Rather than rely on object storage to cope with this kind of data, you just need to reinvent the filer, which it reckons it has done. The WekaIO Matrix is said reinvented filer: it's software that runs on any SSD-packed commodity server or cloud …
-
-
Thursday 13th July 2017 14:29 GMT liranzvibel
Re: RAM requirements
14TB is the total RAM of all instances.
We run in a HyperConverged mode where we take some resources and run concurrently with the applications.
We allocated 2 cores (out of 32) and 13GB (out of 240GB) on each instance. Just a small fraction.
So total RAM allocated to Weka.IO was 60x13 = 780GB.
The remaining 13,620GB remained free for the user applications to use and did not contribute to storage performance at all.
We have worked hard on making sure we utilize minimal resources from the servers, as we wanted the overall hyper converged economics to be very compelling.
-
Thursday 13th July 2017 17:10 GMT undercoverdave
Re: RAM requirements
Could you maybe give a concrete example of the compelling economics?
Being a newcomer, the burden of proof lies on you to show your value, beyond just technologies and buzzwords, what use cases (with concrete numbers and pricing) does this address? Who should be looking at this and say, "Yeah, I'll take that risk, the numbers show it's worth it."
-
Thursday 13th July 2017 17:21 GMT Alistair
Re: RAM requirements
Welcome to the Reg Liran.
Prepare to be mauled by techies. *grin*
I only wonder what criteria are available to handle *stale* status and offloading of said stale data objects.
What is it that puts Weka over ZFS or HDFS or other bulk storage fs' currently available. I have HDFS and ZFS experiences in large data pools - mostly dealing with telecom data types - the biggest issues I've experienced have been lookup latency, (well that and java coders that seem to think that they're talking to an RDBMS rather than a file system).
What tools can be used against this FS that give it advantages?
(the icon is labelled "windows user" but I use it for the crusty old bastard impression)
Alistair.
-
Friday 14th July 2017 08:50 GMT Liran Zvibel
Re: RAM requirements
Hi Alistair,
Thanks for the warm welcome :)
Our FS is flash native, and handles metadata in a much more optimized way than ZFS (and obviously HDFS), further, thanks to our parallelism and scale-out nature, we allow you to run many more workloads on the same storage system.
I can give you two quick examples, through there are countless.
One customer (large EDA company which we'll reveal soon) had tested us for SW compilations, and shared build caches. They did one test of publicly code, where they compiled the full Android project. Over an Oracle ZFS appliance it compiled in 220 minutes. Using the same compute, on WekaIO it compiled in 38 minutes, so almost x6 times quicker.
Another example is that for genomics. We were tested against Isilon Nitro (all flash Isilon). The BOM of our servers was WAY below the Isilon price (or BOM). We finished six parallel large conversion in 55 minutes, where the Isilon finished in almost 2h.
Our FS will improve your workloads if you care about small IOs and metadata (like SW build) or if you care about throughput (like this genomics analytics case).
Liran
-
-
-
Biting the hand that feeds IT
