Sure NVDIMM is new, but PCI Flash?
The use of NVDIMMs in storage controllers is a new idea. However, PCI Flash in storage controllers has been around for years in NetApp.
57 posts • joined 12 Sep 2007
The use of NVDIMMs in storage controllers is a new idea. However, PCI Flash in storage controllers has been around for years in NetApp.
The only option for data protection in VSAN and most other "Server SAN"/"Distributed DAS" approaches is mirroring, preferably a triple mirror. This creates much higher raw/usable ratio compared to parity RAID on external arrays.
This will also drive up power, space, and cooling requirements per usable GB.
The advantage of Server SAN/Distributed DAS is cost. EVO:RAIL requires the use of higher $/GB SAS disks instead of cheap $/GB SATA disks. Nutanix seems to be moving towards SAS solutions as well. However, the $/GB of server SAS HDDs is competitive with the $/GB of disk array vendors' SATA options. The same effect will likely happen with eMLC SSDs at some point. But all-flash array vendors might have the upper hand with their parity RAID, data reduction, and consumer grade flash enabled by aggressive write management, so the era of Server SAN might be limited as most on-line storage moves to all-flash over the next few years.
As for the idea of a local copy of data on the server, that was the idea behind EMC's Project Lightning/VFCache/XtremCache. But with other in-memory and server flash solutions (i.e., Flash DIMMs) gaining traction, big memory servers may become the norm, so local storage may become less relevant from a performance aspect.
Interesting. RDMA over Converged Ethernet (RoCE) is also "InfiniBand" over a different physical interconnect.
What is driving this was an activist investor who wanted EMC to divest its investment in VMware. It seems part two of this is sell off EMC's storage business to an acquirer.
I think if a server vendor owned VMware, it would not be good for the broader market. An independent VMware is more valuable to its stockholders.
So this likely was really about EMC's storage business, post VMware divestment.
That said, while there is overlap, it really is only the overlap between 3PAR and VNX. The rest of EMC's storage portfolio would find no real overlap, especially given HP's support of its HDS OEM relationship is lukewarm.
If HP purchased EMC, it could drive an acquisition of NetApp, most likely by Cisco.
If Cisco purchased EMC's storage business, it may drive an acquisition of NetApp, but is less likely.
Rock was much more than sixteen cores in four core clusters. Originally Rock did not call the cores in the core clusters cores, it referred to the core cluster as a core. The core cluster in Rock was four integer pipelines and one shared floating point pipeline. The four integer pipelines shared an instruction fetch unit and L1 caches. There was a dislike of calling the cluster multiple cores because at that time all CPU cores contained an instruction fetch unit, a dedicated L1 cache, and an FPU. It was only later when marketing decided a high core count suggested advanced engineering that the individual integer pipelines were called cores. This was consistent with the marketing of the various UltraSPARC T processors, which did not have a one-to-one ratio of IUs to FPUs. Rock's advanced features included hidden hardware helper threads to prefetch data (the "Hardware Scout"), the ability to simultaneously run both branches of a code branch ("Execute Ahead"), "reverse hyperthreading" ("Scalable Simultaneous Threading", which turned the four integer pipelines and their paired floating point pipeline into a single virtual core for HPC workloads), and transactional memory.
Rock's four core clusters shared an L2 cache. There was no on-chip L3 cache.
Rock had an in-order pipeline, but could execute out of order via Execute Ahead. If I recall, each Rock integer pipeline had four hardware threads, two for executing code (allowing Execute Ahead), and two for the Hardware Scout (to feed the two execution threads). Only two threads were visible to the operating system. Rock was interesting because it used threading to gain ILP.
It appears of the advanced Rock features, M7 only has transactional memory, although Solaris has used a software based version of scout threading (called Dynamic Helper Threading) since the UltraSPARC IV+ days and this was expanded in the various UltraSPARC T series.
1. FCOE exists because Cisco wants you to buy a boatload of switches.
10Gb iSCSI does the same thing.
2. FCOE exists because EMC is tired of customers reusing their existing FC switches and wants you to rip and replace the entire stack
10Gb iSCSI does the same thing.
3. FCOE exists because iSCSI is significantly lower cost and storage vendors would like to keep customers locked into thinking that storage is special and thus has to be expensive, when it's not and it doesn't have to be.
10Gb iSCSI is not significantly lower cost than 10Gb FCoE. FCoE capable NICs are standard on most blade servers today, and there is not a significant difference in the cost of a 10Gb NIC and an a 10Gb CNA for rack servers. Most 10Gb switches available today are FCoE capable, and 10Gb ports on legacy switches are very expensive. That said, FCoE suffers from the same qualification matrix issues that FC suffers from, while iSCSI generally requires fewer compatibility layers.
4. FCOE exists so that NetApp can finally do a decent block-access from their filers.
Since FCoE is basically FC protocol over 10Gb Ethernet, if NetApp FCoE block access is "decent", than so must be NetApp FC block access. And NetApp iSCSI is widely used.
"Neither EMC nor NetApp can retrofit efficient primary data deduplication to their legacy storage. Score one to Tegile."
As AC above, yes NetApp did "retrofit" efficient primary data deduplication to its Data ONTAP storage operating system with Data ONTAP release 7.2 in 2007,
Also, because NetApp deduplication leverages block checksums which were implemented into Data ONTAP's WAFL filesystem long ago, means there was no "retrofit" required. All that was required was to leverage the existing checksums with a deduplication software module.
The BladeCenter chassis, both the H and E, are tapped out on power, cooling, and space. The current Intel blades do not the high-wattage CPUs, and offer limited memory.
Out of band storage virtualization, that was Invista's domain. It lost IBM's in-band SVC storage virtualization.
However, VMware ESX 6 vVols will provide a new storage abstraction. Perhaps Bourne will have hooks into ESX 6 to provide more automation.
Based on Nimbus Data's blog post on the XtremIO architecture, it appears XtremIO is very similar at a hardware level to Isilon. That is, nodes containing considerable compute and disk drives, interconnected with InfiniBand.
My guess is EMC saw efficiencies of logistics and product development at a hardware level, with the prospect of integration at a software level. This likely encouraged EMC to go for the less expensive XtremIIO over the more expensive Violin Memory, or some of the other less expensive all-flash players.
Given XtremIO is little more than an Isilon-like hardware design with SSDs and block only access, it suggests any delays are either due to changes in the hardware platform for the sake of logistics efficiencies, or if EMC has kept XtremIO's original SuperMicro hardware, it suggests the delays are in the software, not the hardware.
If the latter, perhaps XtremIO's software was not far enough along, and it is taking EMC more time to get it enterprise ready.
That said, the XtremIO solution seems to throw an awful lot of hardware at the problem--four Intel Xeon CPUs and two InfiniBand HCAs per 16 SSDs--resulting in a lot of space, power, and cooing per GB.
For crying out loud. This story was being pushed by Tina Brown's "Daily Beast ", a left wing web news site, and by Arianna Huffington's "Huffington Post", another left wing web site.
Somehow, I doubt Media Matters took notice of America's political left's disgust at the political favors being bought in Washington D.C.
I think the idea of dedicated servers with large locally attached storage makes sense for certain applications. But putting a general purpose app on the world's most expensive RAID controller probably does not make sense.
That is all.
The stronger single-threaded performance should help decision support style workloads, as well as heavier duty jobs in the middleware space, such as supply chain calculations.
Also, with legacy Solaris container support on Solaris 11, those old Solaris applications should run well.
T11′s FCoE standard, FC-BB-5, had its technical draft completed in October 2008, was submitted for publication in June 2009, and published May 2010. FC-BB-5 includes the FCoE Ethernet frame protocol, and the FIP protocol (required for multi-hop). Multi-hop FCoE has been a published standard for over one year, and has been available in final form for two years.
The only IEEE Ethernet standards required to support FCoE traffic are IEEE 802.3-2008 PAUSE and IEEE 802.1Qbb Priority Flow Control. IEEE 802.3-2008 is standard, IEEE 802.1Qbb is complete and was submitted for publication a year ago (July 2010).
One other Data Center Bridging standard, IEEE 802.1Qaz, which includes Enhanced Transmission Selection (ETS) and Data Center Bridging eXchange (DCBX), is complete and was submitted for publication in November 2010.
IETF TRILL is not part of the IEEE, is not part of of the IEEE's Data Center Bridging Task Group, and is not required for FCoE or multi-hop FCoE.
Odd they would not at least support US-IV+ (Panther). It was released in late 2005 like the original UltraSPARC T1 (Niagara).
HP-UX used to run on PA-RISC, and was ported to Itanium.
NSK used to run on MIPS, and was ported to Itanium. And before that NSK ran on multiple proprietary CPUs, and was ported to MIPS.
VMS used to run on Alpha, and was ported to Itanium. Before that, VMS ran on VAX, and was ported to Alpha.
There is no reason these operating systems cannot be ported to x86, especially the Westemere EX processor.
Also, there is considerable hardware design sharing in HP. The HP C7000 Blade System was based on technology developed by Compaq/Tandem. The Superdome 2 is a hybrid of the C7000 and the original HP/Convex NUMA interconnect. With the Itanium 9300 and Intel 7500/E7 sharing Intel QPI technology, the hardware would not be a barrier. Notice how SGI was able to design the Altix UV to use Nehalem-EX processors.
I don't see much reason to port HP-UX to x86, but certainly NSK and VMS could be ported. Then HP could work with Red Hat to qualify RHEL 6.x on a Westmere based Superdome 2.
Cisco had a web conferencing solution, called MeetingPlace, prior to the WebEx acquisition. MeetingPlace was more limited compared to the emerging hosted solutions at that time (Microsoft LiveMeeting, WebEx, GoToMeeting, and WebDialogs). It seemed Cisco likely acquired WebEx so it could keep its MeetingPlace customers from going to hosted solutions.
If Cisco gave up its many WebEx customers to someone else, they could lose some control of their VoIP installed base.
Right on our need to keep Federal meddling out of start ups. Yes, the Feds have been involved in Silicon Valley inventions, but for Federal benefit, not just for the sake of doing it. The best thing the Feds did for Silicon Valley in the 1970s was deregulate the telecommunications industry, which helped drive the innovation which provide a large consumer market for the Internet and mobile communications.
Where Matt is wrong is in quoting Morozov's article. I just read this yesterday, having picked up the FP magazine in an airport. I laughed out loud at how wrong Morozov got it. Granted, this article was written in late 2010, and published in early January. But reading it on 24 January was quite entertaining. Morozov completely jumped the shark on this. Egypt was the Facebook revolution.
I am amazed Asay would use this article as an example. It appears the U.S. government's engagement with youth group leaders (started under the Bush administration), and the drive for freedom of the Internet accomplished in Egypt what took an army to accomplish in Iraq.
Sun bought the company LSC in 2001. LSC had two products, Storage and Archive Manager File System (SAM-FS) and the parallel SAN filesystem Quick File System (QFS).
SAM-FS was interesting because it was an hierarchical storage manager which functioned as a local filesystem, not as a separate product. Since SAM could integrate with other filesystems (SAM-QFS was also offered to the HPC market), there was talk of integrating Sun's ZFS with SAM to create SAM-ZFS.
Combining ZFS' ability to leverage flash nearline storage, and also clone and snapshot midline disk storage, with SAM's archiving to SATA and tape, and integrating this into a storage appliance, one could create a storage system which combines four tiers of storage (flash, nearline disk, midline disk, and tape) and do it in an intelligent and automated manner.
This, combined with Exadata for processing the data could be very useful for large data archives, but it would not solve the disaster recovery problem of off-siting data.
This is IBM's response to a trend of the server access network edge becoming integrated with the compute platform. This trend is best demonstrated by HP with Virtual Connect. HP has taken much IBM blade server share by integrating networking into the chassis in a way which adds unique, differentiated value compared to third-party switches.
This has become a bigger issue with Cisco releasing UCS, which also provides a compute platform with integrated server access networking and unique differentiation.
BNT makes sense, because BNT already provides products for BladeCenter and iDataPlex and BNT is acquirable.
This will allow IBM to build an integrated blade solution which is more competitive with HP C-Class and Cisco UCS.
The InfiniBand version of Xsigo used proprietary protocols for server IP and storage access (Xsigo vNIC and Xsigo vHBA). They submitted these protocols to the Open Fabrics organization, but I do not recall any mention of the Xsigo protocols in the latest Open Fabrics Sonoma Conference.
This does not appear to be an FCoE solution. It appears to be an Ethernet iSCSI front-end kludge to an InfiniBand bridge. So I assume storage goes from GigE iSCSI, to an encapsulated proprietary Xsigo InfiniBand storage protocol, back to native Fibre Channel. This could be done via a standardized iSER (iSCSI over RDMA) protocol. IP traffic could either go from native Ethernet to an encapsulated proprietary Xsigo InfiniBand IP protocol, or via a standardized IPoIB (IP over InfiniBand) protocol.
I assume Xsigo would use its proprietary protocols, as its central management of vNICs and vHBAs are what differentiates Xsigo's solution.
Either way, if indeed it is iSCSI to the host, this is little more than an iSCSI to FC router, with some added management intelligence. But the iSCSI router industry died a decade ago, once native iSCSI storage came along.
SGI would be the world's leader in computing, Convex would have bought HP, not the other way around, and DEC's Alpha would have ruled the processor world.
While it is correct to see the HPC influence on current Oracle products (Exadata is basically an InfiniBand cluster of x86 computers, combined with optimized processing nodes), Oracle does not need to be in HPC to do that R&D (they will do it anyway), and it is stupid to try to monetize R&D over the money losing HPC business (see SGI).
Sun went through so much pain with the Motorola to SPARC transition, and then even more so with the SunOS 4 BSD to Solaris 2 SVR4 transition, there was a "Never Again" declared within Sun's Solaris group. This is what led first to Sun's famous Solaris Application Guarantee, which was actually imitated (the sincerest form of flattery) by IBM a few years ago.
Solaris binary compatibility was so good even drivers worked across versions. Customers got so conditioned to things just working some were quite upset when they found their 32-bit Solaris 2.2 Adaptec SCSI card driver would not work with the 64-bit Solaris 7 kernel. That was a driver bit incompatibility, as the same driver worked fine if one booted the 32-bit Solaris 7 kernel.
IBM had a big problem when they went from the 64-bit version of AIX 4.3 to AIX 5.1. These two versions of AIX were completely incompatible, and it meant for example, you could not move your Oracle 8i database running on an p680 on AIX 4.3 over to a p690 on AIX 5.1 without also moving from Oracle 8i to Oracle 9i. That fiasco is what caused IBM to get its act together on compatibility.
IBM has cleared innovated tremendously in AIX over the last decade, but they have also copied madly from Sun. Those less transparent Solaris commands? Now available on AIX. Containers ... er, Workload Partitions? Now available on AIX. Branded Containers (i.e., Solaris 8 and Solaris 9 compatible containers in Solaris 10), now available on AIX.
The October 13th date is exactly six months after the benchmark report date of April 13th. This is standard procedure for most vendors. Six months is the limit on hardware/software availability. Even if IBM projected all components to be available in four months, it will put a date of six months out on the report, that way if product availability slips, they do not have to pull the benchmark.
I know vendors who would challenge competitor's benchmark reports when a part did not ship by the availability date.
This SPARC/Solaris capability goes back to 1993 with Cray's Business System Division (BSD) Cray Superserver 6400 (or CS6400), the SuperSPARC predecessor of the E10000, and Cray's OEM version of Solaris 2.3. The CS6400 had the ability to hot remove and hot add CPU/Memory boards from a running system. This was not a mature capability, as it required briefly suspending the operating system during the actual physical removal. Some applications could not tolerate the OS suspension. The E10K resolved that, removing the need for a pause. The pause based hot swap was brought to the UltraSPARC-II E3500 - E6500 with Solaris 7, pauseless hot-swap on midrange servers came out with the UltraSPARC-III F3800-F6800.
I wonder how a six-core, twelve thread Westmere at a similar price and/or TDP will compare to a native 12-core Magny Cours.
This is just Intel's OEM blade design (Intel Modular Server) with an SGI logo.
SMB does not seem to be one of SGI's target markets, but there could be a play in small scale HPC.
A Trace Portal Machine (aka "Puffer Machine") probably would have easily detected the powdered PETN in Umar Farouk Abdulmutallab underpants. These puffer machines are where aviation security and safety efforts should be focused.
SVC creates virtual volumes on Fibre Channel attached storage arrays. SVC does not function as protocol converter or storage router. It typically connects to the Fibre Channel SAN fabric close to the storage arrays, typically connecting to the the same Fibre Channel director class switches the storage arrays are connected to.
Unless SVC is virtualizing native FCoE arrays, there is no need to put FCoE attachment on the SVC.
Methane can be Fischer-Tropsched into jet fuel.
The Cisco Nexus 5000 and the Brocade 8000 can be used exactly like the Xsigo and VirtenSys solutions, using FCoE as the storage protocol, and Ethernet as the fabric.
Half-height and blade mezzanine FCoE Converged Network Adapters (CNAs) are available today from QLogic, and will soon be available from Emulex and Brocade. Blade mezzanine CNAs are available today from QLogic, and will soon be available from Emulex.
Storage administration (zones, LUN masking, etc.) in an FCoE environment is identical to a pure Fibre Channel environment. Ethernet management (VLANs, etc.) in an FCoE environment is identical to a pure Ethernet environment. Management concepts unique to Fibre Channel and to Ethernet are mature and robust. Only FCoE specific features (FIP, etc.) are new. This means a minimal learning curve for FCoE compared to InfiniBand or PCIe based networking.
There is little reason to use Xsigo unless you have a need for an InfiniBand network beyond I/O consolidation. Xsigo might be the best solution for an InfiniBand HPC cluster. As there is little existing PCIe based networking in the world (except in the cPCI based real-time and networking space), the only place VirtenSys' technology makes sense is in those markets, where the customers already know PCIe semantics.
Next week Sun was to present a paper at the International Symposium on Computer Architecture:
"Simultaneous Speculative Threading: A Novel Pipeline Architecture Implemented in Sun's ROCK Processor"
The only benefit Itanium had left over Xeon was its Pelliston cache self-healing and its Machine Check Architecture. If Nehalem-EX has both, it is game over, Itanium. Solaris has the RAS infrastructure to take advantage of MCA, and that would provide the alternative for an HP-UX environment.
The Oracle Processor Core Factor has nothing to do with discounts. It has to do with how many licenses are required for a particular processor.
The quad-core SPARC64 is the SPARC64-VII, not the SPARC64-VII+. SPARC64-VII is rated at 0.75 per core (three Oracle licenses per quad-core chip). For an IBM POWER6, two licenses per chip are required (two cores, 1.0 licenses per core).
Originally, Oracle's rule was based on the fact early dual-core RISC chips were about 1.5 times as powerful as the single-core chips which preceded them (hence a 0.75 multiplier), and the first dual-core x86 chips had a much slower clock rate and had a performance on par with the fastest single-core x86 chips (hence a 0.5 multiplier).
The UltraSPARC T1 0.25 multiplier was basically a gift from Oracle to promote the T1 platform. A 0.5 multiplier was more accurate based on performance. Likewise, the 0.5 multiplier for Itanium was a gift to HP. Why Oracle has not rescinded it like the UltraSPARC T is beyond me. Oracle has tripled the licenses required on the UltraSPARC T with three iterations of the chip, and Oracle has only increased IBM POWER licenses by 33% over the course of three iterations, and maintains Itanium at a significant license discount compared to SPARC64 and POWER.
For some reason, Oracle only requires 0.75 licenses on the SPARC64-V single-core processor, the only single-core processor which gets that reduced license.
It doesn't have to make sense. It's Larry.
Cisco has about 50% market share in SAN Directors (big modular switches). In overall SAN revenue (both director switches and top of rack SAN edge switches), Cisco has significant market share.
It is fair to say Cisco's success in the SAN business is what forced Brocade to buy McData and Foundry.
I do not understand why you think servers connected by a Nexus 5000 require Ethernet based storage, or native FCoE arrays.
The Nexus 5000 splits FCoE traffic from the server out of Ethernet and connects to standard Fibre Channel SANs.
So any server using FCoE to attach to a Nexus 5000 can leverage existing FC based storage.
This is a new card, and apparently uses new Ethernet silicon (not the Intel Oplin Ethernet ASIC on the current CNA). The new Emulex NIC is not yet shipping, so it is very possible it does support full offload (TOE), and the associated benefits for iSCSI and iWARP/RDMA protocols.
That said, partial offload may be adequate for iSCSI, depending on the workload.
The benefit of FCoE is it works with existing FC storage, without the need for special gateways. While it requires enhanced Ethernet (CEE/DCE), it only requires that enhanced link until the storage traffic is separated and put onto a standard FC link, which typically will only be from the server NIC to the access layer DCE switch.
There are DCE capable products on the market today. Cisco's Nexus 5000 is the best known at this point, and some other vendors claim to support "lossless" Ethernet and FCoE, which would suggest they support pre-standard DCE features.
One thing the article fails to mention is NICs will have to support these DCE protocols as well. Emulex's and QLogic's FCoE Converged Network Adapters (CNAs) support DCE protocols, Intel's 82598EB "Oplin" NICs support DCE protocols, and some other NIC vendors claim either support DCE features today or will support DCE features in the future with firmware upgrades.
It's called "FlexAddress". It decouples and virtualizes the Ethernet MAC and Fibre Channel WWN. It is very similar to IBM Blade Open Fabric Manager and HP Virtual Connect.
I think the Victoria Falls chip dispenses with the on-chip 10GigE to make room for an SMP interface. So if there are Vic Falls chip bins with bad SMP interconnects, a uniprocessor platform would be a great place for them. The chip cost would essentially be zero. Combine it with a lower-cost platform (SATA vs. SAS disks, etc.), and Sun would have a nice entry box. It would also allow Sun to EOL the T1000 and T2000, which would reduce costs.
Mainframes have not used CISC processors in years. They execute CISC code, but on RISC processors using hardware CISC decoder which converts the CISC operations to RISC operations. This is exactly what AMD and Intel have done for years on x86, ever since AMD's K5 and Intel's P6.
I recall reading somewhere there was significant similarity between one of the IBM mainframe RISC processor cores from the early 2000s and IBM's in-order RISC RS64 processor core from the late 1990s.
I'm sorry, but how is this IT news?
"It is true that you can resource cap the CPU usage using Solaris Containers, but Oracle don't recognise Logical Domains (LDoms) as a hard partitioning method for capping CPU resource. So even if you had a 4 core LDom set up for Oracle you would still need to license all cores present in the server whether you are using them or not."
All you have to do is run a capped non-global container within the LDom which is running Oracle. Oracle does recognize the capped zone as a license limit.
Honestly, all software should be deployed in a non-global zone, and all Solaris servers should have at least one non-global zone established to run software. It makes for more secure computing.
It was OJ in the courtyard with the knife.
Chances are both companies are incorporated in Delaware. Over half of all U.S. publicly traded corporations are incorporated in Delaware.
The best Mellanox ConnectX MPI latency I have seen is about 1.2 microseconds with a switch in between. The switch adds about 200 nanoseconds to the mix, so back to back (the configuration of the ScaleMP system) would be about 1 microsecond, assuming similar performance to MPI.
Certainly NUMA memory management in the OS helps, and I assume the COMA aspects are provided by the hypervisor. This could lead to some unhelpful double buffering, unless the OS kernel is aware of the underlying COMA caching.
This system sounds similar to what Virtual Iron was pitching in the past, but abandoned. Is there a relationship between ScaleMP and Virtual Iron??
Also, is this InfiniBand based approach what is under the covers of the larger ScaleMP systems?
Kidding is confusing which power-mad female board member was behind the pretexting scandal.
Fiorina was canned in February 2005. The board investigation which caused the pretexting scandal was started in response to a CNet article about an HP board meeting almost a year later under new CEO Mark Hurd.
The woman behind the pretexting scandal was HP board chair Patricia Dunn, ironically the same woman who conspired to get Fiorina fired, and who became HP's board chair by ousting Fiorina.
Why on earth would anyone buy a four-core Tukwila when they can get a six-core Penryn?
If Dunny will fit in HP's double-height BL680c blade, that would be 192 cores per C-Class chassis. If HP can increase the memory in the BL680c to 256GB, they will have eight 24 core, 256GB servers in 10RU.
Tell me again why I need a Superdome?