This destroys anything IBM and Lenovo can or will ever come up with. Look for IBM/Lenovo customers to bail, especially considering that there's hardly anyone left to provide Server Hardware Support.
The first fruit of Hewlett Packard Enterprise’s buy of SGI is set to hit the streets in July with the release of a high performance system - the HPE SGI 8600. The system is a liquid cooled petascale box assembled on legacy SGI ICE AXA architecture, and is aimed squarely at punters involved with beefy scientific and engineering …
Wednesday 21st June 2017 23:29 GMT Nate Amsden
Thursday 22nd June 2017 08:18 GMT HPCJohn
Re: 10k nodes and no extra switches
the ICE Infiniband network architecture is that there are blade enclosures per rack.
I thought there were 8 in an ICEX (there are four in an ICE).
the blade enclosures have two or four (for dual rail) dumb Mellanox sswitch blades, as you say.
The switch blades are connected in a hyperbube topology.
there are no big aggregator switches.
Seriously though - when you expand an SGI system like that, someone at the factory runs a program, and the installation engineers come along with extra cables and a plug list of how to hook them up.
On the cluster intself there is a nifty utility to check the topology of the connections.