C2D came along with solid-polymer caps (=> capacitor plague was over)

Yes, Core2Duo with 2 GB of RAM is quite good enough for general office work, web browsing, movie playback and the like. And the 45nm generation of Core2 doesn't even eat all that much power. Any CPU's before that, back to say the Pentium 4 heating radiators, have somewhat less performance and eat more electricity. But the C2D is quite okay. And there was another major change, that came along with the C2D: it was the solid-polymer electrolytic caps. This ended the "capacitor plague" era when motherboards only lasted like 2-4 years. With solid-polymer caps and C2D, motherboards do survive the warranty, do survive twice the warranty, do survive much longer. I work for an IPC assembly shop (which is a somewhat conservative industry) and in terms of desktop-style and PICMG motherboards, there are hardly any boards RMA'ed with solid-polymer caps on them.

Did I hear a rumor about "moving the VRM on-chip" ?

A few days ago, there were rumours that Haswell would have more say in how its input rail power is regulated - beyond the VID pins of today. Some even suggested that the VRM would move into the CPU package. Anyone knows further details? This would be more of a revolution than skin-tone enhancements...

interesting stuff

A couple questions come to mind... do the machines contain a Flash-based data logger, keeping track of temperatures over the service life / warranty period of the machine? As part of AMT 12.0 maybe? :-)

High-temp computing is interesting stuff. If you pay attention to board-level design, there is a lot you can do to help your design survive longer in operation at higher (broader) temperatures. And there's a lot you can *spoil* by careless board-level and system-level design.

MLCC capacitors (for power blocking) are made of several dielectric materials with quite different sensitivity to temperature - even if we speak "comparable" models in the range of tens of uF per unit, typically used to block low-volt high-amp CPU power. Some drop to ~40% of their capacity at -20 *C, some are much more stable.

Cheap Aluminum electrolytic capacitors also lose capacity at low temperatures and their ESR increases maybe tenfold, but even in conventional Al Elyts the chemistry can be slightly modified (alcohol added?) to make them perform better at low temperatures. (I hope the capacitor plague is over by now.) More importantly nowadays, solid-polymer elyts don't seem to have that low-temp problem at all, and they don't dry out at higher temp either. They last much longer. The downside is, that solid-polymer elyts are not made for voltages above say 30V - so you cannot use them at mains PSU primaries :-( So the PSU may well be the weakest spot in any computer, especially the PSU primary, which must contain conventional Al elyts and is typically a point of hot air exhaust, which certainly doesn't help the Al caps' longevity.

Next, in order to compensate for low-temp effects and gradual ageing, there may be room for designing in more capacity on a motherboard, just to be on the safe side, to have some headroom. Connecting more caps in parallel may bring the added bonus of decreasing the actual ripple current per capacitor, which decreases the capacitors' internal heating -> allows for operation in a higher ambient temperature. (The effect of cap addition -> ESR decrease might actually translate quadratically into the temperature difference / derating, which then translates into the cap's service life along some vaguely exponential curve.)

Effectively it all translates into attention to detail, and into board space occupied by the caps and by on-PCB heat dissipation space. Any additional heatsinks (e.g. on VRM FET's) mean mechanical design, which means substantial added cost (apart from designer headache) - so the board designers typically let the FET's run rather hot, because they're relatively insensitive... And space is always at a premium, especially in the ever-more-compact datacenter gear.

Let me suggest an interesting concept: if you could let your gear run at hotter ambient temperatures, you wouldn't have to use air conditioning (artificial freezing) all that much, you could use plain heat transfer more of the time. As far as I can tell, higher ambient temperatures are prevented by relatively sharp "temperature gradients" in the hardware = poor heatsinking.

Heatsinking seems to be a nasty can of worms for PCB and case designers. Especially fanless designs are highly suspicious in principle... It's interesting to see how different hardware vendors deal with this, deduce who's willing to take radical + effective + systemic steps, and who resorts to eyewash... (put shiny galvanized heatsinks on chipset + VRM, run some pointless heatpipes among them, then cover the biggest heatsink by a company logo badge).

I have to admit that in this respect, the top three name-brand hardware makers are generally in a higher league, compared to the noname market - and have been in a higher league for many years back.

One last observation maybe: even noname motherboards that started coming with solid-polymer elyts in the VRM, last much longer. The transition from the "plagued" Al Elyts to solid-polymer elyts also concided with a transition from P4 Netburst to Core2Duo (I'm Intel-based, sorry), which overall ran much cooler. My favourite way of building a long-lasting computer used to be: take an LGA775 motherboard that has enough VRM oomph to support the 130W P4's, and slot in a 45nm low-end C2D or Celeron :-) It tends to take a BIOS update, if the board has some older chipset.

Single point of failure => what about fault tolerance?

Okay, suppose the software can cope with NUMA on this scale... the next question is, how does RHEL or SLES or Windows Server cope with individual component failures? Is RAM and CPU hot-swap there already? What if a whole multi-socket blade goes down?

(Small hardware = small problems. It's so obviously soothing. Unless you sell too many of them, and there's a systematic design flaw... Have to pet the smartphone in my pocket just to feel basically sane again...)

Good idea = use HDDs instead of tapes

Now that's an interesting idea - use cheap notebook drives instead of tape cartridges. Maybe the tape cartridge survives a few more G of a mechanical shock, but owing to its internal magnetic heads and bearings (dust-tight environment), it should OTOH survive many more overwrites, longer hours of continuous operation. Which may also compensate the highter cost of disk drives vs tape cartridges (and maybe tape drives).

Why use some mechanical or electrical multiplexing of the drive lanes? For a smaller number of drives=cartridges, you can just as well use an expander, it may be cheaper than a robot + tape drives. If you have the know-how, you can build your own virtual library along that principle - buy some cheap hot-swappable case (such as SuperMicro SC216 or SC417, or SC847 for 3.5" drives) and build a virtual library on top of that. It's not much of a problem to shut down or spin up a drive in software, or keep it in some shallower stand-by state, and even to watch out for hot-swaps. An important part remains to be solved though: the management software candy on top, and some backup client software. Something to keep track of your "tape-style disk cartridges" and maybe provide a virtual tape interface on demand to the clients. Someone in the open-source camp with plenty of free time could as well start coding something like that :-) Okay, once you run out of drive bays and you need a robot, it's back to the library makers...

There are potential fields of application where it's intriguing to deploy a big robotised tape library, with several tape drives, to perform long-term archival of some data - such as from continuous video surveillance systems (big brother kind of thing) or medical Xray/CT data. I've been told by practitioners who have attempted that kind of thing, that tapes have downsides in this application. The tape drives wear out much too fast - not up to 24x7 continuous operation in video systems. And in the medical systems, the users tend to get addicted to the possibility of having a patient's past history always at their fingertips, so that the library again just keeps huming all the time and the users are disappointed about the access time ("hey it's all in the computer somewhere anyway, so why does it take so long"). Plus, in the medical imaging technology, the data volume just EXPLODES every time a new machine is installed in the hospital (having a higher resolution, being 3D rather than 2D etc). And, somewhere inbetween all that, the tape cartridges are not very reliable after all... It's a crazy world...

RAID software quality from Intel etc

Several issues come to mind.

Historically, Intel has had soft-RAID "support" in several generations of their ICH's - on top of SATA HBA's, up to six drive channels. A few years ago it was called the "Application Accelerator", then it was renamed to "Matrix Storage". I don't know for sure if there's ever been a RAID5/XOR Accelerator in there, or if the RAID feature consisted of some ability to change PCI ID's of the SATA HBA at runtime + dedicated BIOS support (= RAID support consisting of software and PR/advertising, on top of a little chipset hack). Based on the vague response in the article, I'd guess that there's still no RAID5 XOR (let alone RAID6 Reed-Solomon) acceleration in the PCH hardware - what they said means that they're looking at the performance and trying to squeeze out as much as possible out of the software side. Looks like not much is new here on the software part (RAID BIOS + drivers) - the only news is SAS support (how many HBA channels?), which gives you access to some swift and reliable spindles (the desktop-grade SATA spindles are neither), if the ports support multi-lane operation they could be used for external attachment to entry-level HW RAID boxes, and if the claim about expander support is true, you could also attach a beefy JBOD enclosure with many individual drives (unless the setup gets plagued by some expander/HBA/drive compatibility issues, which are not uncommon even with the current "discrete" SAS setups). I'm wondering about "enclosure management" - something rather new to Intel soft-RAID, but otherwise a VERY useful feature (especially the per-drive failure LED's are nice to have).

The one safe claim about Intel on-chip SATA soft-raid has always been "lack of comfort" (lack of features). The Intel drivers + management software, from Application Accelerator to Matrix Storage, has been so spartan that it was not much use, especially in critical situations (drive fails and you need to replace it). I've seen worse (onboard HPT/JMicron I believe), but you can also certainly do much more with a pure-SW RAID stack - take Promise, Adaptec HostRAID or even the LSI soft-RAID for example. It's just that the vanilla Intel implementation has always lacked features (not sure about bugs/reliability, never used it in practice). Probably as a consequence, some motherboard vendors used to supply (and still do supply) their Intel ICH-R-based boards with a 3rd-party RAID BIOS option ROM (and OS drivers). I've seen Adaptec HostRAID and the LSI soft-stack. Some motherboards even give you a choice in the BIOS setup, which soft-stack you prefer: e.g., Intel Matrix Storage or Adaptec HostRAID. Again, based on one note in the article, this practice is likely to continue. I just wish Intel did something to improve the quality of their own vanilla software.

One specific chapter is Linux (FOSS) support. As the commercial software-RAID stacks contain all the "intellectual property" in software, they are very unlikely to get open-sourced. And there's not much point in writing an open-source driver from scratch on top of reverse-enginered on-disk format. There have been such attempts in history and led pretty much nowhere. Any tiny change in the vendor's closed-source firmware / on-disk format would "break" the open driver. And the open-source volunteers will never be able to write plausible management utils from scratch (unless supported by the respective RAID vendor). Linux and FreeBSD nowadays contain pretty good native soft-RAID stacks and historically the natural tendency has been to work on the native stacks and ignore the proprietary soft-RAID stacks. The Linux/BSD native soft-RAID stacks can run quite fine on top of any Intel ICH, whether it has the -R suffix or not :-)

People who are happy to use a soft-RAID hardly ever care about battery-backed write-back cache. Maybe the data is just not worth the additional money, or maybe it's easy to arrange regular backup in other ways - so that the theoretical risk of a dirty server crash becomes a non-issue. Power outages can be handled by a UPS. It's allways a tradeoff between your demands and budget.

As far as performance is concerned:

Parity-less soft-RAIDs are not limited by the host CPU's number-crunching performance (XOR/RS). If you omit the possibility of sub-prime soft RAID stack implementation, the only potential bottleneck that remains is bus throughput: the link from north bridge to south bridge, and the SATA/SAS HBA itself. Some Intel ICH's on-chip SATA HBA's used to behave as if two drives shared a virtual SATA channel (just like IDE master+slave) in the old days - not sure about the modern-day AHCI incarnations. Also the HubLink used to be just 256 MBps thick. Nowadays the DMI is 1 GBps+ (full duplex), which is plenty good enough for 6 modern rotating drives, even if you only care about sequential throughput. Based on practical tests, one thing's for sure: Intel's ICH on-chip SATA HBA's have always been the best performers around in their class - the competition was worse, sometimes much worse.

As for parity-based RAID levels (5, 6, their derivatives and others): a good indicator may be the Linux native MD RAID's boot messages. When booting, the Linux MD driver "benchmarks" the (potentially various) number-crunching subsystems available, such as the inherent x86 ALU XOR vs. MMX/SSE XOR, or several software algorithm implementations, and picks the one which is best. On basic desktop CPU's today (Core2), the fastest benchmark usually says something like 3 GBps, and that's for a single CPU core. I recall practical numbers like 80 MBps RAID5 sequential writing on a Pentium III @ 350 MHz in the old days.

The higher-end internal RAID cards, containing an IOP348 CPU at ~1GHz, tend to be limited to around 1 GBps when _not_ crunching the data with XOR (appears to be a PCI-e x8 bus limit). They're slower when number-crunching.

In reality, for many types of load I would expect the practical limit to be set by the spindles' seeking capability - i.e., for loads that consist of smaller transactions and random seeking. A desktop SATA drive can do about 60-75 random seeks per second, enterprise drives can do up to about 150. SSD's are much faster.

The one thing I've recently been wondering about is this: where did Intel get their SAS HBA susbsystem from? Already the IOP348 contains an 8way SAS HBA. Now the Sandy Bridge PCH should also contain some channels. Are they the same architecture? Are they not? Is that Intel's in-house design? Or, is it an "IP core" purchased from some incumbent in the SCSI/SAS chipmaking business? (LSI Fusion MPT or Agilent/Avago/PMC Tachyon come to mind.) The LSI-based HBA's tend to be compatible with everything around. Most complaints about SAS incompatibility that I've noticed tend to involve an Intel IOP348 CPU (on boards e.g. from Areca or Adaptec) combined with a particular expander brand or drive model / firmware version... Sometimes it was about SATA drives hooked up over a SAS expander etc. The situation gets hazy with other less-known vendors (Broadcom or Vitesse come to mind) producing their own RoC's with on-chip HBA's...

DNS SRV records for HTTP *still* unsupported... (or are they?)

Audio visualization and 2D acceleration are the hot news TODAY? And a very simple web redundancy framework, potentially very useful, known and wanted by informed people for a decade, is still missing: SRV - just a small update to the DNS resolver. Just a few lines of code. The largely technophobic/ignorant web-surfing masses would appreciate it too, once it got into production use. A bug entry has been in the bugzilla for years, even with some early patches. The ultimate excuse from the Mozilla team has always been that there is no RFC standard. There is for SRV, but not specifically for SRV on HTTP.

https://bugzilla.mozilla.org/show_bug.cgi?id=14328

http://support.mozilla.com/tiki-view_forum_thread.php?comments_parentId=6112&forumId=1

The only party who would certainly not appreciate HTTP SRV, are the vendors of content switches for HTTP load balancing / HA solutions.

Mine's the one with BGP4+IPv6 in the pockets and global routing stability painted on the back...

R.I.P. QPI

Now, forget about the QPI folks, okay?

QPI has been the major buzzword around Core i7. Now suddenly it's not needed anymore. Not on a single-CPU desktop system, not when the PCI-e root complex has become a part of the CPU, and the CPU can talk "DMI" (notably similar to PCI-e 1.1 x4) straight to the ICH (sorry, PCH). Besides having an on-CPU RAM controller, of course.

Pump CO2 underground? How efficient is it? Basic physics

Imagine a 500MW coal-fired power plant. I live nearby one. Just think about the chimneys. How power-efficient would it be to pump the whole huge volume of exhaust gasses underground? How deep are the gas wells? How many metres of "water pillar"? It's one Bar every 10 m. Imagine that you'd need to compress a powerplant chimney worth of gas up to hundreds of Bars to pump them underground. Think about the heat produced, apart from the potential energy stored in the pressure difference. Does that seem worthwhile? Wouldn't it take more energy than the power plant would be able to produce?

Or, would you just take some water from down under, let the exhaust gasses bubble through it until most of the CO2 dissolves (and nitrogen bubbles back up), and pump the water back down to where you got it? That might be a tad more efficient... But, would the water take any further CO2, at our surface pressure?

Any solid data on that? URL pointers? Too lazy to do the basic maths myselfs...

@wireless setup

I've actually just installed Fedora 10 64b on an Acer notebook (a brand being dismissed with a grin by many of my colleagues) and IT ALL WORKS OUT OF THE BOX, including WiFi a/b/g/DraftN (Intel chip), bluetooth and a crappy integrated webcam that has a quirky driver in Windows. I did pay attention to having an Intel chipset in the notebook, and I went for a bargain model with an older 65nm C2D CPU and the chipset is two generations old by now, too.

As for WiFi: in the past I've configured WiFi by hand on a Broadcom-based AP with OpenWRT installed. So the first thing I did in Fedora, I tried a few lines with iwconfig. But somehow I couldn't get past WPA2. Only then did I take a fumble through the system configuration menus on the graphical desktop, and guess what: found a WiFi configuration tool, which did the magic with just a few mouse clicks - I managed to enter the WPA2 PSK at the first attempt.

Wow!

Further reading

from IDT and PLX - some PCIe switch chips with a prospect of multi-root architectures. Maybe pre-IOV, but the docs attached give a thorough technical background, answer many of my basic questions.

http://www.idt.com/products/getDoc.cfm?docID=18639469

http://www.idt.com/products/getDoc.cfm?docID=18688297

http://www.plxtech.com/products/expresslane/switches.asp

Interestingly for me, Pericom has not much to offer in that vein... Unsurprisingly, neither has Intel. That LSI paper on IOV mentioned before makes me wonder what LSI has up its sleeve.

PCI-e IOV - multiple root complexes can't talk to each other

Ahh well. So you can put your NIC in an external expansion box, rather than into the server itself. But unless it's a very special NIC that supports IOV, you can only use it from one root complex (= from only one host computer). The IOV standard simply says that the external multi-root PCI-e switch can carry traffic for multiple PCI-e bus trees that don't know about each other (like VLAN's). Each bus tree is "owned" by a particular "OS partition" running on the host computer. At least the part of the bus tree carried by the external switch runs virtualized on the switch, though I guess IO virtualization at PC server chipset level is already in the works too.

Any peripherial board that is to be shared by multiple PCI-e trees must have special "virtualization" support, to be able to keep track of several simultaneous PCI-e conversations with multiple root complexes. Not so very sexy...

I bet the HPC folks would appreciate much more if the external PCI-e switch could cater for direct root-to-root data transfers - for HPC networking purposes. Imagine DMA from one root complex to another root complex (memory to memory). This doesn't necessarily mean that a standard IP networking stack would be involved - perhaps as a slow fallback solution or for management purposes. Rather, I could fancy some dedicated low-latency mailboxing framework. It would really get the multi-root PCI-e monster fairly close to a NUMA, except that we're still speaking distinct "OS partitions" in the IOV lingo. The way I understand PCI-e IOV, such direct transfers are impossible. Maybe via an intermediate "virtual NIC" or some other peripherial along those lines (call it a DMA engine with some dedicated memory of its own) implemented within the external PCI-e switch.

The sort of bandwidth available from PCI-e at least makes very good sense for direct RAID storage attachment. Perhaps not via an additional intermediate storage bus technology (that could be useful as a slow lane for some wider-area SAN interconnects).

erase block size?

So what's the "erase block size" in the upcoming enterprise-grade flash drives? Did I hear you say interleaved erase+write cycles? How many ways/channels?

http://www.storagesearch.com/easyco-flashperformance-art.pdf

Re: how do they scale up

They probably mean the 5085, with 2x SFF-8088 (external x4 multilane SAS). That's two ports, 128 SAS addresses each, or even more with "fanout" expanders (see e.g. the SAS JBOD enclosures by AXUS).

Each ML SAS port can be connected to a daisy-chain or a tree of cascaded JBOD enclosures.

The internal SFF-8087 can also be used for cascading, provided that you have an expander-based SAS backplane in your server that provides an external SAS expansion port for daisy-chaining.

Those 256 drives may as well mean a firmware-side limitation, rather than the max.number of SAS addresses theoretically possible per a daisy-chain / cascaded tree. Still I'd be a little cautious about 256 drives per RAID. There can be real-world glitches that may limit the practically useful degree of cascading, performance with so many drives, choice of RAID level, maximum block device size that your OS can actually take, runtime reliability of such a monster etc.

I also keep hearing rumours of SATA drives being incompatible with some SAS expanders, or that you can only use a single expander per ML SAS port for SATA drives (no JBOD daisy-chaining) etc.

I believe the limit of 256 drives is the same with the older 3xxx series.

The 5085 is on par with an Areca ARC-1680x: same IOP CPU, same number of ports. As far as firmware features and comfort are concerned, nowadays I'd probably opt for the Areca.