Posts by Gordan

Just FYI - you can to PCI passthrough on ESXi as well, and many people have successfully gotten it to work with modified Nvidia cards.

This, however, is probably not a particularly suitable solution because you would need another machine (e.g. a laptop) to run the VM management tools from, whereas if you run KVM or Xen the management can be done from the local machine.

Re: The other way around?

I'm not sure what features KVM has and supports, I use Xen because it is far more mature and performs considerably better.

Recently the Xen guys have been working on adding an additional reset method - bus reset. This may or may not make it into the Xen 4.5 release, I'm pretty sure it isn't going to be in the upcoming 4.4 release, so you are looking at at least 6-12 months before the feature is in the release branch and available pre-packaged for your distro. That is a long time to be hanging on for something that might but is not proven to solve the problem. The Nvidia solution works perfectly now.

It is also not the only issue I have had with Radeons - there are many others. For example, the XP drivers have utterly broken and you cannot glue together multiple monitors into a spanning resolution above 3200x1600, which is completely useless when I need to stitch together two 1920x2400 stripes to get my IBM T221 to work properly. There are other issues as well that I am not going to go into now since they are off topic, but suffice to say that Nvidia suffers from none of those problems.

I would strongly advise you to stick with proven hardware and software. Anything that is bleeding edge and unproven is going to put you at a very high risk of running into bugs and regressions in hardware, firmware and software. This is another reason why I strongly recommend you get one of the Citrix approved workstations for VGA passthrough. In terms of software, something like EL6 + Xen RPMs from the source I mentioned is a good, stable, proven choice, and since you aren't going to get RH support for anything involving Xen you might as well go with CentOS or Scientific Linux, or see if you can get a vaguely reasonably support package on XenServer (based on CentOS).

Re: It's nice to dream...

Actually, you can pass through the GPU better than the CPU. Unlike with the CPU which you virtualize, you can pass the GPU device completely to a VM. Inside the VM, the GPU driver loads as it would if it was running on bare metal and provided you have a GPU for which the drivers work virtualized - ATI technically works but is too broken to be usable in a serious way, Nvidia Quadro cards work without any issues, as do GeForce cards modified into corresponding Quadros, but unmodified GeForce cards don't work because the driver has a whitelist of PCI device IDs which it will allow to boot up virtualized.

Trust me on this - I am typing this on a triple-seat virtualized rig, of which 2 seats are for heavy-duty gaming (with modified GTX780Tis) , and the 3rd is my Linux workstation.

Dual Booting Same Instance of Windows native and Virtualized

This generally doesn't work particularly well. All the underlying drivers will be different, and it is akin to replacing the motherboard with a completely different one - Windows doesn't handle this gracefully at all, and you will more often than not find that instead of greeting you with the login screen it will greet you with a BSOD when it finds that the paravirualized SCSI controller it was expecting to be finding it's C: partition on doesn't exist on bare metal.

Maybe he just doesn't have the space (he already said he has no desk space for a 2nd keyboard). Seriously creating a setup like this is not difficult if you have hardware that isn't buggy. I have a 12-core (24-thread) physical machine, with two VMs given 4 cores (8 threads) each, and I can still run big software builds in Linux dom0 while having two L4D2 or Borderlands 2 gaming sessions on the go on the same physical machine.

Re: The other way around?

@Manolo

There are not one but two options for passing through USB devices in Xen. You can use PCI passthrough to pass the USB controller through, of you can use USB passthrough to pass a specific device through. The former is usually a little more efficient, but the latter is more flexible (e.g. if multipler ports are on the same USB hub and you need to pass different USB devices on the same PCI USB controller to different VMs. For example, I have 2 VMs with a mouse/keyboard passed to each one via PCI passthrough, and it works extremely well.

Re: Number6

I'd just like to point out there are several ARM boards available with SATA ports, including the recent SheevaPlug, GuruPlug and DreamPlug, TonidoPlug2, CompuLab SBC-A510, Cornfed i.MXQ board, Higher end SolidRun CuBox-is, TrimSlice (note: it's SATA port is actually running via a built in USB->SATA adapter because CompuLab couldn't get PCIe SATA working properly for some reason on Tegra2) just off the top of my head.

On the point of how long it takes - I'm pretty sure it would take me substantially less time to set up a DreamPlug with Linux from scratch to act as a DHCP, DNS and NTP server that it would take to install Windows Server and configure the services on an even remotely similarly specced Atom machine. The process for the DreamPlug would be:

1) Extract rootfs to disk (one-liner)

2) Copy modules/firmware to rootfs and the kernel/initrd to bootfs (2 lines)

3) Configure uboot to tell it where to get the kernel from and tell the kernel where the rootfs is (2 lines)

4) Reboot into new OS

5) yum install dhcpd bind bind-utils ntp ntpdate

6) dhcpd would need 3-4 lines in the config to tell it IP ranges DNS and gateway to serve plus any specifics for more fancy things, e.g. assigning specific IPs to specific MACs. NTP should need no extra configuration, and Bind comes configured by default to act as a caching name server.

All of that would take little more time than it did to type this message.

Re: High performance ARM

@AC

How are you meaningfully comparing revenue in a meaningful way between Oracle and a FOSS package?

Also how are you comparing the "volume" between MS SQL and a freely downloadable FOSS database? If we are comparing the number of downloads of MySQL (and don't forget to include the distro ISOs rather than just the deb/rpm/tgz as well as summing the totals for MySQL, Percona and MariaDB) with the number of copies of MS SQL shipped/downloaded from MS I suspect that the "volume" of MySQL deployments is some orders of magnitude bigger than MS SQL.

If you are going to compare statistics then make sure those statistics are meaningful in the context of all the items in the comparison.

Re: LLVM

This was discussed to some extent back when soft-float/hard-float issue on arm was being considered. In the end, the conclusion was that rather than coming up with a dynamic linker bodge that could handle both and rolling fat binaries that contained the payload for both, this taking up double the memory (or requiring awkward strapping that would purge the non-executable payload from memory after mmap()-ing it) it was more sensible to have a clean break. And you know what? This isn't such a big deal because the vast majority of software people run on ARM is FOSS.

The question is really why would you want a single hugely bloated executable that runs on multiple platforms? What problem does it solve that isn't much better solved by just having different binaries? You have to compile it once for each platform before gluing them all together anyway, so what does it actually gain you that shipping 4 different binaries (or a zip file, or shell archive containing all 4) wouldn't do just as well?

Re: "SUSE / Redhat for Linux stuff"

Aarch64 has not been in any official Fedora build thus far. Last time I checked there were only preview developer builds because there were still a few important packages that were broken on aarch64 due to package and toolchain bugs.

GCC took quite a while (a year or two) after ARMv7 hardware was easily available before the hard-float support was working sufficiently well to be suitable for full distro building. It would not be surprised if ARMv8 support at toolchain level takes a similar length of time to become sufficiently stable after the easy general availability of hardware..

Re: High performance ARM

It's funny you should say that Vendor only supports "enterprise class" DBs, and then you mention Oracle - when Oracle now owns MySQL. Oracle DBs don't even feature in the top 10% of biggest databases by size or throughput I have seen in the field, every one of those has been MySQL. Oracle is down there in the noise with PostgreSQL. I'm not saying that PostgreSQL isn't as good a database - far from it; I'm merely saying it doesn't seem to be as popular.

I'll let Lew answer the exact cost part (I had no idea he was reading the reg forums :)). I had already mentioned that seeing a product without a price tag was offputting.

Thankfully, for once the lack of a pricetag does not equal to a lack of a product.

Re: High performance ARM

@Fenton:

Have you heard of ARM's big.LITTLE achitecture? It gives you 4 Cortex A15 CPU cores and 4 Cortex A7 CPU cores. The A7 cores are a little less powerful, but a LOT less power hungry, so depending on the load the system is on, it will use one or the other set of cores (or all 8 cores on the recent Exynos CPU systems such as the Arndale board). So ARM already provides this kind of functionality on a single slab of silicon.

As for your apps - if they are based on closed, proprietary technologies you are indeed screwed, but that was always going to be the case, starting with the cheque you write to the vendor for the licences every year.

Re: Database file compatibility

@Fenton:

You can already do this with both MySQL and PostgreSQL. The on-disk formats and wire protocols are exactly the same, completely compatible, and endiannes independent. I do this sort of thing all the time.

I spoke to them this week and my motherboard is in the post. Definitely not vaporware, unlike the AMD offering. Don't get me wrong - what AMD announced looks _great_. I would love a powerful 8-core ARM (as in, more powerful than Arndale) with 128GB of RAM (mock build everything on tmpfs!) to replace my build farm of Sheev/Guru/Dream Plugs. Unfortunately, I cannot wait for another 6 months for that to become available - I need to get my EL7 ARM build going at a decent pace _now_.

Both Debian/Ubuntu and Fedora/EL based distributions now have their fully featured ARM variants, so for a typical Linux shop running a LAMP or similar stack there really is nothing to port - it has all already been done. I have worked with several clients whose LAMP(ish) stack projects migrated without any problems at all on the ARM variant of the same stack.

The biggest obstacle to addoption, IMO, has been the lack of good, standards compliant ARM motherboards - and by this I mainly mean *TX form factor. Random-slab-of-PCB form factor for the vast majority of ARM machines is a huge pain in the backside, as is a frequent lack of SATA ports for any more serious use. AMD's recent noise about the A1100 Opterons is welcome but the announcement was vastly premature - I spoke to them earlier this week and was told that the earliest these might be availble to developers is 2nd half of this year, which is not what you would expect after the announcement fanfare; call me old fashioned but when they talk about something the way they do I expect to be able to get my hands on one within a week or two at most.

Having said all that, there does seem to be one ARM offering that is head and shoulders above the rest in terms of standards compliance (*TX form factor, SATA), specification (4-core Freescale A9, 4GB of RAM) and cost (~£220 for the motherboard which includes the CPU, RAM and PSU, cheaper than comparable x86 Atom based offerings). To put the price into perspective, this is only about 2x the cost of a SheevaPlug, but has about 7x the CPU performance and 8x the RAM, plus the huge added *TX convenience. Best of all - you can buy one _today_. Google cornfed servers and you should be able to find it.

Re: RedHat?

If the whole point of RedHat is a support contract, then how come that CentOS, Scientific Linux and RedSleeve have a thriving community? Last time I checked Facebook uses CentOS, rather than RedHat, as do many, many other very large companies with thousands of servers.

People run RH clones for reasons or familiarity and stability (as in no chasing of continuously teleporting goal posts), and many companies want to do things that aren't supported by RH and/or have good enough sysadmins in-house that they don't need vendor support.

Re: put an Aakash in the hands of every school kid in the country in 5-7 years

"I am sure that this will be able to run up-to-date web browsers for the next 5-7 years,"

How do you figure that? First Android device was released in late 2008, about 5 years ago. No Andoid device older than maybe 2 years will run a recent browser, and many Android devices released in 2013 will not run Chrome or Firefox, either, since they seem to require ARMv7 (ZTE Blade and Skate were ARMv6).

Other apps follow a similar pattern.

So based on track record of Android progression and compatibility, the chances of the majority of mainstream apps made in the next 5-7 years being able to run on _any_ Android device sold today don't look that great. The apps' backward compatibility with older versions of Android has never been all that great.

Re: iptables is your friend

You're missing the point - if the SMTP traffic is coming from one server (or a small number of servers), that server's connectivity would get limited at TCP level. Connections over the 3 new connections/minute limit would get dropped at TCP level, which means they would never deliver SMTP payload. Processing SMTP payload is expensive, while dropping TCP connections is nearly free. So if he were to drop most of the offending traffic before it ever got past TCP layer to SMTP layer, the server wouldn't get overloaded.

Re: iptables is your friend

The TTL consideration is to stop a malicious party from performing the DoS where they try to fool the throttling into thinking that an IP has exceeded it's new connections limit. In classing terminology terms:

Alice periodically sends email to Bob. Bob throttles new connections from everyone to 3 per minute. Trudy spoofs Alice's source address and starts issuing new connections to Bob. Trudy doesn't care that the 3-way handshake never completes, beause all she is trying to do is fool Bob's throttling system into thinking Alice has opened too many connections in the given time period. Thus, if Alice then tries to send email to Bob, Bob's throttling system will deny Alice's legitimate connection. If we add the TTL check, and if Trudy's discance from Bob is different from Alice's (pretty decent chance), the Bob's server will not treat this as the same connection (it sees a connection source as (IP,TTL) tuple, rather than just IP), and this will allow Alice's connection despite Trudy's DoS attempt.

Of course Trudy can try to figure out what Alice's TTL to Bob is and try to attack it harder, but that's an extra hurdle to cross and we are not really looking at fighting an escalating arms race, just provide a good enough first-pass solution that covers the basic defence.

The iptables throttling fixes the issue by making no server able to open more than 3 (or some arbitrary number, up to the OP to decide what they want to set it to, but 3 is a probably a good start for a personal mail server where very high bandwidth miling lists aren't used) TCP connections to port 25. If it wants to send more than that, it'll have to time out, wait and retry later. Since MTA won't see any subsequent connection attempts from the source server in that minute, it won't be chewing through all the resources it is chewing through at the moment (because it doesn't have to accept the connection or receive the payload to analyze it). Therefore the little SheevaPlug won't be DoSed and will continue working as it's owner requires. The only caveat is that the occassional email from a relatively high bandwidth mailing list might get delayed once in a while.

Re: Wouldn't a vacuum be better?

No, it wouldn't. Vacuum is not thermally conductive for convection, and the platter surfaces are cooled by the gas (air, helium, whatever) inside the drive. Helium is more thermally conductive than air in addition to being less dense, hence less drag and better cooling allow more densely packed platters.

What really surprises me is the 5 year warranty on the model. This is normally reserved for the expensive enterprise grade drives. Does that mean the He6 will be substantially more expensive per TB than the air-filled 4TB model? Or is Hitachi offering the extra long warranty as a sweetener to help customers swallow the technology that hasn't been tested for long-term reliability yet?

Either way, the size is just getting silly. The RAID rebuild times with drives like this are inevitably going to be waaaaay outside anything sensible.