Re: "Dumb" until Qualcomm are able to produce an 8 core chip
"How long does it take for App2 to fill the cache with its own code and data ? If it takes any significant fraction of the time between scheduler runs, the system design is wrong, surely? (Cache fills are too slow or the scheduler is running too frequently on compute-bound tasks or both)."
I think this hits the nail on the head - although a more in depth discussion with regards to the exact type of work-loads could take us a long time... For a SMP machine there is an overhead to keep the cache status in step, you're right, although it's overhead is not comparable to cache refill times in the cases i've seen and some theoretical banter (some of it a bit hand-wavy admittedly), that being the issue. Cache refresh is slow compared to cache retrieval in general, usually a couple of orders of magnitude, and has been for a while - that's just due to the comparitive speeds of the memory sub-systems (i'm assuming something similar to the on-die/module RAM L1/L2 and external DRAM model). Cache refill for a bunch of applications is dependant on lots of stuff, mainly amount, but for simplicity lets say they all have roughly similar requirements - which is essentially a fixed cost per switch. If you have to clear the cache when you context switch you pay that penalty, otherwise you're OK - and L2/L3 caches are going up in size in the consumer space (mainly Intel beefing up their L2) so that's good for a single processor. If you're getting a lot of cache usage per processs, quite common in a quite a few workloads, then the number of processes it takes to blow it out goes down and you start paying the penalty sooner. As the number of processes goes up the work to cache refill penalty ratio goes down in a uni-processor machine. For multi-processor situations, if the processes are altering a lot of global state then a lot of the CPU caches are being dirtied and you're taking some hit on reloading the global data - which may or may not be significant, but will slow you down. If the processes are not hitting global data then the individual caches aren't getting refilled (so much) and the over-head of switch is heading towards zero as you're not switching out (as much)... assuming a decent scheduler with half a clue about processor affinity that is, something that seems in short supply still...
..anyway - the point I was trying to make is that slicing on a single CPUs becomes more and more inefficient as the number of active processes goes up, the same is somewhat true about multi-processor setups, especially with processes altering data common to other processes, but degrades at a much slower rate - and the hardware book-keeping usually not a big overhead, especially when compared to the hit sustained when being forced out to external memory (DRAM).
The big killer for performance in multi-processor (not just SMP) environments we find now is disk IO - for which basic workstations with single SATA drives have the most obvious problems.. some workloads, like compilation, you just hit a wall in scaling really, really quickly (few CPUs, say 3-4 sometimes). A few years back, a Sun T2 with 64 virtual cores and half decent, but single, UW SCSI got up to a dozen times speed up before stopping... solid states drives now make all the difference in getting any decent speed-ups in build/test for the stuff we're running.
All in all, i'm not saying that single, chunky CPUs can't give useful multi-process performance but for a lot of work-loads - in particular when you have relatively 'independent' processes - a half decent multi-processor setup with a reasonably good scheduler will scale much, much better.
Biting the hand that feeds IT
