What is the cost of a benchmark
The CG benchmark is a sensible choice as it tests the key general requirements of "high bandwidth and low latency, and tend to access data using irregular patterns". These are machine characteristics which have wide importance --- not just CG. Another interesting point in Jack's talk at ISC was the total cost of performing a benchmark. I forget the exact figure (left as an exercise to the reader) but it was in the order of $1m for the top machines! This tends to focus the mind on performing a minimum set of benchmarks rather than everything and anything.