DIY ad hoc storage is not for production use...period
Well, it is certainly fun to rummage through the server dust bin and cobble together a storage cluster based on open source software, but who would seriously consider using it as a production storage tier in their organization?
You can get a very low TCA if you build your own white box storage servers or turn to ODMs like QCT and Supermidcro. Google started out building all of its servers as cheaply as possible by doing it themselves. There is really no reason to start with junk servers unless you need to prove the concept before you get the funding you need to do it right.
DIY open source storage software like Ceph is not a walk in the park if you don't have a computer science department nearby. Ceph has a complex, non-P2P architecture with no built-in capability to do charge-back, QoS or reporting...stuff that people are interested in having. Upgrading Ceph has also been difficult. Maybe Red Hat has made some progress along these lines with the release of Red Hat Ceph Storage 2 this past June.
I do agree the determining the TCA for a capacity storage project is relatively easy and coming up with a fully burdened TCO is more difficult, but not impossible. Personally, a single storage administrator should be able to manage 10PB of objecgt-based storage, assuming the cluster is not made of junk servers needing constant attention.