To all the downvoters of my comment above, do you really need to be told that a transistor is a digital switch?
No, I don't need to be told that. You think that a transistor is a digital switch. I know that it isn't. You can wire them into an arrangement where their gain is high enough that it makes no difference and the resulting system behaviour is very digital, but that is in circuit. In isolation there are no ifs or buts, a transistor is an analog device.
I did look into doing something similar to this a few years ago and it got much further than a thumbnail sketch. The goals were slightly different - this was a 38.4kHz 12-bitter with a few niceties (e.g. hardware multiply and divide) and a few oddities (hardware assisted garbage collection). It was a lot simpler than this, estimated at 3,500 transistors and perhaps 3ftx2ftx18" in size, but no integrated circuits anywhere - not even memory. Most of that reduction in complexity was down to the use of threshold logic gates which are a slightly quirky semi-analog system - digital inputs, digital outputs, but internally the processing is very analog in nature which allows for a much richer set of functions than pure Boolean logic. This approach was common for research systems in the 60s to reduce the complexity of the systems by exploiting that very analog nature of transistors.
Utmost respect for the guy though because I know precisely what is involved. My project didn't get further than design, a few test assemblies, and a software emulator and assembler before the transistor I had based it around (BF199) went out of production. They were less than 3p each in quantity and when I saw the cheapest through hole alternative was £1.50 the entire thing went on the back burner.