Turn it off?
Umm. It only needs well defined states to build a Digital Computer. Not actual "off". Though power consumption might be a problem.
An analogue computer is a Model of a system. It uses Analogue summing, log, Multiply, Square and Square Root circuits. Multiply by constant is an amplifier. Divide by constant is a pair of resistors. The number resolution is poor at between 1% to 0.1% full scale, thus roughly 7 bits to 10 bits resolution. The wiring decides the "model" or problem to be solved. Thus an analogue computer is not something for transistors that don't switch off so well, it's not a computer in the same sense at all as it has to be physically wired for a particular task. It's also inherently a "Data flow" processing device with parallelism and serial pipelines depending on the Process to be modelled.
There have been digital logic gates made with devices that don't fully turn on.
ECL
http://en.wikipedia.org/wiki/Emitter-coupled_logic
It's very power hungry, but was much faster than original TTL
A few late 1950s computers used it.
By the 1980s ECL type of design was mostly just used for Dividers from UHF or Microwave.