"All the rest of trading in already-issued shares, currency and commodity markets and related options, shorts etc, while they DO serve a useful purpose in hedging risk, can be and very frequently are used as speculative tools*."
James, you start with a presumption that speculation is in itself undesirable and does not add value to the economy. I disagree strongly.
Almost all economic transactions are speculative. A baker buying flour for his bakery speculates that the demand for bread will sustain the prices high enough for him to make a profit. Investor buying shares in a company speculates that its value will increase over the term of the investment (no matter if he plans to keep the shares for a few seconds or for 20 years).
The only non-speculative (in terms of price risk) transactions are those that reduce your risk (i.e. hedging or liquidation) or those that exploit inefficiencies in the market(s) (i.e. arbitrage). I disregard corruption here, as it is a special case.
Whether a transaction increases your risk (i.e. is speculative) or decreases it (i.e. is a hedge or a liquidation) cannot be determined from looking at that transaction alone. You have to examine the full risk position of the participants. In most cases one side to a transaction will be speculative, the other - a hedge or a liquidation.
The market views of end-buyers and producers will always tend to diverge and it is (one of) the roles of the speculators to bridge the bid/offer spread.
Now, high-frequency trading. I have not yet formed a definite opinion for myself whether its benefits outweigh the drawbacks but I tend to think that they do. The purpose of HFT is determined by its particular algorithm but most will be involved with arbitrage - i.e. exploiting small inefficiencies and imperfections in the market to make almost risk-free returns. There is nothing harmful in that. In fact, this is a vital function because "exploiting" here means "eliminating" these inefficiencies + they create liquidity for other participants.
But this is why HFT is high volume and fast - to take advantage of small imbalances, existing for a short period of time. If you want to speculate - you would hold your position over a longer period, taking risk and then waiting to be proven either right or wrong.
There is a possibility that HFT may try to manipulate the market by artificially distorting one market to momentarily create a spread to exploit but that normally should be impossible because the trade will work against itself (i.e. making buying more expensive and selling cheaper). This can only work, in theory, if you have a leveraged position, say, in OTC market - but OTC markets are not suitable for HFT, so, there...
I have not seen many problems caused by "run-away" HFT algorithms so far. In fact, markets seem to shrug off occasional glitch much quicker and easier than in the old times. The day-traders are the ones that seem to be affected the most by extremely short-term volatility as they are forced to either set the stops wider, which increases the risk, or tolerate them being broken more often, which kills their efficiency.