As an aside I (and doubtless many others) would be delighted (and probably amused) if you were to tell us just how much " fibre to every address" would cost.
Rough estimate, you could probably do it for around £20bn, roughly the same as Crossrail. Even if it cost double that, it would be less than the cost of HS2.
Cost depends a lot on how you do it; with enough political will it would become cheaper to do, eg legislating simple land access for telegraph poles rather than negotiating with each land owner.
I'd much rather that we invested in modern future proof network capability that benefit every household in the UK, and allowing more remote working rather than making the trains a bit better*. High speed, low latency, symmetrical internet connections transforms what the internet can be used for, and would place Britain at the forefront of the modern world. We have cheap government borrowing at the moment, we should use it.
* Crossrail is actually tremendously important, as London is pretty full and needs it. HS2 is a proper vanity project though, the problems with congestion on the railways outside of London can be fixed without the fapfest that is HS2.