I agree. Ever since I worked in the UK IT industry 20 years ago the HRMC assumption is that all sole-trader contractors are only doing it to avoid tax.
However they are doing the same sorts of work as the bigger IT outsourcing operators and consultancies who never get questioned about their IR35 tax status. It is massive hypocrisy.
So either do HRMC staff either have no idea of what IT contractors actually do (probably) or are they just going after easy targets?