I had something like this happen to me about 11 years ago now.
I was working as a self employed contractor with a number of clients (one of them being my accountant), paying my taxes, NI, etc.
One of my clients was a local council, and I worked for them 2 to 3 days a week depending on what work needed to be done. No benefits at all (holidays, sick pay etc) but the tax man deemed I was employed by them so they had to start taxing me at source.
In the meantime I was still being assessed as a self employed contractor and the income from that particular source was still included in the assessment. So essentially I was paying tax twice for the same income!
My accountant did appeal it for me (and being a client/friend did not charge me to do so) but they maintained they were in the right.
So it got to the point where I just had to stop working freelance as I just could not afford to carry on.
So took a 'proper' job in the public sector and have been here ever since!
Took me quite a few years to get myself financially stable again as well, the fuckers.