What about corporation tax which was paid?
Not sure of the details of this case, but back when I did contracting (stopped doing so at the start of 2000 partly because of IR35) I took a nominal salary to cover for situations when I had no paid work and then took the balance of the income in the form of dividends. However, the company had to pay corporation tax on the profits (from which I took the dividends) though I could use that to offset some of the income tax I paid on the dividends I received. If it turns out that the money was ALL salary and liable for income tax and NI then would HMRC repay the corporation tax?
What I remember of my contracting days, you could reduce your NI burden by taking a smaller salary and then dividends for the rest, but the total of corporation+income tax paid ended up the same as what you would pay for income tax if all the income was salary. With NI was that you were liable for both the employers and employees payments out of the money you made from working, so the actual amount of NI which I ended up paying wasn't much different to what I had contributed when an actual employee (though I know that I am not taking account of what my employer would have paid when I was an employee).
I understand the motivation behind IR35 and that I was taking advantage of a loophole which existed at the time, but IMHO the amount the government made from me when contracting (due to the rate I was being paid being higher) was significantly more despite the loophole!