As I understand it, the rational veggie/vegan choice is "I do not want any part in killing or hurting an animal" (with definition of hurt being very much up for debate). I actually don't mind so much: if you don't want animals to be killed because of you, fine. Go for it.
If there's a by product that would exist anyway, by using the fivers you are not in any way killing or hurting the animal, since no animals have been killed or hurt specifically to provide that material. In other words: no extra animals die as a result of fivers. Their consciences are therefore completely clear.
(Of course, then you get to the "none shall pass my lips" brigade who insist on different ovens/plates/forks. That's entirely different and completely irrational, because accidentally imbibing a trace amount of fat does not increase the number of animals killed or hurt for your lifestyles. "None shall pass my lips" is more of a religious point of view than a practical, logical, rational or ethical one).