Women notice artificial fibres on men, and avoid it. Heck, somehow some women can tell the difference between cashmere and normal wool just by looking at it
The ability to spot a decent bit of cloth comes quite naturally to those of us brought up in a textile manufacturing environment.
I can unerringly pick out the most expensive suits from a rack without looking at the price tags. Thankfully suits are no longer mandatory at work.
"In any case, as CrazyOldCatMan observes, natural fabrics are more comfortable and breathable."
Better for outdoor activities too. It's not a coincidence that farmers in moorland areas wear woollens.