To be honest flying on the trim isn't a major issue in terms of the airframe, however it doesn't allow accurate flying. Unless you're the autopilot which in some cases does it all on trim.
Some helicopters, e.g. Lynx, Wildcat, are flown almost entirely on the trim.