I tend to agree with your unexpressed concern - yes, it's not fair to 'punish' big and diverse group of people for words and actions of their head honchos. Too much collateral damage. Not to mention that collective responsibility is never just, it's a leftover from tribal societies.
If some specific individuals leading the crusade against constitutional rights would see their dirty laundry out in the open...that'd be much more fitting.
OTOH, those big honchos don't seem to have any qualms about invading the life of millions to catch few perpetrators. Where's the fairness in that.