It's not gender or colour
It's simply toxic overall.
SO reached critical mass and the arseholes took over. It's the most probable result of any large grouping - the people yelling insults drive away everyone else, and then anybody joining later on finds a toxic group, and leaves.
It needs strong moderators early on to prevent that, which SO did not have.
SO used to be a useful place to ask and answer questions, and now it isn't because you will be shot down for doing either.
My most popular answers now have multiple "You're an idiot, RTFM" comments. Yep. I wrote that bit of manual after answering the question, so future people don't have to suffer SO.
So yes, any women etc will rightly assume the place is toxic towards then. Because it is.