There is a harsh reality to affirmative action and diversity initiatives: They sometimes make people of color and women uncomfortable. While many of us support affirmative action, it really is a double-edged sword.
This sword was in full view in the recent comments of Geraldine Ferraro. Well known as the first woman to run on a major party's Presidential ticket, she has also frequently and honestly said that she knows she was picked for the vice-president spot mostly because she is a woman. That doesn't mean that she wasn't qualified to be VP, but the Democratic party and Walter Mondale looked around and decided (most likely due to a poll) that they needed a woman for the ticket.
The backlash against affirmative action is such that knowing you are chosen "just because" you are a woman or a person of color can be debilitating. You start to think others "know" why you got the job. You fail to remember your long list of qualifications and focus on your plumbing or your ethnicity. This is what anti-affirmative action forces want. This distracts you from noticing that someone's frat brother was hired or that, despite other worthy candidates, a Texan is chosen to balance a Presidential ticket. Sure he's qualified, but he will also deliver a wad of electoral votes.
Yes, I will admit it can be fun, sometimes liberating, to be the only girl on a football field. It feeds your need to be seen as not girly. You know people are watching. But it can be this glare that can break you. Ask Shannon Faulkner, the first woman to enter the Citadel. Try being the only black woman engineer at a conference. Yes, recruiters will certainly notice you, but everything you say will be scruitinzed. Far too often, a misstep will be attributed to your skin color or gender and not your lack of experience. For every Sally Ride there is a Lisa Nowak.