With sloping top tubes, stand over is less of an issue for finding a proper fitting bike. Women typically have smaller hands and brake reach and drop bar curl might also be problematic. Shops will often swap the stem at no charge, but no one wants to purchase a bike and then have to replace a lot of the parts. I suspect that these are more of a problem for people who fit the very small frame sizes, like 48s.
I am so tired of the mostly women's colors when trying to find cycling clothing to purchase. I don't want pink or aqua and I do not want flower patterns on my clothing. Often there is no high viz green/yellow in women's jerseys. While i don't see non-women specific bikes being much of a problem, it matters a lot with it comes to clothing.