The real problem here, of course, is that there is just no way to measure what makes for "better" clerks. What makes for a good clerk is that her boss likes her. And assuming that they aren't drinking and cite-checking at the same time, most clerks thus do just fine. But evidently some older male judges still feel uncomfortable with young women. Or their wives do. Some just prefer to hang out with those who share their political or ideological views. Many judges have quirky personal preferences (for tennis players or history buffs). Others just don't think there's any benefit to diversity in their chambers. And so, not unlike the guys at the Sigma Chi house, some judges just go with their instinct, and in some cases that means their choices are overwhelmingly male. That has little to do with personal politics, by the way. If Clarence Thomas has one of the court's best records for diverse hiring, liberal icon William Brennan had one of the worst.
As usual, Dahlia's got it right. As a former (non-SCOTUS) clerk, I know that there are loads of qualified candidates for every spot, so a judge has the luxury of making a hire based on a hunch that the candidate will be a good "fit," a person the judge will feel comfortable with. So why don't some of the justices feel comfortable working with women (or other people who aren't white men)?