I prefer to be the ride leader, but will only do so in certain situations. There's a group of people that I've ridden with for years, all of us a capable of being the lead, sweep, or middle of the pack. There's no egos, and we all know what to expect...we're all riding our pace, which is pretty similar for each rider.
I rather ride with unfamiliar riders when there's just 2 or maybe 3 of us, not in a bigger group...and I watch closely. I've met some outstanding riders that way, and some others I would NOT invite to a group ride, until their skills & maturity are more developed. I've also ridden with some skilled riders that AREN'T a particularly good fit to have on a group ride.
When I'm leading, I tend to watch my mirrors a lot. I'm actively trying to keep the group from getting split too far apart, and making sure no one is putting them self or another rider in jeopardy. I won't ride with people who's ego exceeds their talent, or think a group ride a pissing contest.
My level of comfort with other riders has nothing to do with pace. It has everything to do with skill, smoothness, and being level headed. I've dipped out of lots of rides when it just didn't feel right.