I have a Blackhawk as well and it has no skin contact at all, do you have the right one for your gun?? And XD has a very good finish Mine is the new finish.
Size is really irrelevant if you are a good shot. I carry a .22 most the time in the summer cuz it's small and light goes good with shorts and a tshirt. Only time I carry the 40 in the summer is when Im dressed for a night out and have jeans and a belt.