I'd suggest wearing your gear most, if not all, the time. It's understandable to want to be comfortable, but safety first. With that being said, I rode 270 in shorts/tshirt yesterday for a quick trip to pony. Sawmill to rt3. At 80-90mph, the wind sucks! I had helmet and gloves on but the tshirt was irritating me and the heat on my bare legs was not enjoyable. I won't do that too often. There are some sweet 'summer' jackets out there. i have one from joe rocket that offers the same air flow as if i'm not wearing a jacket...but keeps my tshirt in place and prevents the irritation.