Actually not wearing a helmet does affect all other riders. Don't wear a helmet and get in an accident that would have been minor with a helmet, but without one gives you a decently major concussion that comes to costing a few hundred thousand dollars in medical bills. Guess what? Those medical bills get passed on to every other rider through raised premiums. Another thing that comes along is alot of rather uneducated people will see the unhelmeted rider and thing "gee, he's dumb for wearing basically nothing" or will see them crashed and horridly injured from a truthfully minor crash that a well dressed rider would have gotten out of with a few bruises and think "Motorcycling is too dangerous, it should be outlawed." People remember the bad better than they remember the good, rarely will anyone remember the guy who has the jacket, pants, helmet, gloves, boots.
Seatbelt laws were made to help prevent unnecessarily high medical bills, helmet laws are the same way.