Trying out as many as you can and finding out for yourself what you prefer, is definitely the way to go. However, the intended purpose of the firearm is important too. For instance my Ruger LCP is fantastic for comfortable concealment if you plan on doing that. There are many ways to easily hide it whether you IWB, pocket-carry, whatever.
I wouldn't want to use the LCP as a home defense handgun. I'd want something with higher capacity, and in pretty much any other larger caliber. Nor is it a firearm I want to shoot often for fun shooting. I put a few mags through it at the range to stay familiar, then it goes back in the bag.
Kyle's list is a great start.