Sounds like a supermoto would work for you just fine. If your goal is to get to the trails and then take off riding them on your motorcycle, a dual-sport is the way to go. Supermoto can handle the dry pack with ease. Just know you aren't going to make it very far on street tires if its muddy.
I wasnt even talking about "speed" on the smaller displacement bikes. Wind will blow you around, engine wound out at highway speeds, vibrate your brains out, etc. If you are on a track or in the twisties, those little bikes are ideal. If you need to spend an hour on the road @ 60mph to get to the twisties, you might want something with a little more size to it.
Then again, if this is strictly going to be your commuter, an sv650 or KTM (http://www.990superduke.com/) might be the better choice.