Tim.. If I had to guess, your speculation is correct about the turbo oil seal. My GTI with a KO3s is starting to do the same and it's simply from over-boost and mods. They're such a small turbo that they generate an enormous amounts of heat and they tend to have a short life after upping the boost so much. Keep in mind they were intended on running 8-10psi from factory; while after being chipped I'm spiking 23 and holding 19-20psi.
What happens to me personally, is that after idling for say 5-10 minutes I will get the obvious blue smoke which indicates burning oil, not to mention the putrid smell. If I drive the car it goes away and no smoke under WOT conditions and no smoke at start up. You can do as kenny says and check the exhaust side of the turbo for any obvious signs of leakage.
I don't think there is a way to prevent this, post stressful mods. Many on vortex are aware of this problem as well, and although they have installed turbo timers and etc, it doesn't seem to be of much help. I always try to give my car some extra idle time to cool down if I take a hard, short trip..