Yeah man, you did nothing wrong IMO. I had almost the exact same thing happen to me a couple years ago. However, I explained the situation to the cop just as you did, and apparently he had a decent amount of common sense, because he let me off with a warning.
If I were you, I'd fight it.