I would have tried a little harder to get it to go away. Any sort of barrier will help when approaching an animal you may think is harmful to you. A broom, hockey stick, piece of 2x4 or even a sheet of plywood can help you shoo off any animal.
If the animal is either 1 charges you and shows aggression or 2 doesn't seem the least bit afraid of you then its time to do something else. Calling pest control would be number one, only in an immediate situation should you kill the animal.
I'm all for hunting and all for pest control, a raccoon trying to find a meal isn't a pest.