All depends on the itch you want to scratch. The feeling of it being more risky may have been due to the people in your group. If the only day you did was a Novice track day, you need to give that another shot because Novice track days are a crap shoot more often than not. Huge difference in riding abilities.
There's hardly anything better than rolling out of hot-pit with your buddies and chasing them around with your knees on the ground. I still get the butterflies when I'm taking my warmers off for a session.
For @NinjaDoc, I think track riding is a great option.
You can find a cheap track bike for cheap that will keep you more than entertained riding the hell out of it on the track. You wont feel bad about that sitting in your garage while you aren't riding it because it was cheap and you don't have to put insurance on it.
Since the bike is cheap, you wont care when you drop it.
You wont have a bike payment, or an insurance payment to pay when you aren't riding.
The environment you ride in is safer than street riding although some in this thread may argue this.
You can push yourself harder than you ever did on the street and feel better about doing it because of points 2 and 3.
IF there were something to happen, medical professionals are closer than they ever would be on a SEO ride.
Having a dedicated track bike as the only 2 wheeled vehicle in your garage may re-ignite your passion for riding because it's something new, you cant do it whenever you want so your riding is even more carefully planned than it is now, and there are completely different challenges.
The only thing with Nivin is, you need sleep damnit! Don't pull a 12+hr no-sleep shift and show up to the track the next morning! You need to be sharp and on your A game for your safety and the safety of others.