You're overthinking it. Sounds like you are feeling guilty when you DON'T ride.
If you wake up on a beautiful Sat morning and don't feel like making it to the meet point and making up reasons not to go, then don't go. If the morning of you commute you are indecisive on bike or car, take the car.
However, if you feel like I've got to get to bed early and need to get up to make the ride, then go.
If those days you decide not to go end up stringing up to a few weeks, months, years, then so be it.
I've gone through times where I hardly ride and use that one cloud in the sky as an excuse not to take the girl out. I've take a season off. My passion came back the following year. probably stronger. But my mentality changed. It wasn't just to rack up miles and speed, but it was to meet friends, wander mindlessly to detox my brain, go down half a mile to get some milk, slab it 3 hours to visit a friend for 30 minutes. purpose changed, reasons changed, but passion was same...yet different.
Ride your own ride.