You've probably gained more fitness than you realize by doing what you are doing. But if you have settled into a routine level of effort, you'll have to step outside that comfort zone to see any real improvement. Pick up the pace at times. Hammer up a few hills. Make yourself sweat like you did when you first started riding.
It all depends on what you want to do. There's a lot to be said for being able to do what you are doing now.
