I think your basic problem is thinking that there is such a thing as "wasting" time. Life is for experience and any experience is valid. In the proper context sitting and staring at the wall is a valid experience. The trick is balance. . . . not to let one single experience happen more than necessary to do the "experience" job.
It sounds like in the context of your life and needs you did exactly the right thing.