You need to ride more than once a week. Break it up to shorter distances as mentioned above. Your body will then begin adapting. You can't do one big ride to account for numerous shorter rides and get the same benefit. You also should make sure you are fueling yourself appropriately and getting all the nutrients your body needs. The reason you feel like you want to enjoy riding at your current effort is because you think that anything more than your current effort will result in the experience you described above. As you ride more and your body adapts (which it will do quickly based on current described activity), you will get faster and have the endurance to go longer without killing yourself.