If you do physical work all day, then at the end of the day, you just want to sit in an easy chair.
If you sit at a desk all day, then at the end of the day, you want to get out and do something.
I don't know that there's a good cure for it.
One thing that is not obvious is that "getting tired" is not the same thing as "keeping fit". You can be on your feet all day and be tired at the end of the day with very little exercise. Or you can work certain muscular groupls without really getting your heart rate up.