Part of it could be a fitment issue. As for you not enjoying being out there, I'm not sure how to respond to that, as I have been cycling most of my life (from 3 years old). Not because of "fitness", but because I enjoy it. Maybe cycling just isn't for you?
But before you give up, go to someone who knows about bike fit (-ment). You probably have at least your saddle too low and too far forward or back, maybe your handlebars are too low/high, too far forward, or whatever. It's difficult to say much of anything without actually seeing you on the bike, and hopefully also to check if something is wrong with the wheel hubs or whatever.
Edit: Elbows hurting can either be because your handlebars are too straight, that they're too wide, that you cycle with straight arms and "locking them". Or all of the above.