I'd give a taller stem a try. But if you do decide to try out flat bar bikes, there are a lot of great vintage mountain bikes that can be nicely repurposed to ride on trails and gravel with smooth tires. Although I mainly ride road bikes, I'm a big fan of vintage MTBs.