Today I went to the shop to return the bike. I went early enough to ride the Palm Beach Lake trail, which is very lovely and flat. The shop had a large Verve and I swapped for that bike so I rode about 10 miles with it. The Large bike was much better for me with no hand numbness experienced. I was in a much more upright position with a larger % of my weight on the seat, so overall a much more comfortable ride. I agree the problem is my lack of core strength, but that is why I am riding to start with.
I hope the large FX4 is similar enough to the Verne to work as well, but at least now I know what comfort should feel like.