I have two bikes with 650b. One drop bar with 47mm WTB byways, and a flat bar with gravel kings, 650x48. These are pretty much what I ride all the time, and I have some other good bikes - carbon racer, steel drop bar, dual sus MTB. They're all collecting dust. Because like they say.. Once you go Phat, you'll never go back.
The only caveat - 650 may change your handling and bb height a bit. On my road bike the handling got a bit quicker but I love the cushy ride of the 47mm wtb's which I just set up tubeless. And I've got to be a little careful to watch out for pedal strike on my flat bar. But, do it a couple of times and you'll remember not to pedal and lean at the same time. Not a big deal. And none of this will be an issue if you get a bike specifically designed for 650.
And.. if you go with a decent wheel set your bike does not have to weigh a ton. And if it does, so what.