I'd just get the fatbike. There doesn't seem to be any other type of bike that people police as heavily as a fatbike, it's really quite remarkable just how many rules people try to impose on those riding them.
"Look man, I don't mean to burst your bubble or anything, but you know that if you ride that fatbike on anything aside from snow or sand, you may create a paradox that will result in the heat death of the universe, right? I mean, you wouldn't want to destroy everything we know and hold dear by taking that bike on summertime singletrack, or, God forbid, pavement of any type, would you?"
I'd just not worry about what other people tell you, I'd get one and ride it like mad. They look like fun.