The late Mark Whitehead was at a party where I was about 25 years ago. At that time he was training some big names / was one of the big names. Being a cyclist/tri kinda party, hosted by a great vegan cook the subject came up. His response stuck with me.
He said something like "I would never train a vegan". Five to ten min of arguments similar to what is in this thread started before someone asked him why.
He then said something to the effect of - He didn't want to train any athlete who who looked at food other than how to maximize their performance.
That was the core there. Not that it couldn't work fine, better, worse, just that no point in training someone that limits what is legally available.