Hey mate!
Good on ya for riding! Keep it up!
I am in the same boat as you. I used to play Rugby until I got hurt and couldn't run anymore and ended up keeping the Rugby diet without keeping the Rugby exercise so I went from a girlish 20 stone (280 lbs) to a fat bast*** 27 stone (380 lbs). Unlike you I am also ugly as sin and look like a brute so I am twice cursed.
Fortunately for me I can do something about the weight.
I also believe that spandex is a privilege and not a right and look like a German tourist in a Speedo (*DON'T* Google this! You will go blind! For those of you who have seen it... My condolences.) but the extra padding around "the taint" makes a world of difference on the saddle so my advice would be to do what I do which is to wear sweats or cargo shorts over the spandex. It hides the nasty and it makes you look less like a wanna be speed daemon.
Don't pay the tossers poking at you any mind. Just smile and wave at them. At the end of the day they don't matter. What matters is that you can loose weight but they can't change the fact that they are bitter unhappy little gobs who are so pitiful that the only way they can feel good about themselves is to put someone else down.