I rode by someone on a MUP about 10mph faster than they were walking. Then went back on the correct side and passed another walker coming the other way. All seemed completely safe and within the normal. They may be in their walking forum or HOA meeting discussing how these crazy people on bicycles are flying around on their neighborhood MUP and it is dangerous. Imagine the horror if someone on a bike loses control or isn't paying attention hits someone walking!!!
Obviously there are limits. Not everyone will agree on what they are. You are in a bicycle forum that probably most people ride self powered bikes. You'll get a general opinion that reflects that fact.