If you are riding 40mph or 50mph down hills in your group ride with only half a brake.
I wouldn't be surprised if a bad rear wheel wobble could induce a full bicycle shimmy.... and send you into the bushes again. Potentially also putting other people at risk.
You could also break spokes, making your wobble worse. It may depend on rider weight and characteristics. You could also damage your tires/tubes.