So it's doing this while the handlebars stay straight ahead? If so, then I'd check the clamps at the base of the handlebars to ensure they are tight. Since I dont own one or have one available for closer inspection, I have to assume this is your problem.