I almost always unclip my right foot and lean right--away from traffic. However, thanks to Mr. Newton, unclipping on the right creates a force to the left---into traffic. Normally I unconsciously lean to compensate. There have been a few occasions where I came Very close to falling left---but it has been many years since I did a clip-fall. However, it could have happened at any of those close calls .... or at any time hereafter. Mostly it happens in situations like you described---other things going on, a sudden or unusually forceful unclip, a startling incident ..... Stuff happens.
You are fine. Just ride on.