We're not supposed to pass in an intersection, except for separate lanes of course. Debatable whether it applies asan " intersection" where there isn't through cross traffic.
Bad move on his part, but within his rights. If we leave room enough for a pass on the left, we take the blame when someone passes. Even if we signal left, we both get some blame.