I'll reiterate what others have posted: most likely a bent frame or fork.
A really subtle way to have this happen is for your fork to be bent in a way so that the front wheel is just a wee bit to one side or the other. Is the front wheel perfectly centered in relation to the head tube - and I mean PERFECTLY? If it is, take it off, flip it around 180 degrees, put it back on, and see if it's still centered.