Whether people are saying it or not, you'll get some resentment from some of the B's out there, especially if there's someone out thats never podiumed and was pushed off by 'sandbaggers'. We've had races out here in SCNCA where whole podiums in the 4's had points to move up to 3. While its within your right until your kicked out of the category, it can be construed as poor taste (if you haven't won and have the points stick around).
You can always work for someone as well in the B's, but being a strong rider that's fairly strong that may be easier said than done.
I don't know the culture or attitudes of your race; maybe everyone is okay with it. Typically though in a tiered race system like that it is assumed you move up when you dominate that ranking, and if racers can predict the podium before the bell even starts, some aren't going to bother lining up.