I find that for big races (read large purse, championship race, or a race with a lot of history), officials will pull lapped riders. Officials tend to show some "leniency" at smaller races, assuming the course is reasonably wide and not too technical.
As far as your fellow racers go, if you are lapped and you hang at the back you won't get any flack. If you start mixing in with the field, other may yell at you -- assuming they noticed you got lapped.
My experience with lapped riders is that they generally got lapped for a reason and if they move to the middle of the pack, they will probably open a gap. I've yelled at people for that reason. If they are at the back and I end up behind them then that's my problem.