If I plunked down in a spot that was perhaps a bit too close to an existing and already established group or individual, I would maybe move on, especially if I wasn’t given any signs of acceptance. Hard to say though, if there were other sites nearby I would move. If sites are far and few between, the other group should be aware of that and be accepting that they need to share a site. The sites are all public land ?, thus everybody is welcome but first come, first serve. The Appalachian Trail might be an example where the available sites that are near a water source means that others may be camping near you, get over it. I would not be popping down a tent in a site already occupied just because there’s room for one more tent.
As to to the built shelter ?, you own nothing. There’s nothing to say to somebody using a shelter you set up a day before and came back to. You move on as it’s public land and as stated, you own nothing,