I think some way of letting people move down (rolling window of points in the last X number of races moves you down if you do poorly enough, points expire after a certain amount of time but only drop you to a floor set by your "FTP" or other power metric they calculate, etc), is key. Then, don't let racers pick their category but auto-assign them (or let them pick if they want to race up a category) but not down (they know how to do this because I can't enter women-only events). Sure, there'll still be people who figure out ways to game the system, but, if your goal in sandbagging is to win and winning moves you up, you've got to make a choice. I know some crafty guys who consistently get results way above their power (and others with lots of power who underperform). Rewarding guys who race strategically seems the right approach and should make races better in the long run.
There was a time when all the other stuff of travelling and waiting and that shared experience was a big part of things for me. Now that I'm older with kids and their commitments, I'm happy if I can just get up at 5:30 on a Saturday, get into a race by 6:30, and be done to eat breakfast with the kids by 8 so I can get on with my chores for the day. That'll especially be true if their sports ever start back up again.