I'm roughly in the same situation as you. Older, and I ride almost every day, sometimes in fast groups on the road. Advice:
- Bicycling is inherently dangerous. Some situations are more dangerous than others, such as riding in the middle of an aggressive group, or doing warp-speed descents in swirly winds etc. Fear is good; that and experience is what keeps you alive.
- Cross and gravel bikes have higher bottom brackets than pure road race bikes - for ground clearance. For this reason they are LESS stable.
- Higher handlebars put less body weight on the front of the bike, which is also less stable. I've only ever ridden one bike in which the bars where at a higher position than my saddle. On a high-speed descent, it was terrifying.
- Thick section rims are more aero, but catch crosswinds. Plus they are heavy. If you are doing high-speed time trials, then you maybe need 50mm rims, otherwise they are heavy and less stable.
- If you ever hope to hang with the 'fast crowd' on group rides, you'll want a (very light) road race bike, so no Checkpoints, or gravel stuff, fat tires or steel frames or discs.