A month ago I posted this thread in MTBR...
https://www.mtbr.com/threads/proteck-your-neck.1196586/
The EWS study linked there found that, aside from helmets which are a given...
knee and elbow injuries are common but easy to armor
Shoulder and hand injuries are common, take longer to heal, and hard to armor effectively
Neck, back, and pelvis injuries are uncommon but drastic.
There are tons of options for pads. You can get full chest protectors, backpacks with spine protectors, neck rolls for FF helmets. Like you I was kind of trying to figure out what's really worthwhile. A few weeks after that post I broke my thumb in a very stupid commuting crash (stopped to take a photo and lowsided) and some armored moto-style gloves would have prevented the damage. But what were the odds? Well, probably pretty good. It's not the first time I've broken a finger, though it's been a long time.