Yeah, I think fear (for various reasons) is a factor.
I watched a guy, hands literally literally shaking as he was lined-up on the boards before a basic warmup paceline. He had completed the beginners course and the track only had maybe 7 people on it, but adrenaline nonetheless.
I'm not fearless, but I do have a "If a kid can do it, so can I..." attitude about a lot of things.
Maybe quelling fears should be a point to stress. I mean, track racing is, mile-for-mile, safer than many other genres of cycling. Think about how many incidents happen in a local crit day vs a local track day [knock wood]. Same for MTB, BMX, CX, etc... Maybe road TTs are safer mainly because they are forbidden to ride close to each other.