Will a stiff, aero aluminum or carbon frame be faster than a lugged steel bike? Absolutely.
Will the extent to which it's faster make a difference to you as a Cat 4? Probably not.
Will building and riding the kind of bike that makes you love riding, love racing, and love lookin' at it matter to your enjoyment of the sport? Absolutely.
Build up something beautiful. Ride it, race it, have fun. I've known people who've raced lugged steel at Nats (recently!). It's fine for you. If, down the line, you decide you want something performance-oriented, then buy a cheap, frank, no-nonsense alu race frame, move your parts over, and hang your pretty frame on the wall. But for now - do what you want to do, and don't worry about it.