Down tube shifters - they are kinda like shoelaces. Takes longer, you've gotta sit down to tie them, they take a little skill. But it can be learned by just about anybody and become so natural that you do not notice it is somehow "wrong". Yes, riding with others, you see that you ride a little differently. You naturally expand the range of acceptable pedal RPM instead of shifting to stay exactly at the "magic" RPM. You plan out hills.
There are perks. Your bike will get a tiny bit lighter and more aero. Simpler, less to go wrong. Cleaner "cockpit" area (as you have already noted). Riding with others, you get to silently smirk when shifters give others issues. Yours never get damaged in crashes or when the bike falls over. You can change wheels, cassettes, freewheels, numbers of cogs, spacing on cogs and the shifter simply doesn't care. Adjust the derailleur stops. Maybe tweak the cable length a touch but not usually needed. Done. You can use any rear wheel made between the 1960s and now. (There are dropout spacing issues, but your shifter doesn't care.)
We (tourists, sport riders and racers) rode with DT shifters for 50 years. Every Tour de France (and every other road race) was won on them. No one complained and there were few incidents. A few missed shifts here and there. Occasionally they needed to be tightened but if yours have the D-ring, it can be done while racing by the rider; he doesn't have to drop back to the team car.
Drawbacks? The biggest by far - you are now a dinosaur.
Ben - certified dinosaur; 2 1/2 DT bikes, 2 1/2 fix gears, a concept eons older (The Mooney can do both.)