Yes, smaller cogs / chainrings are less efficient than larger ones for the same gear ratio. The smaller radius requires the chain to fit around a tighter curve, which drives up the friction at every interface between the various parts of the chain. And there's a lot of interfaces, because chains are made up of a lot of very small parts.
But roller chains are really, really efficient. (If I'm remembering right from when I was in engineering school 20 years ago, they're one of, if not the most efficient ways to transfer mechanical energy across a linear gap.) So the actual difference in loss wouldn't likely amount to much.
The difference in wear rate, however, is not subtle. If we assume that the wear per revolution on any given tooth is the same, then tooth wear is a function of the number of teeth. So, if one of my main cruising gears is a 42x18, which it is, then I could get the same gear ratio with a 21x9. (Leave aside the fact that 21-tooth chainrings aren't a thing.) But the 21x9 would wear out twice as fast.
--Shannon