If you are riding with others that are good cyclist already and have the correct type bike for the job, then you are severely disadvantaged and are probably just running out of glycogen (energy) trying to keep up with them.
In time you'll learn how to pace yourself, but not if you are trying to keep up with otherwise half decent cyclists. Once you figure out what pace will let you ride all day, and for how long you can go all out then slow back to that all day pace, then you'll be able to work on getting faster to keep up with others.
Mostly it's time and mileage behind you at that point. But there are methods that will help you get to that ability quicker.