For me it happens too if I ride not much for a while. What seems to be my problem is not being able to do a pelvic tilt. That means my back has to bend more than is comfortable.
Exercising before a ride, just doing a pelvic tilt and getting it as low as possible, could help you sit better on your bike.
For me, often after half an hour riding, I can drop into the drops, but the first half hour my pelvic tilt doesn't work right and riding in the drops is uncomfortable.