40 psi is already pretty low. What is the recommended pressure listed on the sidewall? Tire size?
I would be cautious of going below 35.
I'm interested too, as I would like to start utilizing dirt and gravel roads for touring myself.
Washboard is one of those frustrating things you'll have to put up with when using gravel roads. The less used roads will have less of it. Look closely to see which side of the road has less - use both sides of the road to minimize the frustration when no cars are around.