Do you stretch before/after? How tight are your hamstrings?
I had fairly severe back pain on my longer rides for many years. I had a good chiropractor (or so I thought at the time), who would somewhat help, but that would never really go away. The more I rode either mountain or road, the more back pain I had. I always attributed it to a herniated disc I had.
That was until I visited a physiatrist, who pointed out that my hamstrings are extremely tight and that causes most of my back pain problems. Took me several PT sessions to learn proper stretches and several months of daily stretching and while I still have my herniated disc problems, that does not bother me on a road bike anymore (technical/downhill mountain biking is a different story though).