You said you got with a fitter and decided to wait and see. If you change things up, how is that waiting and seeing? You also said the fitter thought you might want to be more upright, but you seem to be going against that and wanting to go lower.
I'm confused.
What is the actual problem you are having on the gravel bike that you think is fit related?