I think it's time to revisit this thread with my latest frame fit... So I switched back to 46cm bars with a normal drop and slammed a 110mm stem as seen in this photo (5mm longer reach than stock coming from these new bars..)
It felt great at first, but suddenly I felt like my saddle was about 1.5cm too low. That it when I crossed the line into too much handlebar drop. I will take 1.5cm of spacers and put them underneath the stem to compensate.
It's been a very long journey, with never ending changes to my fit. It doesn't end here.
With the handlebars 1.5cm higher, and the saddle 0.5cm higher, that would be my baseline fit as of right now. I can now ride up hills I used to walk, and ride for much longer than before without any back pain whatsoever!