Your main problem I believe is the front derailleur. The Shimano FD-5504 was designed for
- 9 speed drive trains (should work fine with 10 speed because spacing and width of chainrings [not cogs] are essentially the same - sure others will say that you will die if you don't adhere strictly to spec)
- Shimano road shift levers (wont work with MTB shifters)
- 52-42-30 front chain ring set. (road not MTB sizing)
Looking at the inner cage plate, you will notice that it is very large (about twice the height of a FD designed for a double) and is shaped, not flat. This shaping is designed to work with indexed shifters and helps lift the chain to the next ring. If you pair it with "non-spec" sized chainrings, this shaping can actually degrade the shifting. Also, because you are using smaller rings than designed, the FD has to be set higher than normal; often so the "bottom" of the cage misses the chainstay, I recommend that if you are going to use "MTB sized" chainrings, then it is better to use a front derailleur designed for a double crankset. Jan Heinne does a better job explaining this; see
https://www.renehersecycles.com/trou...h-sti-triples/
I had this very problem with an Ultegra triple FD-6503 shifting non-spec sized triple chanrings. Changed to the double Ultegra FD-6500 and it works much better.