Do you mean something like
this? You might just be better off removing those completely, not practical at all for daily riding. A finger behind the seat tube pushing back on the wheel should do the trick to get things lined up for proper tension. Your chains tension doesn't need to be tight, in fact a little slack is better to keep things running smoothly as long as the chain won't lift off the teeth of the cog or chain ring.