I agree with the idea of just trying out a weekend trip to see if you like doing this. If the tires on your bike are in good shape you will be ok.
I find a 26x1.5inch tire to be a good all around width for riding on all kinds of surfaces, and any reasonable tire will work well for predominantly paved road and dirt riding.
there are many good tires at a reasonable cost for your needs, and they will last for years, so perhaps see if you actually like bike touring----plus you may have to buy a rear rack, tent, camp mat, sleeping bag etc.....so hold off on new tires if three existing ones are new.
if you really want new tires, then people can give good suggestions.
And yes, 25-40 dollar range generally will mean a good tire.
do you have the stuff to try touring, panniers?