If you don't ride down twisty trails at 50 mph or otherwise near the edge of performance the tires are designed for, then probably not a big deal. If for some reason they wear out sooner, well, then you'll know. Or at least have some anecdotal experience to support believing one way or the other.
If a tire has a rotational arrow, I mount it as the arrow indicates. If I don't see it and later notice the tire is on backwards, then I change it. Not so much that I'm worried about safety and performance concerns it might cause, but just because it bugs me knowing that arrow is pointing the wrong way.