My quick assessment: At certain high speeds, you've got wind directly in front of you and to the side. This clash of the winds may create turbulence causing instability in your steering.
Conclusion: You probably have more control at slower speeds.
Of course, I'm not sure at what speed you have to go, but if you can effectively go fast enough to create an airfoil around you sufficient enough to keep the crosswind from effecting you, then you'd be OK. I'd imagine that you'd have to be going very fast ---> faster than 50mph probably.
Disclaimer: There are those that think I'm full of of it, so take this with a large grain of salt.