getting weight lower makes a huge difference to how a bike handles. Even having heavier items at the bottom of panniers, and also having more or less balanced panniers for weight, makes a diff in how a bike handles.
simply put, that rear basket high up like that is the worst thing you can do for your bike.
Borrow some panniers, and you'll see that with the same weight, the bike will be a lot more stable, and the "tipping point" that can come rather easily with a high up load like that, will be a lot less.
same with front panniers, lowrider racks work because of exactly the same principal as the rear.
now sure, granted, I don't know this bike, I know my 1990 tourer is more flexible than my newer bikes, but still, I and all the others here who toured back in the day on bikes like yours, generally had decently behaving bikes with panniers on them.