Try shoe glue
I had the same need and I used something advertised as waterproof shoe glue. It claims to bond pretty much every kind of material, both natural and synthetic. It's given me a good result gluing on a couple of flag patches. After the glue has set, the bond seems strong and flexible which is what I need since the pannier material is constantly being deformed.
I'm not sure what the technical name for this kind of glue would be in English. The glue comes out of the tube very sticky and goopy and yellowish. It has a strong solvent smell. You are supposed to apply to both surfaces and then hold under firm pressure.
This board will not allow me to post links, pictures, or Cyrillic characters. The product came from Russia and in English its name would be Moment Marathon (brand) waterproof glue. You can find it on Google if you search for the name. The tube is yellow and brown with a red cap, and it has a picture of a shoe on the front.
I hope this helps somebody.