ps, Im sure you've figured out by now that jeans are a complete no go. Heavy as heck, bulky, wont dry for ages.
"Off" bike clothes are easy to be compact and light. There are light t shirts that dry quickly, and I even have a button down collar short sleeve shirt that is my to go travel shirt. Light material and like pants, is compactly rollable, and dries quickly after hand washes. Bonus is that along with long pants, I can look totally non "bikey"
my long pants and this shirt is all I need for off bike wear, and both weigh so little its great, and compact.
I do also prefer riding with tights if its cool, rather than using my Royal Robbins pants, which would need a strap for the chainring side not to get on chain, plus I prefer tokeep them clean and only for off bike, so I do bring tights too.
bottom line is that its really easy to have a fairly light set of clothes, and compact which is always going to be an advantage for your unicycle reality.