Be polite and ask... it's not hard.
Just the other day I was about 4 miles from home after a 45 mile ride. I was on a MUP and came up on a rather large and athletic person and jumped on his wheel. He knew I was there and picked up his speed, at that point I asked if it was OK to sit in his draft. I told him I would stay back enough and not attempt to pass. He was cool with it. After 3 miles I came to a crossing where I get off the MUP and said "i'm dropping off, thanks for the draft". He wave and continued on.
BTW, he was wearing a NYPD cycling team jersey.