I saw a comically small smartphone on Amazon. Like 2.5" or something. I think it's got battery for standby for only about 4-6 hours.
If your network uses SIM cards, just toss yours into the phone before the ride. Considering the battery life, you'd need to keep it turned off unless you need it.
Main advantage is this is a phone that can access maps, email, and social media. I've been in a situation where I was somewhere without my phone. I had my wife's number but she wasn't answering. Had I had access to a smart phone I could have looked at an email with directions or found someone on social media who knew where to meet.
The disadvantage other than the battery is a crap camera.
You can also buy Zoolander flip phones on Amazon for cheap.