Sierra Madre to Baldy Village is 3,500ft over 22 miles. It's not exactly the toughest hill around. I don't know anything about your climbing history or ability, but I do know that I'm not the strongest climber out there (not by a long shot)-- and last 4th of July I rode 40 miles from home, climbed GMR, and rode 40 miles home. About half of GRR is rollers, so it's a bit easier than other local climbs of similar length.
__________________
