Another approach - make a 2nd roller frame that sits behind the rear of the rollers and mount on it a big flywheel belted also to the rear roller. (You might be able to run both belts on the same wheel or put a 2nd wheel on the other side.) With this, you could have a flywheel big enough to work and up off the floor to accommodate the big diameter. The final setup would be kinda like a tender behind the locomotive. With a belted flywheel, you also get to gear up the flywheel so it spins faster. (Inertia goes by the square of the radius/diameter but in direct proportion to the rotation speed. With the right flywheel and gearing, you could get exactly you and your bike's worth of inertia for real road speed. "Hang on ma, I'm going 40. It's going to take me a while to stop.")
If the flywheel was big enough, braking would tend to push you back and you might drop off the rollers. It might be worth considering how you would crash if that happened.
Ben