Just like anything used for safety - inspect it once in a while. If you see cracks, delamination, breakdown in the structure, throw it out. If it's only funky helmet pads, just replace them. And wash that helmet once in a while with mild detergent. I have 10yr old helmets in near perfect condition which I don't hesitate to wear.
The only cardinal rule I follow is if I find damage to a helmet or have crashed and the helmet took a hit - I destroy it before tossing it in the trash.