Every material has its own idiosyncrasies. But the most forgiving bike material is probably aluminum. For some reasons paint seems to stick to aluminum frames really well whereas it chips off of steel frames if you look at it too hard. The funny thing is- on cars, it is the opposite. Just look at any old Land Rover. Aluminum frames are also a lot less prone to small dents and dings as steel frame. And to state the obvious- aluminum doesn't rust and steel does.
The most worrisome material is probably a carbon-aluminum glue jointed frame. The thoughts of galvanic corrosion, glue failure, headtube overtightening and aluminum lug cracking are never far from your mind.