I had a carbon post that was glossy with decals. I didn't like how it looked on my matte frame and did not like the color of the decals, so I got light sandpaper and wet sanded it down to remove the decals and gloss. I then gave it a quick respray of clear matte paint.
It was fine. There was ample clear coat on there, so I never got to the actual carbon fiber.
Your frame probably has several layers of paint, but in sanding it, you may end up taking off all layers of paint, exposing the carbon. That isn't necessarily bad or dangerous, but it might not look good.