Ok, the attachments can be deleted.
However, you have to delete them from the attachments page, and they seem to automatically disappear from the message.
The attachment will automatically be removed from the message.
However, I agree that this would be better managed through the editor.