You can't remove an annotation in the document viewer at this time.