That is a pretty good advertisement for drilling holes.
The fact you still have gutters is proof that they should be there.
That's not to say you haven't made a point. They need to be sealed and painted to avoid moisture entering between panels.