Well, it raises the point of the voice and gesture roles in the act of doing magic.
Facts:
With the exception of Range: Voice spell, the sounds or the voice of the magi has nothing to do to where the target is affected (Arcane connection and Sight range allows to affect target which are out of Voice range). So we can rule out that Voice carries the magical power to the target.
Since spell can be casted in many various languages with some minor breakthrough, it is not linked to the word itself, so it is not specific words of power (calling Enochean or Adamic or something else).
Finally, some mages are gifted with Silent magic, prooving that casting words are not a requirements.
Based on these three observations, I would see that words and gestures are more techniques to assist the mage in harnessing, focusssing and controlling magic to obtain a certain effect. So they are not need to make magic, but needed by the mage to help him obtain a predictible result.
If we consider gesture and words as mental and physical exercise to channel the magical energy, as long as they are performed in one way or another, it should not hinder the mage. However, if the mage does not "feel" his movements (through a PeIm: Veil of Touch) or does not hear his mantra (PeIm: Veil of Silence or Captive Voice), I don't think he can benefit from them. Hence, I would say he has the -10 for Voice, but not the -5 for gesture: he is invisible, but he feels his movement which help him centers his magical power.
One way to go around would be to design a small enchanted hearing: Voice from the Closed lab (InIm, Arc range, Hearing), which will allow the mage to hear what is happening in his lab, where he could have left the pouch with is voice trapped in. Then I probably would allow silent casting without penalty, or maybe only a -3 for the strange effect of hearing really his voice and not feeling it through his skulls (which is why people don't recognize their own recorded voice).
After a season of practice, I would completely wave off the penalty as the magus would be used hearing his real voice.