From what I can tell, nothing caps at 0. If a penalty takes something below zero, it goes below zero.
A zero cap only applies if a negative total makes no sense, for example you can't roll -3 botch dice.
Spell penetration can absolutely go below zero. It is entirely reasonable to successfully cast a formulaic spell with -9 penetration. A magic resistance of 0 will stop it. Heck, a magic resistance of -5 will stop it. But it will still work on someone who has no magic resistance because zero magic resistance and no magic resistance aren't the same thing.
So yes, wound penalties modify rolled soak - as everyone else has said.
Incidentally, if you're wondering how to get a MR of -5 :
Parma 1, relevant art at 8, standing in a divine aura of 6. Divine aura (according to realm interaction chart) has a penalty of -18 to all supernatural rolls and totals - and Parma-based magic resistance is a supernatural total. Base MR in this case is 13, which you subtract the divine aura penalty from for a MR total of -5. Of course, anyone casting spells at you who is also in said divine aura also has a -18 penalty - assuming they're also using magic-realm powers.
Of course if they're standing outside the divine aura then you're pretty much screwed... unless you're carrying a relic.
I'm a bit unsure what you mean by "a negative damage roll" really?
Remember that damage is not rolled seperately from the attack for combat damage, and as for spells, the wound penalty goes on the casting roll, not the damage roll (since it's technically not the magus doing the damage).
But the thing to remember (and this may seem slightly abstract) is that Ars Magica uses an arbitrary baseline for damage. Meaning that you can add any arbitrary number to damage and soak, and it will not change anything as long as you remember to add to both damage and soak.
If negative damage bothers you, add +50 to all damage and all soaks. Since you will add 50 to the damage and then subtract soak (which is 50 higher) it will not change anything. And then you just use the damage table on p. 179 like always.
Ofcourse, if you mean negative total damage (that is, post soak), it means you're not inflicting any kind of wound.
When Fallout lost the GURPS license, they redid the core game mechanics based on the pen-and-paper RPG a whole lot of them were playing in the office (Ars Magica). However, they found that using a stat range from -5 to + 5 in their SPECIAL system confused a lot of their test audience, so they just added 5 to everything.
