This one (for the transmutation issue). While I can't come up with an explicit rules quotation to show you, there are examples of multiple spells being active upon the same target at the same time. Mutual exclusivity can be clear but it isn't always.
There is the detail regarding the different ways that people change in to animals in the Bjornaer chapter of HoH: Mystery Cults. A person who has been changed in to an animal is now susceptible to both corpus and animal spells. If the authors had intended any complications to arise if that former person was now targeted with a muto animal spell to turn them in to a third thing they would have mentioned it in this discussion.
I think that we won't find a rules quote because this isn't a complication that has been added to the game. If one thinks from the perspective of "this spell has this effect" then you don't necessarily think to deal with the question of which spell works, they both work.
In matters where two spells work against one another there are some specific guidelines in the discussion of counterspells. My take is that in most instances, like two rego spells pulling a target in different directions, the more powerful spell should probably "win" but both spells will take effect.
My usual logic cares about the most recently cast spell... I may be off base, as usual.
Greg the Grog is turned into a cow for duration moon. During that time, someone casts a spell to turn the cow into a canary for sun duration. Well, Greg is already turned into a cow, rather than being CONSTANTLY changed into a cow. The spell isn't re-acting on him, so he stays a canary for sun duration, before reverting to a cow. There's nothing to trigger turning him from canary back into a cow, even if the spell is more powerful, et cetera.
Specifically I'm using the principal to "spot dispell" bloodline target transformations. For if I still want to use my houd-men as hounds or my horse-men as horses. I can turn one of them into a quadruped with a spell while leaving the bloodline enchantment in place and not affecting the others.
Spell effects don't have magic resistance by themselves. A being affected by spell A resists attempts to manipulate A's effect by magic with its own MR, though. See for both ArM5 p.85f Magic Resistance.
Certain spells can only target magical effects up to a given magnitude. See for this ArM5 p.156ff Vim Spells, especially the guidelines in it.
But otherwise the effect of spell A does not resist or undo an effect of spell B to 'overwrite' it on its target. It is also not dispelled by B, unless B's spell guideline or spell description says so.
So if B should cease to affect the target, because
it's duration expires,
it is dispelled,
A affects the target again.
Should the person who caused effect A take up the control of A on the target again after B was cast (see ArM5 p.111 Ranges), he also can override the effect of B on it without casting a new spell.
If the two magi who have cast spell A and B both can and do continue to control the respective effects, this is a moment where Finesse and Concentration rolls decide, just which controlled spell effect takes the upper hand for a moment.
We may then get a situation like those, which finally spawned more refined ways of magical contest like Apprentices p.57f Bellum and ArM5 p.89f Certamen.