Eh, the other way around. I'm showing that with this invisibility bit, allowing the pink dot defense does not truly protect you from the invisible attacker. However, disallowing the pink dot defense does stop the abuse of turning attacks invisible to prevent them from harming you. I'm showing that with the pink dot defense in place, the clever invisibility person can use invisibility to essentially make themselves invulnerable and make their attacks nearly impossible to defend against.
That was how I'd originally read it way back when, but I was told that was not what was meant so many times. Those who said so said the intent of that comment was that you can't do things like using non-bold gestures, non-firm voice, forceless casting, etc. I'm still not positive. However, in the specific case of multiple casting, I really cannot provide a valid argument against their argument. The reason is that you're not altering that spell, you're just doing it a bunch of times, so any rule about options to alter that spell doesn't really apply. I could provide an argument against Quiet Casting and Still Casting since they're actually about options when casting that spell. So, though I had originally thought it would be like you said, I still can't find RAW to actually support Multiple Casting not being allowed with Fast Casting.