Regarding the base effect, I think the base effect that Gift of Vigor is built on is the conveniently vague "Direct the flow of bodily energy" (compare to the first line of the spell description).
Otherwise I agree with you.
I definitely would allow Gift of Vigor to be enchanted into the familiar bond (either way) and controlled by the other member of the pair. It just makes so much sense from a story perspective!
I agree with gerg. I would point out that the ability to transfer fatigue between magus and familiar was a standard familiar power in previous editions (where Gift of Vigor, with the same "this is the closest..." fluff, was already present).