Hi,
I can only speak as a watcher from the outside, rather than a master, secret or otherwise.
The idea that there are limits to Hermetic Magic appears very early. IMO, this is mostly a reaction to D&D. No easy healing in AM! No arbitrary creation! D&D creates an arbitrary distinction between divine and wizardly magic, but AM shall do this better! We want our wizards to be powerful, but not all-powerful! Naturally, AM5 is clearest about these limitations, and their nature, because that's what AM5 does.
Twilight does not exist in the 2nd Edition core rules. This was quickly added, however, and never went away. AM5 softens its impact.
Already present in AM2 core: Botches. Covenants. Houses, pretty much as we know them today, though sometimes with some tweaks. (Bjornaer have Heartbeasts, for example, but Shapeshifters are not enemies.) Grogs, companions, troupes. Demons were utterly evil. Parma existed, but smothered one's ability to perceive the wonder of the world.
Faerie Magic is ill-defined until AM5, but was generally interpreted as ranging between 'weird' and 'natural', because that's what the cool kids were all about during the 90s.
Longevity Potions were potions, and have increased in effectiveness over the years. And have gotten much cheaper.
Warping is all-new for AM5. AM5 also makes explicit that human nature is mortal, and to become immortal is to become less; this fits the milieu but had not been explicit. But Decrepitude has been around as long as I have.
Limits on Art scores have loosened across editions, consistently; a score above 20 was once extraordinary.
Game balance is a secondary issue, I think: A game can be balanced at many power levels. Also, the solutions that work (or do not) for AM's Mythic Europe do not necessarily apply to Mythic Enlightenment Europe! You might do better absolutely ignoring AM and spend time reading, say, Strange&Norrell (later than your period by a bit). For me, AM started out as yet another D&D 'heartbreaker': A better D&D, with a real medieval setting and without having to make excuses for powerful wizards. You might want to start with that... and you might not.
BTW, I also second (third?) the old OoH book. Atmospheric.
Anyway,
Ken