Same here. We force the character to take some effort to change specialty, though. If you have an astronomy specialization in artes liberales you are not going to get a change to Trivium unless you actively pursue Trivium studies when earning XP in that field.
On the one hand, i barely use specialisations, on the other hand i would like to have the option for multiple ones...
And the result of that wish is mostly a lot of attempts to draw up rules for it that simply dont work well at all.
IIRC, the most interesting i could come up with was allowing characters to spend 5 XP(possibly requiring a Minor Virtue as well, i´ve forgotten) to get an additional Ability at Score 0 that only covers a specialisation. It works at least halfway to decent, but it´s still open to some abuse if SG isnt careful. If used for a super useful specialisation it can increase the final score far too much, at the same time, most are not quite that good so it becomes a hard choice wether to increase the parent skill or a specialisation, and like that, it´s a fairly good option.
Another workable variation was that you could spend XP like in an Ability to add more specialisations like the usual ones to an Ability. So if you placed 15XP you could get an extra specialisation or two, depending on how you count the original one or not. This one is almost impossible to really abuse and still useful so it´s probably the best one overall.
I would add that the player should give an in-game justification as to why the new specialisation was appropriate. This would usually be related to the source of some of the XP received since the last level, or what the character had been doing with the Ability recently, or possibly what other Abilities or Arts have increased recently. For example, if the character has recently increased his Craft: Swordsmith Ability, it would fine, I think, when his Area Lore Ability increases to change the Area Lore specialisation to swordsmiths.
It's not a big deal, so I wouldn't be too picky as to precisely what the justification was --- just has to be a bit plausible.