Interesting. (Good work!)
I would assume it's a bit like having a stone door (or a metal one) - inarguably stronger than any oak, yet how often do magi employ them?
("A stone door, sodales? A bit overdone, isn't it?...") 8)
So I'd think that any stone towers would stick out like sore thumbs, and be the source of much rumour and gossip.
And your saga would never enjoy the differences that unexpected cultural quirks can provide. Generic is fine, but it is what it is.