The key issue is to choose the books that will serve you best. If you want to run a game on the cheap, I'd recommend sticking to fourth edition, and would furthermore consider using PDFs as they're cheaper. e23 allows you to get the 4e core rules for FREE, and access to many of ArM's books for relatively little money (for example, Covenants costs 15$ as a PDF, but as an MSRP of 30$). 4e books are also more likely to be found in bargain bins or used on eBay and so on.
Which books to purchase is, again, a matter of what kind of saga you'd like to play and what would benefit you. You shouldn't limit yourself to 4th Edition books even if you do play 4e - the bulk of the information can be converted fairly easily, and 5e has superior supplements overall.
Here are some recommendations. All are excellent so you should choose which to purchase based on the content.
Covenants: Excellent for the rules on customizing laboratories, which can really add variability and things-to-do for your magus' time in the lab, as well as for the rules on how to design a Covenant, which can really springboard ideas for the saga and present lots of options. It also contains detailed rules for running the covenant poltiicaly and economically, and rules for physical book quality and casting from text.
Houses of Hermes: True Lineages: If you want insight into how Hermetic Law operates, this is the best source. It also provides tools for Mercere trade (of raw vis and so on). Otherwise, unless you are particularly interested in House Bonisagus or Tremere, I wouldn't get this book - it presents an excellent version of House Tremere, but you can do without it; it presents rules for Original Research, but these are problematic and you would be better off creating House Rules off the rules of Anicent Magic, which are available for free here.
Houses of Hermes: Mystery Cults: Unlike the other HoH series, this book provides crunch that can IMHO greatly improve the saga even if you don't play these specific cults. You can pick up the strange powers and ideas and run with them to make cool NPCs, secretive cults in opposition to the PCs, and so on. I think the Merinitia section is especially useful there, presenting a host of interesting and very varied philosophies, societies, and exotic powers. As a bonus, you get excellent descriptions of four Houses, although I actually have issues with most.
The Mysteries: You can get the original The Mysteries for 4e; it is more colorful and flavorful but very poorly balanced compared to the 5e version (The Mysteries Revised Edition). Whichever version you choose, you'll receive lots of secretive societies and strange powers to brew adventures around. I don't know whether it's better to pick this one or Mystery Cults on a budget; if you are not particularly interested in the House descriptions, I think TM or TMRE is better as it presents more off-the-wall options.
Realms of Power: The Infernal: You can use it as a source of Infernal opposition, so if you like this type of story it's invaluable. I actually am not very impressed with its usability in-saga so far, but I guess I just never got into it.
The only take on Faeries out there that's detailed to some extent is Faeries: Revised Edition, for 4e. I personally found it to be too full of prose, and not very interesting. It does, however, present an interesting take on Faerie and several faerie creatures, so I'd strongly consider it for a 4e saga on a shoestring, if you're into faeries.
I wouldn't purchase Guardians of the Forest if I were on a budget given that you have The Dragon and the Bear; both are excellent tribunal books, but one tribunal book is enough.
Likewise, Ancient Magic is an excellent book, but essentially will provide you with one adventure framework so is rather expensive. Unless you engage in several, not-connected, "seeker" adventures (i.e. adventures seeking ancient magical secrets) - if that sounds like a saga you want to run, AM would be an excellent buy IMHO, as it provides both the mechanics and several excellent and interesting saga ideas, complete with minisettings, key NPCs, and whatnot.