That is a really fascinating idea and a lot of useful stuff has already been said, but I still feel that my two cents can be helpful.
It was recently discovered that the neo-babylonians managed to do calculations with infinitesimal numbers in about 500 BCE in order to describe the orbit of Jupiter. Historically the babylonian language was not deciphered until late 1800's, but that need not stop you. In 500 BCE the babylonian language was used by scholars to write things down but they mostly spoke aramaic, which was never lost to time. Perhaps your players could find an aramaic text describing mathematics with infinitesimals?
There is a palimpsest (of greek/eastern roman origin) with a text written by some greek mathematician, I think it was Archimedes, that was deciphered in after year 2000 by using some advanced techniques to read the underlying text. In that text they also found some mention of use of infinitesimals when it comes to figuring out the area of a geometric shape, something which is not very far from modern intergrals. It is possible that copies of this otherwise lost text could exist in the middle ages but not survive into modern times. Generally there are a lot of lost works, especially greek ones, where we know that they once existed because we have other preserved ancient texts that reference them. It is entirely possible that these works had not been destroyed in the medieval era but were simply rotting away in some collection ready to be found by your players.
Newton based a lot of his calculus on the description of how objects move over time and the required observations aren't all that hard to do. They are bothersome and time consuming and it takes some intellectual skill to come up with the idea of doing it in the first place. You could have your players analyze the movement of celestial objects, ballistics, etc. and make insight rolls to simulate the process of hitting on the right idea for analysis.
Then there is integration. Traditionally in modern schools we teach differentiation first and integration after. However you dont have to start calculus off with differentiation, and there are a lot of naturally occurring reasons to want to integrate. Most of these reasons center around wanting to determine the surface area of some geometric object. The good thing about this is that complicated shapes are quite easy to get (you can simply draw them) and it is similarly quite easy to start at the problem of determining their area with infinitesimals. You can also do three dimensional integration but it is a lot harder to wrap your head around. Of course Imaginem magic can make it a lot easier to visualize 3D objects (literally) and thus to describe their volumes.
I would argue that a system of mathematics that has the number 0 and a concept of infinity is a requirement for the invention of calculus and for me that is enough of a reason to consider the discovery/invention of both the be breakthrough-point yielding steps on the road to the discovery of calculus. This way you can also start the story-line off by having your players discover, or better yet set out to find a book or teacher to introduce them to the concept of zero. Later you can follow that up with having them learn about or invent infinity.
I would simulate this process by having them do experimental labwork in natural magic, that is experimental philosophy as described in A&A. I would probably invent some discoveries to be made within a natural magic equivalent to Philosophiae. This experimental Artes liberales could do things like:
- construct buildings by inventing a superior method of construction saving money on building costs.
- Make better astrological predictions to get bonuses to lab totals.
- Improve the use of siege weaponry and archery by improved ballistics (and perhaps aimed spells).
- Make a superior system for keeping track of covenant finances to minimize losses.
- Use the superior covenant finance system to find out where the Pound of Enumerus goes.
You can probably come up with some more.
Since this breakthrough is (at least in the real world) entirely unmagical I would treat the process as something that can be done without magic. However I would also find ways to let magic help. Intellego is the obvious candidate as it allows you to know information precisely even when that information would otherwise be difficult to get to know. Creo is another good candidate especially combined with imaginem as you can easily conjure up complicated shapes and overlay images of rulers, boxes etc. to help in measuring your subject matter. Additionally it is much easier to magically conjure say, a line segment that is precisely 1 inch long than it is to create that line segment without magic.