It was before the roll in Feng Shui 1, but the "rules as written" in Feng Shui 2 implies after the swerve but before success/failure is declared is valid.
Here's the relevant past thread, which lacks any official responses:
https://forum.atlas-games.com/t/rolling-the-fortune-die/10153/1
Essentially the read it gives is that you add the fortune die to the swerve, implying that the swerve is already rolled. No where does it state you must choose to roll the fortune with the swerve. The examples given in the text at not especially clear on the subject.