Back when some spices were worth more than their weight in gold there were stories to justify this, probably spread by the spice merchants themselves. For example cinnamon was said to stolen at great risk from the nests of giant eagles. In fact there was a very long chain of middlemen with a sellers market at most stops and the spices were farmed or gathered in the ordinary way. And lots of people, albeit mostly people a long way from Europe, knew this.
So, any other good stories about the origins of spices?
What should magi find if they investigate where their luxuries come from?
You'll just have to wait for The Cradle and the Crescent, eh? 
Lachie
While I appreciate sourcebooks they shouldn't stop us thinking for ourselves, eh? Especially before they are out. Anyway, many spices come from further east than that book will cover - I expect there will be wonders in the markets of Palmyra and Samarkand and terrors on the road between but as for the sources of spice and silk, well, it's still rumours and travelers' tales.
It's not the most medieval of sources but I found this book interesting:
amazon.com/Scent-Trail-Woman ... 760&sr=8-1
It concentrates on scents/perfumes but the snippets of history and ideas may be useful.
Lachie