If you've already set the Dialogue Manager GameObject's Other Settings > Warm Up Conversation Controller to On or Extra and you're still seeing a performance spike in the first conversation, then run the Profiler in Deep Profile mode and look for this possible culprit:
This is caused by Unity preparing/handling a dynamic font for one or more Text components.
If possible, switch to TextMesh Pro in this case.
If you must continue with Text components, see this search result for solutions: https://www.google.com/search?q=unity+cachefontfortext