Improving Phone Agents: Transitioning to Lindy for Customization

Hi! I'm building phone agents with Retell and want to switch over to Lindy so I can better customize the agents. My test calls so far are not impressive as the voice agent has long (4-5 second) delays before responding. that has been overcome with many voice agents out there today. i choose different models of LLM and I dont see the difference. DO i have to purchase a higher upgrade before even trying the phone agent to see if it is fast enough?