Understanding AI Agent Skills and Knowledge Base Integration | Lindy Community

Understanding AI Agent Skills and Knowledge Base Integration

how do AI Agent skills work? I have an AI Agent step with the knowledge base skill, but how do I give it the documents I want it to search? I've tried setting up an unconnected knowledge base step loaded with the documents, but the AI agent doesn't seem to be able to search it. Do I have to connect them? I don't want to tell the AI Agent what the flow is (that's the point of the agent, it decides what steps to take, right?) My goal here is to have an agent with access to a few different knowledge bases, plus some other skills like executing various code blocks, each useful for answering specific types of questions, and for the AI agent to be able to choose which to use to answer a user's questions in chat depending on the question. My current setup:

20 comments

Aayush M.
·
·
You would need to connect the knowledge base with the AI agent
Jason R.
·
·
so add the knowledge base as a step after the agent? would that not tell the agent it's supposed to flow from itself to that next step (which isn't exactly what I want, I don't want to prescribe the flow)? What if I have multiple knowledge bases each on different topics?
Aayush M.
·
·
Nope just add it as a skill if you have multiple KBs In the AI agent just define what you want it to do
Jason R.
·
·
right, so I did that, the issue is I can't actually add documents there, there's nothing to click on when you add the skill to populate documents. when I test the agent, it does a search but it returns no results, because there are no documents.
Jason R.
·
·
similar issue with "run code" -- I need to give it code to run, but I can't in that box
Aayush M.
·
·
Yeah so think of skill as what tools you want to give access to. If you just want to search in your knowledge base, add a knowledge base step, upload your docs and in your query you can add something, If you have multiple docs, and you want a specific doc to be searched based on the input, after this knowledge base step add a AI agent with knowledge base as a skill and write in the prompt what knowledge to search when
Jason R.
·
·
I'm still not quite understanding how the agent is connected with specific tools. What I want is the agent to have access to:
the code tool, to run a specific code block
the knowledge base tool for a series of docs about subject A
the knowledge base tool for a series of docs about B
I don't want to proscribe the flow, so it's not Message received > search > ai agent. I want it to be message received > ai agent decides which of those three tools to use based on the messages so far So given that, where do I connect up the actual knowledge base step? If I put it before the AI agent after message received, my understanding is the search will happen after the message, then we get to the agent. That's not necessarily the right order. If I put it after, then the agent decides but has to choose an exit to get to the knowledge base which isn't quite right. I want the agent to perform potentially multiple steps, like running code and multiple searches across many knowledge bases, and responding to user questions or asking new ones to the user. So there's not a set flow and exits here. I thought that's what the AI agent was supposed to be able to do?
Aayush M.
·
·
you should then start with 1 main agent - that will decide what agent to pass on the task to:
Aayush M.
·
·
i agent can't run 3-4 tasks it might not do a good job
Aayush M.
·
·
1**
Jason R.
·
·
ok, I was kind of trying to avoid the branching with the AI agent, I thought that was what it was for. I have a lindy set up with a ton of branching, but since the flows aren't necessarily in the same order, it gets complicated to handle all eventualities. It works, but I thought I could radically simplify with the agent step. Given the above, what is the agent for, vs a normal conditional?
Aayush M.
·
·
This looks interesting my friend / though an AI agent might not be able to figure out what you wanted to do above. The branching can get messy / but this looks super messy 😂 What are you trying to solve? I'll see if I can help
Jason R.
·
·
sure! I'm trying to create a chatbot that can guide non technical users when implementing email authentication DNS records (DKIM, DMARC, and SPF). The agent should be able to do four basic tasks for the user:
1.
answer general questions about DKIM, DMARC, SPF, and email authentication (why it's important, how it works, etc...)
2.
give a user step by step instructions for how to generate these DNS records using whatever email sending system they're using (ex: mailchimp) and whatever domain name and/or from email address they plan on sending from
3.
give a user step by step instructions for how to install these DNS records after they generate them, using whatever DNS management system they use (typically their web host, ex: GoDaddy)
4.
validate the records have been installed correctly, and give final instructions for how to activate them in their email sending system
To answer general questions, the agent should have access to a knowledge base with documents that I provide on best practices, what these technologies are, why they're important, how they work, etc... To give step by step instructions for generating records, the agent should be able to ask the user for their email sending system and their domain and/or from email address, look up how to generate records from a knowledge base that has documents on how to generate these records on supported email sending systems, and supply personalized, step by step instructions back to the user. The agent should reject questions about sending systems it has no knowledge of. To give step by step instructions on how to install the DNS records on the DNS system, the agent needs to know the user's DNS system, as well as what records they were given by the email sending system to install. It would then look up instructions in that knowledge base, containing documents on how to install DNS records on common DNS systems, and provide personalized, step by step instructions on how to install them. To verify records have been installed correctly and give followup instructions, the agent needs to know which email sending system is being used, as well as what records should be validated. It needs to be able to call a code tool with the correct arguments to make DNS calls to validate the records are installed correctly, give understandable responses if they aren't (and offer to give instructions for fixing it, back to option 3), and give followup instructions based on a search of the knowledge base of email sending systems for how to turn on validated records in that sending system (there's typically a "validate" button you have to click in the email sending system as the final step) All of these paths except for maybe the first one require some prerequisite knowledge from the user. You can't give instructions for how to install records without knowing what records you want to install and which DNS system you're using, for example. And the path's aren't necessarily linear. A user might start by generating records, then installing, then validating, but if something goes wrong you might go back to installation or generation, etc... My complex attempt above accounts for these various paths and loopbacks, but it is, as you see, very complex. I was hoping the AI Agent step could reduce that complexity. Instead of having to proscribe each path and all the branches off each path to other paths, I could have an agent, with a prompt describing the four things it can do, and tools it can pick up to help. The agent would read the user's query, decide on a tool to use (search the knowledge base of email senders about DNS generation, or run the code tool to validate DNS records, etc...) and ask followup questions where necessary (what is your DNS host? etc...)
Malcolm C.
·
·
As an aside, Zapier agents have no problem with being given as many tools as you want, and then choosing what to do with them, the idea of branching for each tool seems nuts. (Of course, Zapier agents have a lot of problems too, which is why I’m hanging out here!)
Jason R.
·
·
yeah, I really don't get how tool use works in AI Agents in Lindy. For example, I just made this. The idea being if the agent thinks it should search the knowledge base it can follow that exit, do the search, and then it loops back to the agent again. But it doesn't really work. What happens in my test is the agent thinks to itself and uses the internal knowledge base tool, returns no results, apologizes to the user for no results, chooses the knowledge base exit, gets results, then replies again with the correct answer:
I apologize, but I'm currently unable to find the specific instructions for generating DNS records in Action Network. While Action Network is one of our supported platforms, I'm experiencing an issue accessing the detailed instructions. Would you like me to:
1.
Provide general information about email authentication records?
2.
Try searching again with different parameters?
3.
Wait for our system to be updated with the Action Network instructions?
Please let me know how you'd like to proceed. Here are the steps to generate your DNS records for Action Network with theseminal.com:
1.
Email support@actionnetwork.org and provide them with your reply-to email address that uses theseminal.com
2.
Action Network will send your domain to SendGrid (their email delivery service)
3.
SendGrid will generate your public key, which will include both DKIM and SPF records
4.
Action Network's support team will send these records to you
5.
Since your domain is theseminal.com, they will also generate DMARC records for you at the same time (this is required for lists over 5,000 subscribers)
Once you receive the records from Action Network's support team, I can help you with the installation process. Just share the records they send you, and I'll guide you through the next steps.
Malcolm C.
·
·
Doh! One of my laughs of the week was when one poster in this community mentioned that their agents were getting stuck in a loop handing things back and forth! We’re in a wild world and just wish agents were a mature thing already! Reminds me of Zapier circa 2014…
Jason R.
·
·
my big complex one above actually works! somehow. But I think I could get this simpler one to work if you could set the parameters of the tools you give the AI agents. Seems odd you can't, like how else can/would they even work?
Jason R.
·
·
Aayush M. any thoughts here? or anyone from Lindy who can explain how AI agents with a knowledge base tool actually work?
Aayush M.
·
·
Jason R. hey man, sorry I will check on the weekend and let you know?
Jason R.
·
·
no worries, thanks!