Recently, Meta released the Llama 3 and 3.1 families of models, including an 8B, 70B, and 405B parameter model. The 3.1 model natively supports tool calling, making it one of the best open-source large language models for agentic automation.
This article will discuss tool-calling in LLMs and how to use Groq’s Llama 3.1 tool-calling feature to build capable AI agents.
In this article, we cover
The basics of tool calling.
How to use Llama 3.1 on Groq Cloud for tool calling.
How to use Composio tools with LlamaIndex to build a research agent.
But before that, let’s understand what even tool calling is.
Note: We will use the terms tool calling and function calling interchangeably. As they more or less refer to the same concept.
What is Tool Calling?
Tools are functions that allow large language models (LLMs) to interact with external applications. They provide an interface for LLMs to access and retrieve information from outside sources.
Despite the name, LLMs don't directly call tools themselves. Instead, when they determine that a request requires a tool, they generate a structured schema specifying the appropriate tool and the necessary parameters.
For instance, assume an LLM can access internet browsing tools that accept a text parameter. So, when a user asks it to fetch information about a recent event, the LLM, instead of generating texts, will now generate a JSON schema of the Internet tool, which you can call the function/tool.
Here is a schematic diagram that showcases how tool calling works.
What is Groq?
Groq is an AI inference provider that hosts open-source models such as Mistral, Gemma, Whisper, and Llama. They are the industry-leading platform for offering the fastest AI inference, which makes it ideal for building capable agentic systems.
This article will showcase how you can use the Groq cloud to build AI agents using Composio tools.
Create Groq API key
You will need an API key to use the LLMs hosted by Groq Cloud. So, go to Groq Cloud and create one.
Copy the API key and save it somewhere secure.
Llama 3 Tool Calling in Groq
Let’s explore how to use a tool called Llama 3 Groq.
Begin by installing Groq. pip.
Set the Groq API key as an environment variable.
Now, create an instance of Groq with Llama 3.1 model.
Groq also has a fine-tuned Llama 3 model ('llama3-groq-70b-8192-tool-use-preview') for tool-calling, which performs on par with GPT-4o with some trade-off with reasoning.
Next, define a simple Calculator function to demonstrate how the Llama 3 tool calling works.
Create a schema for passing the function to the Llama 3 model.
In the above code block, the calculator tool is defined as a function within a structured schema to be passed to the Llama 3 model.
The tool is specified as a function with the name.
"calculate".The tool's purpose is to evaluate a mathematical expression.
The
"parameters"section outlines the expected input:It expects an object with a property called
"expression".The
"expression"parameter is of type string and represents the mathematical expression to be evaluated.
The
"required"array indicates that the"expression"parameter is mandatory for the tool to operate.
Now, let’s make an API call with a prompt and tool to the model.
When it sees an appropriate request for a tool call, the model will generate a response containing the tool to be called and the required parameters.
You can observe that we received a response for tool calling with the tool's name and its parameter.
Now, we can call the actual function using this.
Finally, we iterate over the tool calls. In this case, we only have a single tool response. We call the respective functions, append them to the messages, and send them to the model for the final response.
Tool Calling in Building Agents
Tool calling features played a pivotal role in unlocking LLMs' agentic potential. Complex tools that wrap API services can automate real-world jobs, such as searching content over the Internet, crawling web pages, interacting with databases, etc.
Now, let’s see how to use Tavily—a search engine optimized for LLMs and RAG, to provide the model with the latest world affairs.
We will use Composio’s Tavily integration for this.
Install Composio on your system.
Groq’s API endpoints are fully compatible with OpenAI, so we can use Composio’s OpenAI plugin to handle Groq's responses.
Go to Tavily's website and create an API key.
Now, add Tavily integration with Composio.
Complete the authentication by providing the Tavily API key when prompted on the terminal.
Define Composio tools and instantiate Groq client.
Now, define the system prompt and user prompt.
Next, define a ChatCompletion request using the prompts, tools, and model.
Extract the required information from the response, such as the content, thread ID, and source URL.
Pass it back to the LLM for generating an answer.
That’s all; that is how you can leverage Composio tools and Llama 3’s tool-calling ability to create better AI assistants.
Building Agents with LlamaIndex
We saw how to use Llama 3 from Groq for text completion and tool calling and also built a small research agent. However, you can make even more complicated agents with frameworks like LlamaIndex.
LlamaIndex abstracts handling responses from LLMs so that you can build agents faster.
Let’s see an example of how to use LlamaIndex’s FunctionCallingAgents.
Begin by installing Composio’s LlamaIndex plugin and LlamaIndex’s Groq extension.
Import libraries and define search tools using Tavily.
Create a Groq instance and define a prefix system message.
Finally, define the agent and execute it with human input.
With this, you have created an internet search agent using Llama 3’s tools calling ability.
Putting everything together.
Next Steps
In this article, you learnt the fundamentals of tool-calling and how to implement using Composio tools and Groq’s Llama 3 to build simple agents.
However, with Composio's extensive collection of tools, you can create even more sophisticated agents. Be sure to read our blog post on building a software engineering agent that can automatically write code for you and explore our examples to discover more innovative agents. Discover the effectiveness of Meta Llama 3.2's vision performance.